Peer review, the long-standing custodian of good science, is now being seen by many as a cabalistic exercise that assures the self-interests of the scientific community. With accountability being the new mantra, Indian scientists can no longer afford to spend public money on third-rate science. Sadly enough, however, the Indian scientist is least inclined to revamp the "black box" for the sake of scientific excellence
First among equals
WITH the ivory towers of Indian science
Cracking up under economic pressure
And the winds I of liberalisation blowing
fircely into the cobwebbed windows of
the scientific edifice, the talk in scientif-
ft bes. all of a sudden, hinged on
hmm accountability and com-
e - words that had apparently
id to evist in the Indian scientist's lexicon.
Survival is the keyword now. And
a another name for survival.
dwindling iritty and more hands
pft ims iL the jungle of Indian sci
Misbowd to become all the more
odaim As research costs have
i have become increas
lp digpeadent on society for their
Wom The caw for supporting sci
at the heart of every bud
jVdL coancting scientific success
0-" d objectives for health, eco-
nomic compititiveness, defense, transportation agriculture and the like. But
what is rhe assurance that only the best science is financed.
To answer this question, one must
Must delve into a black box called the peer
Review system, the centuries-old way of
self-regulation practised by scientists.
The tradition of asking scientists of
repute to judge the quality of others'
work goes back over 400 years to the
birth of the Philosophical Transactions of
Britain's Royal Society. It is now tightly
woven into the fabric of scientific communities of all disciplines. To quote one
recent definition: "Peer review is an
organised method of evaluating scientific work which is used by scientists to
certify the correctness of procedures,
establish the plausibility of results, and
allocate resources (such as journal
space, research funds, recognition and
special honour)."
The question is, can peer review, at
the centre of the profession's claim to
autonomy and society's demand for
accountability, continue to serve as a
mechanism of financial allocation and
quality control? Furthermore, with
accountability being the new mantra, it
becomes pertinent to ask whether our
s&T funds are going to the best people
and the best projects, and whether they
are being harnessed to the urgent needs
of our economy and people.
There is growing concern among
scientists and policymakers about the
soundness of the peer review system in
the present scenario. In particular, the
allocation of research funds is an area of
decision-making where the peers
increasingly find others looking over
their shoulders, at least in the us and UK,
where the money involved runi into billions of dollars.
But can one ascribe the same set of
problems to peer review in India? To a
large extent, yes, though the scope is
widely different. In India, for instance,
'the majority of research funding is in
the form of intramural grants, money
which is given to universities and
research institutes as project money for
which, unfortunately, there is no peer
review. Says Dinesh Mohan, a professor
at the Indian Institute of Technology,
Delhi, "It is this money that one should
be looking at. The funds distributed by
the s&T funding agencies, called extramural or sponsored grants, are a miniscule part (about Rs 150 crore per
annum) of the total R&D budget (about
Rs 4,000 crore per annum). So why
should one even bother about peer
review when the amount involved is so
little?"
Cynicism of scientists like Mohan
notwithstanding, Rs 150 crore forone of
the poorest countries on earth is a large
sum of money and scientists cannot get
away by spending it haphazardly.
Moreover, does the peer review operate
only to evaluate merit or should it also
help establish priorities? Can it or
should it be effective in changing the
direction of a programme or in allocating resources among programmes within the agencies themselves? These questions are significant because they challenge the assumption that peer review is
the most effective way to allocate
resources in the best overall interests of
both science and society.
Judging from various accounts, it
appears the grants peer review in India
leaves a lot to be desired. The latest
department of science and technology
(DST) statistics on extramural grants lists
about 17 funding agencies, including
socioeconomic ministries like the ministry of human resource development,
accounting for about Rs 127 crore for
the year 1993-94. Of these, the DST and
the department of biotechnology (DST)
accounted for more than 60 per cent of
the total money distributed. The DST
and DST also happen to have "wellestablished" peer review mechanisms.
Of the rest, institutions like the
Defence Research and Development
Organisation, the department of atomic
energy, the Indian Council of Medical
Research (tcmk) and the Indian Council
of Agricultural Research have not-sorigorous systems of evaluating proposals; some others, such as the department
of coal, may or may not resort to peer
review for proposal evaluation.
To understand the problems faced
by grants peer review in the 1990s, it
would be instructive to look at what it
was like 25 years after independence
budgets were generally increasing often
rapidly, and there was much im c
petition for cash. Says P I LavallimiL
ToCrly With the DST, "The main ps
peer committees at the subject level was
quality control: weed out the sub-standard proposals and fund the rest. The
funding agencies were relatively comfurtable with a system which left the
details of the spending to the scientists."
Things are changing now. The idea
advanced by the British physicist and
science policy gum John Ziman - the
science funding has entered a steady
state is controversial. unlikely to grow at a rate faster tbm
economy. As scientific oppo
grow rapidly, competition for
has become fierce, within and
specialities and between individuals
research groups and institutions,.For
every winner in the peer review game
there are now several losers.
In addition, the Indian government
now appears to be more interested
accountability for the money it spends
on R&D than in preserving scientific
autonomy. Politicians now want more
say in what science will do. And
a move towards controlling "
expenditure through measures of
put and "performance indicators
emphasis in government policy
research with potential applications
on industrial exploitability all
unease. While it is being str
the importance of backing
research without strings artached to
becoming known, the suspicion still
lingers that decisions are shifting
towards short-term payoffs.
Remarks H Y Mohan Ram, a Proffe
sor in the botany department of Delhi
University, "All this reinforces the
ambivalence many scientists feel about
peer review. They want to defend it as a
my of leaving science to scientists and
as their bulwark against attempts to t0q
lor research projects to specific goals."
However, not many are happy with the
way grants peer review operates.
With operational problems of the
grants peer review much in evidence
does it play any role in influencing sci
ence policy and setting up priorities for
scientific research? The less the amount
of public money available to science, the
Water will be the desire to influence, if
not cowol. its distribution. Says S R
Valluri fanner director of the National
Aerospace Laboratory, "There has been
usil accountability in the expendi-
e an science through public funds,
possibly in space and atomic
whose plan budgets are mainly
ow-specific. It is this lack of
accountability that seems to have caused
the distortions in the practice of science India.
S Ysshpal. former University Grants
commossion chairperson, believes the
government will not hesitate to support
qpwimmes that are clearly seen to
ngotional purpose. "However," he
we took recourse mostly to the
bqdli -in-aid schemes for sup
Grant-in-aid is support
which a scientist wanted to
mquo% and it was only
@Pintoru'eiy
onme to him to carry
oft bis worit. Thus,
is no inherent
in such
grant except to the extent planned
by the scientist himself.
If the manner of determination of
Budgets of
MEN fiW Scientific
dkorol Research
OF 69 Planning
P" it any indi
a fink rela
nates wi thin
scientists mind
continues on paper
as bureaucratic
procedure and
ends behind the
dosed doors of the
funding agency.
Ir
esocenred
hoe lesult
Coliaction
b-ad projects which do not necessarily have clear input-output relations no nationally felt needs."
Provided by this background, the sci-
Entists seek funds for research conceived
By them . In turn, of the same
Scientific cimmunity are asked to
Thus become inward-looking. The system has
immud-looking. Says
Valluri, "It was not that the scientists'
intentions were not honourable. It
almost seems that instead of first defining national priorities and deriving
strategically targeted R&D programmes
out of them, they ended up taking the
viewpoint that whatever we wish to do
will be good for the country."
With too many projects chasing too
few funds, the demand far exceeded the
supply. This situation has led to the now
familiar manoeuvering by some scien-
tists to have access to and, if possible,
control or influence the distribution of
these limited funds. The leverage is
through membership in committees for
sanctioning of grants, awards and honours or even appointment of senior staff
in s&,r departments.
So much for the criticism of peer
review. But what actual evidence is there
to back them up? There is
very little systematic data.
Much of the criticism is
based on assertion, or
anecdotal evidence at best.
As Chubin and Hackett
concede in their book,
"Peer review is air intensely private process that
originates within a scientist's mind, continues on
paper as bureaucratic procedure and ends behind
the closed doors of the funding agency."
Says A R Rajeswari, joint advisor in
the DST, "There are no performance
indicators by which one can judge the
quality of grants peer review. Firstly,
very few funding agencies conduct studies to assess the output of the projects
sponsored by them. And assuming they
have information on these projects, they
are extremely reluctant to part with it.
For instance, a recent proposal to evaluate the performance of projects sponsored by the fcmR was shot down by the
scientists." She deplored the !act that
many scientists do not even furnish a
project report for years after the project
is over, making it difficult to assess the
accountability of many of these agencies.
Say Chubin and Hackett, "Peer
review leaves few clues in the public
domain, and many participants in the
system insist upon minimising public
access to information. For example, the
names of scientists who succeeded in
their quest for support are accessible,
but the names of those who did not, are
generally unavailable to independent
investigators."
Evidence is sorely lacking on peer
review practices, largely because the
reviews themselves, recorded in the files
of journals and funding agencies, can be
obtained only under assurances of strict
confidentiality. In India, there is no
facility like the Freedom of Information
Act as in the us, which allows scientists
and the public alike to request access to
verbatim copies of proposals that had
been supported by funding agencies.
The inherent difficulties of grants
peer review have been exacerbated in
recent years by budget constraints and
by turther bureaucrutisation of science.
Some fear "the incipient dismantling of
the peer review system", brought about
by research fraud, university lobbying
for "pork barrel" grants, disputes over
intellectual property, and increased
secrecy in scientific research.
Despite these problems, few alternatives to current procedures or criteria
for grants peer review have been
advanced. When some are put forth ',
they inevitably become entangled in
value conflicts; for example, processing
submissions more efficiently versus
choosing more carefully or continuing
established investigations versus promoting promising but risky work.
Further, peer review is built so
deeply into the brickwork of science
that many refuse to examine and
improve it, fearing that any significant
change would weaken the entire edifice.
For some, questioning peer review is
to challenge deeply held values
about progress and prospects for
the society.
We are a voice to you; you have been a support to us. Together we build journalism that is independent, credible and fearless. You can further help us by making a donation. This will mean a lot for our ability to bring you news, perspectives and analysis from the ground so that we can make change together.
Comments are moderated and will be published only after the site moderator’s approval. Please use a genuine email ID and provide your name. Selected comments may also be used in the ‘Letters’ section of the Down To Earth print edition.