Daniele Fanelli, a researcher at University of Edinburgh in Scotland is analysing the increasing bias in scientific studies. He did a similar research in June 2010. Fanelli speaks to Sugandh Juneja on what is new in his latest study. Excerpts:
What are the findings of your study?
The study analysed over 4,600 papers published across disciplines between 1990 and 2007. The frequency of papers, that declared a positive result for a “tested” hypothesis, was measured. This frequency has grown by over 22 per cent. The increase has been stronger in social and biomedical disciplines. Also, the US published significantly fewer positive results than Asian countries (particularly Japan) but more than European countries in particular the UK.
What is the difference between your present study and the one you did in April last year?
The study conducted in April found a correlation between academic productivity and frequency of positive results in American studies. It suggested that the hypothesis on a relation between pressures to publish and bias might be true (See “The file drawer bust”, Down To Earth, June 15-30, 2010). But it did not prove that actual changes were occurring in science. The new study looks directly at the frequency of positive results over the years. It finds there is a change occurring in how we produce and publish science. It is changing in the direction that, many fear, is that of rewarding only a specific kind of result.
How did you come to this conclusion?
A growth of 22 per cent in the frequency of positive results in 17 years does not seem to have any reasonable scientific explanation. It must necessarily be the consequence of certain changes in the system. Either journals are selecting more positive results or scientists are manipulating their data and submitting fewer negative results.
What sort of increase in the reporting bias does your study register? Is this discipline specific? What will be its impact?
Let me clarify that we do not know for sure that there is an overall bias in the literature. We cannot exclude that negative results are published in more obscure journals, which the study excluded, but which are accessible to the scientists. What we know for sure is that the need to get positive results is increasing. If the system favours positive results, then scientists might pursue more obvious research questions and be less objective in how they report their results.
How can we ensure that negative results get reported?
Journals should move onto a system of fully blind peer-review, in which authors only submit introduction, materials and methods. This would force editors and peer-reviewers to select papers based on question and validity of methods, not on the results obtained. Also, studies that replicate previous important findings or challenge existing paradigms should be rewarded more. Funding institutions should reserve a percentage exclusively for such projects.
The study was published in Scientometrics on September 11.
We are a voice to you; you have been a support to us. Together we build journalism that is independent, credible and fearless. You can further help us by making a donation. This will mean a lot for our ability to bring you news, perspectives and analysis from the ground so that we can make change together.
.png)
Comments are moderated and will be published only after the site moderator’s approval. Please use a genuine email ID and provide your name. Selected comments may also be used in the ‘Letters’ section of the Down To Earth print edition.