Seeing the Positives in Negative Results: Why it is Important to Publish ‘Negative’ Research Findings

The weekly news cycle is routinely scattered with announcements of new scientific research that promises to change our understanding of health and disease. Unsurprisingly, the research that ends up as news headlines tends to be the more exciting information–scientists located a new gene for Alzheimer’s, researchers cloned an animal, et cetera. What often gets left out of the popular media and even from scientific journals, however, is some less exciting, but equally important research–negative findings, or null findings.

Negative findings refer to results that do not support the researcher’s initial hypothesis.[1] Usually, a researcher predicts that their results will support the alternative hypothesis–that there is a difference between the two variables they are investigating–rather than the null hypothesis–that there is no difference between the variables. These results are often called non-significant, because the researcher did not find a statistically significant difference between the control group and the intervention group.[2] 

Despite the importance of negative findings for advancing research, ‘negative’ research results are significantly less likely to be published than positive results.[2] There are several reasons that negative findings are not published as often as positive findings. In the research world, there is a negative value or stigma attached to null findings, and there is a positive value attached to significant findings. We place a cultural value on getting positive results and correctly proving your original hypothesis. The very naming of the terms themselves–negative, null, or insignificant findings–frames them as something bad.

There are several reasons that negative findings are not published as often as positive findings. In the research world, there is a negative value or stigma attached to null findings, and there is a positive value attached to significant findings. We place a cultural value on getting positive results and correctly proving your original hypothesis. The very naming of the terms themselves–negative, null, or insignificant findings–frames them as something bad.

The resulting stigma attached to null findings makes them a low priority for publication. Many reputable journals are less likely to accept and publish negative findings because they generally have a lower citation rate, and thus a lower impact factor, and are often controversial.

Also due to this stigma, many scientists who get negative results deem their work to be useless and a waste of time. As a result, they may choose not to publish them.[1] Many scientists–particularly young ones–even fear publishing negative results could negatively impact their career, even forcing them out of research altogether. This fear is not unfounded. Researchers who spend a lot of time and money on the ‘wrong’ project will likely find that their research is published with a low impact factor. As a result they could receive less funding for future research. One site even warns (in bold), “Do not publish negative results as a young scientist. Leave it to the senior scientists who already have a successful career and can afford it to publish negative findings for the sake of good science!” Even well-established scientists experience this pressure to publish only their significant findings. Because scientists are involved in research as a career, they naturally find themselves having to compete for positions and funding for their research.[2] Publishing negative findings could frame a scientist as ‘unsuccessful’ and make it harder for him or her to secure funding for future projects.

Ironically, this obsession with championing only ‘positive’ results in the name of ‘advancing science’ may actually inhibit the advancement of science. One important reason to report negative findings is to make it known to other scientists what research has already been attempted. If negative results are not published, other researchers can’t find these studies and may waste precious time and resources on investigating questions that have already been looked into. Reporting negative findings can also inform scientists as to the efficacy and effectiveness of various interventions, and can help inform what should be done differently in similar research in the future.[1]

There are even some dangers in failing to report negative findings. When negative findings are not reported, theories that are untrue remain widely accepted and unquestioned–even when those theories can have harmful consequences. In 1998 for example, the highly regarded journal, The Lancet, published Dr. Andrew Wakefield findings that the Measles Mumps Rubella (MMR) vaccine causes intestinal inflammation, leading–Wakefield claimed–to the development of autism. This theory was later debunked after investigations finding that Dr. Wakefield had falsified results and four subsequent studies published in The Lancet were unable to replicate Wakefield’s findings. Now imagine those four studied had been rejected from publishing because their findings were ‘negative’–they supported the null hypothesis and found no significant difference in rates of autism between vaccinated and non-vaccinated children. Failure to publish these ‘negative’ results would have allowed Wakefield’s theory to prevail without question, potentially leading more and more parents choosing not to vaccinate their children. This example reminds us that publishing only positive findings leaves us with an incomplete picture of a phenomenon.

Despite the importance of publishing negative findings, however, papers that report negative results are still less likely to be published and the amount of non-significant data reported is steadily declining.[2]

In light of this dearth of published negative findings, a handful of groups have emerged in recent years whose goal is to promote the publishing of negative findings. A group of scientists in France, for example, founded Negative Results, a scientific journal dedicated to publishing high-quality research regardless of whether or not it supported the researcher’s initial hypothesis. Negative Results hopes to reverse the trend of a publishing bias toward positive results by “demonstrating the importance of disseminating all results while still maintaining the higher standards of excellence at the journal and in the scientific literature.” Numerous other journals have stepped forward to support the importance of publishing negative findings. The peer-reviewed journal Translational Behavioral Medicine, for example, issued in 2017 a “call for state-of-the-science papers with null findings to help further the field of translational science.”[1]

One paper points out that historically, “the noblest aspect of science is its supposed transparency in presenting all sides of a story.” These researchers then add that while this seems reasonable, it is easier said than done.[2] According to the founders of Negative Results, today there is “no widely accepted and widely known procedures [for] how to handle negative findings which are not used for a publication.” Because science is–or at least, should be–a collaborative discipline, individuals and groups within the scientific research community must work together to take action to create an effective system for publishing negative findings.[1] This effort could also be helpful in translating and sharing study results across countries–a historically large obstacle to disseminating research. Making the publishing of negative findings the norm, not the exception, should eventually reduce the stigma attached to negative findings. 

Perhaps, in the field of research, we need to re-learn the lesson we were taught in kindergarten–that there is value in learning from our mistakes. Or maybe, if we re-conceptualize ‘negative’ findings as something positive, we will no longer view these findings as ‘mistakes’ at all. Going forward, as one researcher points out: “Papers with null findings that have a strong research design and use rigorous methods with appropriate statistical analyses”–that is, papers that have negative results but are not ‘mistakes’–must be included in the evidence base.[1]

References:

[1] Miller-Halegoua, Suzanne M. (2017). Why null results do not mean no results: negative findings have implications for policy, practice, and research.Translational Behavioral Medicine, 7(2): 137.