‘Positive’ Studies More Likely to Make It Into Medical Journals

Confirming suspicions that studies with low-key results often get neglected, a review finds that research is more likely to end up in print if it has a certain 'wow' factor. Based on the findings of five studies, a new Cochrane systematic review estimates that trials are 1.78 times more likely to be published if they are perceived as important, reveal a positive effect, or offer scientifically significant findings.

Kay Dickersin, a coauthor of the review, says this is no minor matter. “If positive results are published more often than negative, what we think we know isn’t really what we know. We might think a drug works, when it really doesn’t work, because the negative results haven’t been published,” says Dickersin, director of the U.S. Cochrane Center at John Hopkins University’s Bloomberg School of Public Health.

Scientists have spent hundreds of years worrying about “publication bias,” Dickersin says, but researchers only began tackling the issue in depth during the 1990s. The new review examined five studies from the 1990s, including some by Dickersin herself.

The review appears in the latest issue of The Cochrane Library, a publication of The Cochrane Collaboration, an international organization that evaluates medical research. Systematic reviews draw evidence-based conclusions about medical practice after considering both the content and quality of existing medical trials on a topic.

All of the studies found that publication was more likely for trials with positive findings. The studies suggested that 41% of negative trials—those that show a drug or treatment has bad effects or none at all—make their way to print. By contrast, the researchers estimated that journal editors publish about 73% of positive studies.

Two studies looked at how long study publication took. One found that positive studies showed up in journals an average of 4.3 years after initial enrollment of patients compared with 6.5 years for studies with negative findings. The other study, which took a slightly different approach, reported an even larger gap.

The results are not surprising to those familiar with the medical research world, but “we now have an estimate of how big the problem is, combining all the studies that examined publication bias in the same way,” Dickersin says. “It’s empirical data and no longer a hunch that we have had for several hundred years.”

It is unclear from the data, however, whether investigators fail to submit negative studies or journal editors prefer not to accept them.

What should the research world do? Dickersin says it is vital to create registries that include all studies that are in the works. “Unfortunately, there are very powerful lobbies, and there’s inertia that has prevented registration of all human trials in all countries,” she says.

Still, trial registration does not solve the whole problem, she says, because publication does not always follow registration for study outcomes or results. That means the world might know there was a study but never find out what the researchers discovered.

Trish Groves, MD, deputy editor and senior research editor of BMJ, says the findings of the review sound valid. “However, the usefulness of the review is limited by its inability to add much to what we already know or to shed light on current editorial decision-making and practice,” she says.

Editors at BMJ try to avoid publication bias by focusing on the question asked by a study under consideration and the quality of the methods used to find an answer, Groves says. “Whether the answer was positive or negative takes relatively low priority.”

Groves adds that two of BMJ’s top 10 research papers in 2008 had negative results. (The journal chose the list using a variety of criteria, including citations, downloads, and letters about the studies.)

As for the future, review author Dickersin says researchers should do more to examine how factors like language and funding sources affect whether a study is published. Researchers should also consider ethical issues regarding patients who take part in studies, she says. “If patients had participated thinking they’d contributed to knowledge and then that study is never published, we’ve broken the covenant with the patients. That’s the ethical side effect of the failure to publish.”

Source: Reported by Randy Dotinga, Health Behavior News Service

(View the Daily News Archive)