In 2013, the federal government spent over $30 billion to support basic scientific research. These funds help create knowledge and stimulate greater productivity and commercial activity, but could we get an even better return on our investment?
The problem is that the research conducted using federal funds is driven - and distorted - by the academic publishing model. The intense competition for space in top journals creates strong pressures for novel, statistically significant effects. As a result, studies that do not turn out as planned or find no evidence of effects claimed in previous research often go unpublished, even though their findings can be important and informative.
For instance, a top psychology journal refused to consider studies that failed to replicate a disputed publication claiming to find evidence of extrasensory perception. In addition, the findings that do get published in these journals often just barely reach the statistical significance thresholds required for publication - a pattern that suggests selective reporting and publishing of results. Not surprisingly, other scientists often cannot reproduce published findings, which undermines trust in research and wastes huge amounts of time and money. These practices also create a shaky knowledge base for science, preventing scholars from effectively building on prior research.
This pattern of publication bias and failed replications, which is drawing attention in fields from psychology to medicine, has prompted great alarm within the scientific community. Now there are signs that these concerns have spread to policy makers. The Obama administration has asked for public comment on how the federal government can 'leverage its role as a significant funder of scientific research to most effectively address' the replication crisis in science - a question that should be carefully considered given the evidence that current policies are not working.
One approach is to require researchers to share data, especially from studies conducted with public support. For instance, the National Institutes of Health and the National Science Foundation already require grantees to share data from their research. These sorts of requirements encourage transparency but, even if widely adopted, are unlikely to appreciably reduce bias in which studies are actually published.
Others advocate requiring the registration of trials before data has been collected. For instance, some social scientists have voluntarily begun to preregister analysis plans for experiments to minimize concerns about selective reporting. Unfortunately, the demand for statistically significant results is still likely to create publication bias. For example, federal law and journal policies now require registration of clinical trials, but publishing of trial results has been found to be selective, to frequently deviate from protocols and to emphasize significant results. Access to trial data could be increased, but such a step is again unlikely to change which studies are published in the most influential journals.
Instead, my colleagues and I propose a radically different publishing model: Ask journal editors and scientific peers to review study designs and analysis plans and commit to publish the results if the study is conducted and reported in a professional manner (which will be ensured by a second round of peer review).
This procedure encourages authors and reviewers to develop the strongest possible designs - including those that replicate previously published studies - and eliminates perverse incentives to find or emphasize significant results after the fact. A new scientific format called Registered Reports using this approach has already been adopted at numerous journals across the social and natural sciences.
In a new white paper, I propose that the American Political Science Association offer options for articles in a Registered Reports-style format. Researchers in other academic disciplines and scientific associations are starting to do the same.
Unfortunately, overcoming the inertia of the current system will be difficult, which is why altering the incentives created by federal science policy is so important.
Scientists would change their ways much more rapidly if federal funding encouraged publishing in journals that used Registered Reports or other formats intended to minimize publication bias. Conversely, journals would be more likely to change their policies if it would help them attract research from top scientists. Appropriately enough, the best way to encourage scientific innovation might be to rethink how we organize the scientific enterprise itself.
Entities 0 Name: National Science Foundation Count: 1 1 Name: National Institutes of Health Count: 1 2 Name: American Political Science Association Count: 1 3 Name: Obama Count: 1 Related 0 Url: http://ift.tt/1miDLUD Title: As Elsevier Connect turns 2 - spotlighting our contributors Description: When we launched Elsevier Connect two years ago, we aimed to create a platform where members the global research community could learn about each other's work and engage in conversation. Since then, what has made Elsevier Connect thrive its contributors, who come from all corners of the science and medical world.
Post a Comment for "To Get More Out of Science, Show the Rejected Research"