Nosek et al., 2015 Science

Brian Nosek et al. present in this communication an informative introduction to standards set-up by the Transparency and Openness Promotion (TOP) committee to facilitate a culture of open research (e.g. the publication of null results). A comprehensive table is provided summarizing the eight standards and three levels of TOP guidelines. LINK

The Economist reported (link) that the biggest foundation in the world, the Bill & Melinda Gates Foundation, have a new policy requiring that all research funded by them has to be made available to everybody for free. This is really interesting, since it will not allow researchers receiving funding from this foundation to publish their work in several high impact journals. It will be interesting how many other institutions will follow, especially since we also reported already, that the Wellcome Trust (2nd biggest foundation) supports researchers to publish their work in freely available data repositories.

A manifesto for reproducible science

Manufo et al published an open access manifesto in Nature Human Behavior for reproducible science. This interesting piece summarizes the problems in science and presents several key elements that need to be widely implemented to optimize the scientific process. The authors refer to current analysis showing that several elements indeed improve the current situation. However, the implementation of these elements, like improving methods, reporting and dissemination, reproducibility, evaluation and incentives are only slowly adopted. It was interesting to read about the changes in evaluation and that especially this part needs further ideas since the authors point out that the peer-review process is changing due to different factors. But the trend towards an open peer-review process comprises new problems. Importantly, the authors also claim that science will not work without a very well implemented evaluation process.

By reading so many publications and talking to people about the irreproducibility crisis, one can learn a lot about different ideas which are repeatedly presented by the same people. However, one also gets the impression that the right ideas certainly exist, but that the scientific community is divided in groups who are A) passionately fighting to improve the situation, B) a group that is very slowly moving towards implementing measures for higher reproducibility but C) also a group ignoring the situation all together. The challenge will be to establish a mechanism to reach and change the mind set of all researchers. LINK

Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity

This recently published survey identifies the most frequent problem in science: sloppy research.  The survey was performed among participants of the World Conference on Research Integrity and contained 60 questions whereas from the 1500+ participants 17% replied by answering the questions.  However, even though fabrication of data ranked high when asking for the impact on truth but the frequency for this dishonorable behavior seemed to be pretty low. In contrast to performing sloppy science in form of selective reporting, selective citing, and flaws in quality assurance and mentoring which ranked much higher when asking for frequency!

This is actually nicely mirrored in two publications dealing with the costs for the society.  The publication by Stern et al. 2014 numbered the costs for fabricated and falsified data to 400.000US$ for each paper.  This cumulates to costs of 58 million US$ for the period of 1992 to 2012 according to the authers.  BUT, the costs for unreproducible data in the US alone were numbered to 28 billion US$ every year by Freedman et al. in 2015. These examples clearly show that the hyped stories about falsified data by news media are only a minor problem.  The real issue seems to be the smouldering problem of sloppy science that eats up so many resources and needs to be targeted. LINK