Nature’s 2016 Reproducibility Survey
In 2016, Nature surveyed 1,576 researchers about reproducibility in science for the feature “1,500 scientists lift the lid on reproducibility” (Baker, M. Nature 533, 452–454). The charts below are recomputed directly from the raw survey data, which is licensed under CC BY 4.0.
Is there a reproducibility crisis? #
Which statement about a 'crisis of reproducibility' do you agree with?
Have you established procedures for reproducibility? #
Share of all respondents, by when the procedures were established
Publishing replication attempts #
Have you ever tried to publish a reproduction attempt? Share of all 1,576 respondents
of researchers (471 of 1,576) have published a replication attempt of some kind.
Have you failed to reproduce an experiment? #
Share answering yes, by discipline — most scientists have experienced failure to reproduce results
How much published work in your field is reproducible? #
Distribution of answers within each discipline, ordered from most to least confident in the literature. Each panel shows the share of that discipline's respondents giving each estimate.
What factors contribute to irreproducible research? #
Share of all respondents rating each factor's contribution to failures to reproduce
What would improve reproducibility? #
Share of all respondents rating how likely each approach is to improve reproducibility
Source: Baker, M. “1,500 scientists lift the lid on reproducibility.” Nature 533, 452–454 (2016). doi:10.1038/533452a. All figures on this page are computed from the raw survey data released alongside the article.
Caveat: the survey was e-mailed to Nature readers and advertised on affiliated websites and social media as being about reproducibility, so the sample is self-selected and likely over-represents researchers already concerned with the issue. Percentages use all 1,576 respondents as the denominator; “I can’t remember” and blank answers count as no.