Previous replication initiatives
Psychology
2014Many Labs 1Replication rate ~ 77% (10/13)▼
A large collaborative replication project involving 36 labs worldwide that tested 13 classic and contemporary psychological effects across multiple samples and settings. Replication statistics: 13 effects attempted; 10 successfully replicated (≈77% replication rate).Ten effects replicated consistently across sites, while three showed weak or no evidence for the original results. The study demonstrated that many psychological effects are robust across different contexts, though some depend strongly on experimental conditions or sample characteristics.
2015Reproducibility Project: PsychologyReplication rate ~ 36% (36/100)▼
An open, large-scale, collaborative effort to systematically examine the rate and predictors of reproducibility in psychological science, involving 72 researchers from 41 institutions who replicated studies published in three prominent psychological journals in 2008. Main findings: 100 studies attempted; 36 successfully replicated (36% replication rate).
Observed effects were 50% smaller on average (i.e., 50% the size of the original effect size, on average).
2018Experimental Philosophy – Reproducibility ProjectReplication rate ~ 70% (28/40)▼
A collaborative replication project in experimental philosophy (x-phi) that aimed to assess the reproducibility of findings at the intersection of philosophy and psychology. Replication statistics: Approximately 40 experiments attempted; specific replication rates varied by study. Key findings: The project demonstrated variability in replicability across experimental philosophy studies, with some effects replicating consistently while others did not, highlighting the need for rigorous replication in this interdisciplinary field.
2018Many Labs 2Replication rate ~ 46% (11/24)▼
An extension of Many Labs 1, this project aimed to replicate 28 findings across 125 labs and 36 countries to assess the generalizability of psychological effects. Replication statistics: 28 findings attempted; 14 successfully replicated (50% replication rate). Key findings: The replication rate was approximately 50%, indicating variability in the reproducibility of psychological effects and emphasizing the importance of context, culture, and methodological factors in replication success.
2022Motoki and Iseki (2022) – Sensory Marketing ReplicationReplication rate ~ 20% (2/10)▼
A pre-registered replication of ten influential sensory marketing studies conducted online with non-WEIRD (non-Western) consumers in Japan. Replication statistics: 10 studies attempted; 2 successfully replicated (20% replication rate). Key findings: Effect sizes were approximately half of original values. Successful replications involved sound symbolism studies with within-participants designs and larger original sample sizes. No studies involving visual factors, between-participant manipulation, or interactions between factors could be replicated.
Observed effects were 50% smaller on average (i.e., 50% the size of the original effect size, on average).
2023Boyce et al. (2023) – Student Replication ProjectsReplication rate ~ 49% (86/176)▼
A dataset of 176 replications conducted by graduate students in a methods course over eleven years. Replication statistics: 176 replications attempted; 49% judged successful. Of the 136 replications where effect sizes could be numerically compared, only 46% had point estimates within the prediction interval of the original outcome (versus the expected 95%). Key findings: Larger original effect sizes and within-participants designs were associated with replication success, indicating low robustness in the psychology literature.
Observed effects were 54% smaller on average (i.e., 46% the size of the original effect size, on average).
Social Science
2018Camerer et al. (2018) – Nature/Science Social Science Replication ProjectReplication rate ~ 62% (13/21)▼
A large-scale replication effort examining the reproducibility of findings across various social science disciplines, including economics, sociology, and political science. Replication statistics: 21 studies attempted; 13 successfully replicated (61.9% replication rate). Key findings: 62% of replications found significant effects in the same direction as originals, though effect sizes averaged about 50% of the original effect sizes, emphasizing the importance of replication in social science research.
Observed effects were 50% smaller on average (i.e., 50% the size of the original effect size, on average).
2025Examining Replicability of Online Experiments (Holzmeister et al., 2025)Replication rate ~ 54% (14/26)▼
A replication study examining 26 online behavioral experiments, stratified by impact level. Replication statistics: 26 online experiments attempted. Key findings: Overall 54% replication rate with replication effects being approximately 45% of original effect sizes, revealing large disparities in replication success by impact tier, suggesting that higher-impact studies may face greater replication challenges.
Observed effects were 55% smaller on average (i.e., 45% the size of the original effect size, on average).
Biomedical
2025Brazilian Reproducibility InitiativeReplication rate ~ 26% (13/97)▼
A multicentre initiative in Brazil to assess reproducibility of biomedical experiments run in Brazil (replicated papers needed at least 50% of authors to be based in Brazil). The initiative selected approximately 60 experiments from Brazilian published articles (including cell-viability assays, RT-PCR, and rodent elevated plus maze), with each experiment replicated in 3 laboratories. Replication statistics: 60 experiments attempted; replication success varied between 15% and 45% depending on criteria used. Key findings: Initial results show significant variability in replication success, with replication rates ranging from ~15% to ~45% depending on the specific criteria applied, highlighting challenges in reproducing biomedical research even within the same country's research context.
Observed effects were 40% smaller on average (i.e., 60% the size of the original effect size, on average).
Cancer biology/ oncology
2011Bayer Healthcare's replication reportReplication rate ~ ~33% (33/67)▼
"In a correspondence in Nature Reviews Drug Discovery, three researchers at Bayer Healthcare reported the results of a survey of 23 scientists in their oncology division. They found that across 67 replication attempts, only "~20-25%" yielded results that closely matched the published findings. They reported that 'In almost two-thirds of the projects, there were inconsistencies between published data and in-house data that either considerably prolonged the duration of the target validation process or, in most cases, resulted in termination of the projects because the evidence that was generated for the therapeutic hypothesis was insufficient to justify further investments into these projects.'"
2012Amgen's replication reportReplication rate ~ 11% (53/6)▼
Over the course of ten years, scientists in the haematology and oncology department of Amgen tried to confirm published findings in preclinical oncology that were related to their work. In a 2012 commentary in Nature they reported they could only reproduce 6/53 findings (11%). A finding was considered non-reproduced if the finding was "not sufficiently robust to drive a drug-development program".
2021Reproducibility Project: Cancer BiologyReplication rate ~ 40-78% (46-91/116)▼
The team replicated 50 experiments across 23 high-impact cancer biology papers. They originally set out to replicate 53 papers, but protocol problems, uncooperative authors and others factors forced them to scale back to just 23 papers. The project evaluated 158 reported effects across those papers and found that replication effect sizes were dramatically attenuated, with a median 85% reduction relative to the originals and 92% of replication effect sizes smaller than initially reported. Using five replication criteria applicable to both positive and null findings, only 40% of positive effects and 80% of null effects replicated, yielding an overall replication success rate of 46%. Using a less strict definition of replication (was there a statistically significant effect in the same direction?) the replication rate was 78%.
Observed effects were 84% smaller on average (i.e., 16% the size of the original effect size, on average).
Economics
2016Camerer et al. (2016) – Experimental Economics Replication ProjectReplication rate ~ 61% (11/18)▼
A collaborative initiative to replicate influential experimental economics studies published in American Economic Review and the Quarterly Journal of Economics between 2011-2014. Replication statistics: 18 studies attempted; 11 successfully replicated (61.1% replication rate). Key findings: a 61% replication rate with replicated effect sizes being 66% of the original effect sizes.
Observed effects were 34% smaller on average (i.e., 66% the size of the original effect size, on average).
Neuroscience
ONGOING#EEGManyLabsReplication rate ~ TBD%▼
An international multi-lab replication project focused on replicating 27 influential electroencephalography (EEG) studies across multiple laboratories. Replication statistics: 27 influential EEG studies attempted; replication rates varied by paradigm and analysis approach. Key findings: Results varied significantly by paradigm and analysis method, demonstrating that EEG findings are context-dependent and highlighting the importance of methodological transparency in neuroscience replication efforts.