Re-analysis initiatives

An ongoing initiative founded in 2022 and led by Abel Brodeur that systematically reproduces and replicates studies published in leading economics and political science journals, and has been expanding into psychology (through collaborations with Psychological Science and Nature Human Behaviour) as well as public health and ecology. Results are collated in a growing discussion paper series — more than 240 replication reports so far, many of them produced at the "replication games" the institute organizes around the world. Reports are written at the level of the paper and cover computational reproduction (re-running the original code and data), checks for coding errors, and robustness re-analyses that vary analytical decisions on the original data (e.g. changing control variables, samples, or estimation and inference methods); a smaller share of re-analyses introduce new data and so constitute replications in the stricter sense. Flagship result (Brodeur et al. 2026, Nature): across 110 articles (79 in economics, 31 in political science) from 12 leading journals with mandatory data- and code-sharing policies, over 85% of published claims were computationally reproducible, coding errors were found in about 25% of studies, and 72% of originally significant estimates remained statistically significant with the same sign under robustness re-analyses.

The Social Science Reproduction Platform, developed by the Berkeley Initiative for Transparency in the Social Sciences (BITSS) together with the American Economic Association's Data Editor, crowdsources and catalogs attempts to assess and improve the computational reproducibility of social science research, following a standardized reproduction protocol, and is widely used in teaching. It collates computational reproductions — re-running the original paper's data and code, what we call technical replications — rather than replications on new data.