Replication Rate by Author h-index

Are papers by eminent researchers more replicable? This page plots the replication rate of original papers in the database against the h-index of their authors — the mean across the byline, the most-cited coauthor, or the first or last author alone — using author metrics from SciSciNet (derived from OpenAlex). Note that h-indexes are the authors’ current values, not their values when the original paper was published (see methodology below).

Rate = success / (success + failure + reversal); inconclusive and unrecorded outcomes excluded, where success means the outcome recorded in the database. Unit: replication effect (n = 7,015 — originals matched in the author h-index dataset). How this is defined.

Replication rate by mean h-index of all authors

0%25%50%75%100%<20: 60.9% replicated (615/1010), cluster-bootstrap 95% CI [57–65%]61%<20n=1,01020–39: 57.2% replicated (1145/2003), cluster-bootstrap 95% CI [54–60%]57%20–39n=2,00340–59: 48.3% replicated (777/1608), cluster-bootstrap 95% CI [44–53%]48%40–59n=1,60860–89: 49.8% replicated (501/1006), cluster-bootstrap 95% CI [46–54%]50%60–89n=1,00690+: 50.5% replicated (243/481), cluster-bootstrap 95% CI [44–58%]51%90+n=481Mean h-index of all authorsReplication rate (%)The Metascience Observatory

All determinate effect-level replications whose original paper matched in the SciSciNet snapshot, grouped into fixed h-index ranges. Each replication attempt counts once. Hover a bar for its 95% interval from a paper-cluster bootstrap (1,000 resamples of original papers with replacement), which accounts for multiple replications of the same paper not being independent.

Modeled replication probability by mean h-index of all authors

A probit regression of replication success on log₁₀(1 + h), plotted as a predicted-probability curve with a 95% confidence band — the same data as the bars above, without the arbitrary bin edges. Bin rates are overlaid as dots for comparison.

0%25%50%75%100%050100150<20 (mean h 13): 60.9% replicated (615/1010), cluster-bootstrap 95% CI [57–65%]20–39 (mean h 30): 57.2% replicated (1145/2003), cluster-bootstrap 95% CI [54–60%]40–59 (mean h 49): 48.3% replicated (777/1608), cluster-bootstrap 95% CI [44–53%]60–89 (mean h 72): 49.8% replicated (501/1006), cluster-bootstrap 95% CI [46–54%]90+ (mean h 114): 50.5% replicated (243/481), cluster-bootstrap 95% CI [44–58%]h = 0: predicted 77.0% replicated (95% CI 69–84%)h = 13: predicted 61.0% replicated (95% CI 58–64%)h = 25: predicted 56.5% replicated (95% CI 54–59%)h = 38: predicted 53.8% replicated (95% CI 52–56%)h = 50: predicted 51.8% replicated (95% CI 50–54%)h = 63: predicted 50.2% replicated (95% CI 48–53%)h = 75: predicted 49.0% replicated (95% CI 46–52%)h = 88: predicted 47.9% replicated (95% CI 45–51%)h = 100: predicted 47.0% replicated (95% CI 44–50%)h = 113: predicted 46.1% replicated (95% CI 42–50%)h = 125: predicted 45.4% replicated (95% CI 42–49%)h = 138: predicted 44.7% replicated (95% CI 41–49%)h = 150: predicted 44.1% replicated (95% CI 40–48%)Probit fit ± 95% CIBinned rate ± bootstrap CIMean h-index of all authorsReplication rate (%)The Metascience Observatory

Slope β₁ = -0.408 (cluster-robust SE 0.078, z = -5.24, p < 0.001) per tenfold increase in 1 + h. On average across the sample, a doubling of 1 + h shifts the predicted replication probability by -4.8 percentage points. n = 6,108 replications across 4,211 original papers; standard errors are clustered on the original paper.

6,108 replications plotted · 907 determinate rows excluded (no SciSciNet match)

h-index source. Author h-indexes come from a SciSciNet-v2 snapshot (derived from OpenAlex; snapshot 2026-01-29, lookup generated 2026-07-14). Papers are matched by DOI, then joined to their authors and each author’s h-index; authors missing from the snapshot are dropped from the mean and max.

Current, not contemporaneous. An author’s h-index is their value today, not at the time the original paper was published. It therefore bakes in everything that happened since — including citations to the original paper itself and to the replication debate around it — and is confounded with career stage, field citation norms, and team size. Treat these charts as descriptive, not causal.

Matching. Original papers are matched by DOI: 7,241 of 8,445 replication rows (85.7%) have a matched original with author data; rows without a DOI or without a SciSciNet record are excluded from the charts.

Units. The chart counts every determinate replication attempt once. Reversals count as determinate non-replications; inconclusive rows are excluded. First-author h-index is used as the last-author value for single-author papers.

Probit model. The curve is a maximum-likelihood probit fit, P(replicated) = Φ(β₀ + β₁ · log₁₀(1 + h)), on the same determinate rows as the bars. Standard errors use a sandwich estimator clustered on the original paper (with the usual G/(G−1) small-sample correction), and the band is the delta-method 95% interval on the linear predictor pushed back through Φ, so it cannot leave 0–100%. This is the equivalent of Stata’s probit, vce(cluster) followed by margins / marginsplot. The model assumes the probit link is the right shape; where the curve and the binned dots disagree, the dots are the less model-dependent summary.

Data: replications_database_2026_08_01_120356.csv.

← Back to the replications database