Replication Rate by Author h-index

Are papers by eminent researchers more replicable? This page plots the replication rate of original papers in the database against the h-index of their authors — the mean across the byline, the most-cited coauthor, or the first or last author alone — using author metrics from SciSciNet (derived from OpenAlex). Note that h-indexes are the authors’ current values, not their values when the original paper was published (see methodology below).

Rate = success / (success + failure + reversal); inconclusive and unrecorded outcomes excluded, where success means the outcome recorded in the database. Unit: replication effect (n = 7,382 — originals matched in the author h-index dataset). How this is defined.

Replication rate by mean h-index of all authors

0%25%50%75%100%<20: 60.7% replicated (620/1022), cluster-bootstrap 95% CI [57–64%]61%<20n=1,02220–39: 57.3% replicated (1153/2011), cluster-bootstrap 95% CI [54–60%]57%20–39n=2,01140–59: 48.6% replicated (791/1629), cluster-bootstrap 95% CI [44–53%]49%40–59n=1,62960–89: 50.0% replicated (513/1025), cluster-bootstrap 95% CI [46–55%]50%60–89n=1,02590+: 51.3% replicated (254/495), cluster-bootstrap 95% CI [44–59%]51%90+n=495Mean h-index of all authorsReplication rate (%)The Metascience Observatory

All determinate effect-level replications whose original paper matched in the SciSciNet snapshot, grouped into fixed h-index ranges. Each replication attempt counts once. Hover a bar for its 95% interval from a paper-cluster bootstrap (1,000 resamples of original papers with replacement), which accounts for multiple replications of the same paper not being independent.

Modeled replication probability by mean h-index of all authors

A probit regression of replication success on log₁₀(1 + h), plotted as a predicted-probability curve with a 95% confidence band — the same data as the bars above, without the arbitrary bin edges. Bin rates are overlaid as dots for comparison.

0%25%50%75%100%050100150<20 (mean h 13): 60.7% replicated (620/1022), cluster-bootstrap 95% CI [57–64%]20–39 (mean h 30): 57.3% replicated (1153/2011), cluster-bootstrap 95% CI [54–60%]40–59 (mean h 49): 48.6% replicated (791/1629), cluster-bootstrap 95% CI [44–53%]60–89 (mean h 72): 50.0% replicated (513/1025), cluster-bootstrap 95% CI [46–55%]90+ (mean h 114): 51.3% replicated (254/495), cluster-bootstrap 95% CI [44–59%]h = 0: predicted 76.1% replicated (95% CI 68–83%)h = 13: predicted 60.8% replicated (95% CI 58–64%)h = 25: predicted 56.5% replicated (95% CI 54–59%)h = 38: predicted 53.9% replicated (95% CI 52–56%)h = 50: predicted 52.1% replicated (95% CI 50–54%)h = 63: predicted 50.6% replicated (95% CI 48–53%)h = 75: predicted 49.4% replicated (95% CI 47–52%)h = 88: predicted 48.4% replicated (95% CI 45–52%)h = 100: predicted 47.5% replicated (95% CI 44–51%)h = 113: predicted 46.7% replicated (95% CI 43–50%)h = 125: predicted 46.0% replicated (95% CI 42–50%)h = 138: predicted 45.4% replicated (95% CI 41–50%)h = 150: predicted 44.8% replicated (95% CI 41–49%)Probit fit ± 95% CIBinned rate ± bootstrap CIMean h-index of all authorsReplication rate (%)The Metascience Observatory

Slope β₁ = -0.385 (cluster-robust SE 0.079, z = -4.90, p < 0.001) per tenfold increase in 1 + h. On average across the sample, a doubling of 1 + h shifts the predicted replication probability by -4.6 percentage points. n = 6,182 replications across 4,214 original papers; standard errors are clustered on the original paper.

6,182 replications plotted · 1,200 determinate rows excluded (no SciSciNet match)

h-index source. Author h-indexes come from a SciSciNet-v2 snapshot (derived from OpenAlex; snapshot 2026-01-29, lookup generated 2026-07-14). Papers are matched by DOI, then joined to their authors and each author’s h-index; authors missing from the snapshot are dropped from the mean and max.

Current, not contemporaneous. An author’s h-index is their value today, not at the time the original paper was published. It therefore bakes in everything that happened since — including citations to the original paper itself and to the replication debate around it — and is confounded with career stage, field citation norms, and team size. Treat these charts as descriptive, not causal.

Matching. Original papers are matched by DOI: 7,331 of 8,892 replication rows (82.4%) have a matched original with author data; rows without a DOI or without a SciSciNet record are excluded from the charts.

Units. The chart counts every determinate replication attempt once. Reversals count as determinate non-replications; inconclusive rows are excluded. First-author h-index is used as the last-author value for single-author papers.

Probit model. The curve is a maximum-likelihood probit fit, P(replicated) = Φ(β₀ + β₁ · log₁₀(1 + h)), on the same determinate rows as the bars. Standard errors use a sandwich estimator clustered on the original paper (with the usual G/(G−1) small-sample correction), and the band is the delta-method 95% interval on the linear predictor pushed back through Φ, so it cannot leave 0–100%. This is the equivalent of Stata’s probit, vce(cluster) followed by margins / marginsplot. The model assumes the probit link is the right shape; where the curve and the binned dots disagree, the dots are the less model-dependent summary.

Data: replications_database_2026_09_28_155521.csv.

← Back to the replications database