Correlates of Reproducibility
Each page of the replications database looks at one predictor of replication success at a time. This page puts them side by side: first as simple correlations between each predictor and the replication outcome, then jointly in an interpretable regression model that shows how much each factor contributes once the others are held fixed.
How the outcome is coded on this page
Analyses here score each replication attempt as 1 = success, 0 = failure or reversal, and 0.5 = recorded as inconclusive, using the stored result (the “reported” criterion; rows with no recorded result are dropped, n = 138). This differs from the site-wide success-rate definition (“success / (success + failure + reversal); inconclusive and unrecorded outcomes excluded”), which excludes inconclusive attempts entirely — see how outcomes are classified. Analyses are at the effect level; because one original paper can be replicated many times, all confidence intervals are paper-cluster bootstrap intervals (1,000 draws, resampling original papers), which are wider and more honest than intervals that assume independent rows.
Correlation table
Pearson r is computed on the transformed predictor shown in the second column (heavily skewed predictors are log-scaled); Spearman's ρ is rank-based, so it uses the raw values and is unaffected by any monotone transform. Each row links to the page that examines that predictor in depth, and each uses every effect for which that predictor is available, so n varies by row.
| Predictor | Pearson scale | n (effects) | Papers | 95% CI | 95% CI | ||
|---|---|---|---|---|---|---|---|
| Original p-value | log₁₀ p (floored at −10) | 656 | 471 | -0.176 | [-0.274, -0.083] | -0.130 | [-0.228, -0.034] |
| Original publication year | raw | 8,266 | 5,779 | 0.016 | [-0.025, 0.053] | 0.056 | [0.022, 0.088] |
| Journal impact factor | log₁₀ IF | 7,486 | 5,155 | -0.080 | [-0.114, -0.046] | -0.077 | [-0.112, -0.040] |
| Journal rank (SJR percentile) | raw | 7,593 | 5,228 | -0.082 | [-0.112, -0.048] | -0.113 | [-0.144, -0.083] |
| Citation count (first 2 years) | log₁₀(1+c) | 7,163 | 4,861 | -0.084 | [-0.116, -0.049] | -0.089 | [-0.122, -0.053] |
| First-author h-index | log₁₀(1+h) | 7,108 | 4,826 | -0.055 | [-0.092, -0.019] | -0.051 | [-0.089, -0.011] |
| Last-author h-index | log₁₀(1+h) | 7,061 | 4,789 | -0.047 | [-0.091, 0.001] | -0.054 | [-0.095, -0.011] |
| Mean author h-index | log₁₀(1+h) | 7,108 | 4,826 | -0.085 | [-0.117, -0.054] | -0.083 | [-0.116, -0.050] |
| Max author h-index | log₁₀(1+h) | 7,108 | 4,826 | -0.075 | [-0.108, -0.045] | -0.073 | [-0.111, -0.041] |
| Author overlap (shared authors) | raw | 7,771 | 5,454 | 0.179 | [0.152, 0.209] | 0.280 | [0.254, 0.303] |
- Outcome coded 1 / 0.5 / 0 as described above; 95% CIs are paper-cluster bootstrap percentile intervals.
- The p-value row uses only exactly reported p-values (type “=”, 656 effects) — stricter than the by-p-value page, which also admits p-values whose reporting type is unrecorded. Reported p-values run down to the smallest representable number, so for the Pearson correlation log₁₀ p is floored at -10 (p < 10-10 treated as 10-10); Spearman's ρ is unaffected.
- SJR percentile is oriented so higher = better-ranked journal. Journals whose OpenAlex impact factor is exactly 0 are excluded from the impact-factor row (358 effects), since a log scale cannot represent them.
Summary table
The same estimates in a compact, shareable form.
Correlates of reproducibility
across 8,307 replicated effects from 5,813 papers
| Predictor | r | 95% CI | Spearman ρ | 95% CI |
|---|---|---|---|---|
| Author overlap (shared authors) | 0.18 | [0.15, 0.21] | 0.28 | [0.25, 0.30] |
| Original publication year | 0.02 | [-0.03, 0.05] | 0.06 | [0.02, 0.09] |
| First-author h-index | -0.05 | [-0.09, -0.02] | -0.05 | [-0.09, -0.01] |
| Last-author h-index | -0.05 | [-0.09, 0.00] | -0.05 | [-0.10, -0.01] |
| Max author h-index | -0.08 | [-0.11, -0.04] | -0.07 | [-0.11, -0.04] |
| Journal impact factor | -0.08 | [-0.11, -0.05] | -0.08 | [-0.11, -0.04] |
| Mean author h-index | -0.09 | [-0.12, -0.05] | -0.08 | [-0.12, -0.05] |
| Citation count (first 2 years) | -0.08 | [-0.12, -0.05] | -0.09 | [-0.12, -0.05] |
| Journal rank (SJR percentile) | -0.08 | [-0.11, -0.05] | -0.11 | [-0.14, -0.08] |
| Original p-value | -0.18 | [-0.27, -0.08] | -0.13 | [-0.23, -0.03] |
What the correlations say
Every prestige-flavored predictor — journal impact factor, journal rank, citations, and author h-index in all four variants — correlates negatively with replication: more prestigious venues and more eminent, more-cited work replicate slightly less often. The single strongest correlate is author overlap (ρ = 0.280): replications that share authors with the original succeed far more often than independent ones. Smaller original p-values predict success (ρ = -0.130 on the exactly-reported subset), and publication year has at most a weak positive trend. All of these are small by conventional standards — no single feature of a paper comes close to determining whether it replicates.
A simple joint model
The correlations above are pairwise, and the predictors overlap (highly cited papers sit in high-impact journals written by high-h authors). To see what each factor contributes with the others held fixed, we fit a fractional logistic regression of the 0 / 0.5 / 1 outcome on all predictors at once. Covariates are standardized, so coefficients read as log-odds per one standard deviation and are directly comparable; the AME column converts each into percentage points of predicted replication score. Only mean author h-index enters (the four h-index variants are too collinear to separate, r ≈ 0.9), and the p-value — missing for most rows — is added only in Model B, which is restricted to the much smaller exactly-reported-p subset.
Model A — all effects with complete predictors
Publication year, journal impact factor and rank, first-2-year citations, mean author h-index, and author overlap.
| Covariate | β per +1 SD | SE | z | p | AME (pp per +1 SD) | +1 SD means… |
|---|---|---|---|---|---|---|
| Publication year | 0.064 | 0.039 | 1.63 | 0.103 | 1.5 | 10.3 years later |
| log₁₀ impact factor | 0.004 | 0.045 | 0.09 | 0.924 | 0.1 | ≈ ×2.4 higher impact factor |
| SJR percentile | -0.105 | 0.043 | -2.43 | 0.015 | -2.5 | 10.5 SJR percentile points higher |
| log₁₀(1+cit. first 2 yrs) | -0.073 | 0.044 | -1.64 | 0.100 | -1.7 | ≈ ×3.5 the first-2-year citations (+1) |
| log₁₀(1+mean author h) | -0.074 | 0.035 | -2.09 | 0.037 | -1.8 | ≈ ×1.9 the mean author h-index (+1) |
| Shared authors | 0.466 | 0.061 | 7.63 | < 0.001 | 11.1 | 1.8 more shared authors |
Fractional logit (quasi-MLE), n = 6,305 effects in 4,293 original papers; standard errors clustered on the original paper. Covariates are z-scored on this model's own estimation sample, so coefficients are comparable within the table. AME = average marginal effect, in percentage points of predicted replication score per +1 SD.
Model B — adds the original p-value
Same covariates plus log₁₀ p (floored at -10), on the small subset with an exactly reported original p-value — read with caution.
| Covariate | β per +1 SD | SE | z | p | AME (pp per +1 SD) | +1 SD means… |
|---|---|---|---|---|---|---|
| Publication year | 0.081 | 0.127 | 0.64 | 0.522 | 1.8 | 5.6 years later |
| log₁₀ impact factor | 0.024 | 0.142 | 0.17 | 0.867 | 0.5 | ≈ ×2.4 higher impact factor |
| SJR percentile | 0.127 | 0.123 | 1.03 | 0.302 | 2.9 | 9.5 SJR percentile points higher |
| log₁₀(1+cit. first 2 yrs) | -0.054 | 0.139 | -0.39 | 0.698 | -1.2 | ≈ ×3.4 the first-2-year citations (+1) |
| log₁₀(1+mean author h) | -0.161 | 0.131 | -1.23 | 0.219 | -3.6 | ≈ ×1.9 the mean author h-index (+1) |
| Shared authors | 0.192 | 0.114 | 1.69 | 0.090 | 4.3 | 2.8 more shared authors |
| log₁₀ p-value | -0.466 | 0.114 | -4.07 | < 0.001 | -10.5 | ≈ ×440 larger p-value |
Fractional logit (quasi-MLE), n = 451 effects in 357 original papers; standard errors clustered on the original paper. Covariates are z-scored on this model's own estimation sample, so coefficients are comparable within the table. AME = average marginal effect, in percentage points of predicted replication score per +1 SD.
Coefficients at a glance
Points are standardized coefficients; horizontal lines are cluster-robust 95% confidence intervals. ● Model A, ◆ Model B. Hover a point for details.
Reading the model
Author overlap dominates Model A: it is by far the largest coefficient, echoing the by-author-overlap page. Journal rank and mean author h-index retain small negative effects once everything else is held fixed, while impact factor and citations — strongly correlated with both — contribute little of their own. In Model B the original p-value is the strongest predictor in the model, and it absorbs much of what the prestige variables appeared to carry; but Model B rests on a small, unrepresentative subset (initiatives that report exact p-values), so treat it as suggestive. As always, these are observational associations, not causal effects.
Coverage
Source: replications_database_2026_08_01_120356.csv — 8,445 effect rows, of which 8,307 have a recorded result (5,813 original papers). Predictor coverage among those rows: publication year 8,266; journal impact factor 7,486 (after excluding 358 with IF = 0); journal rank 7,593; citations in the first two years 7,163; author h-index 7,108 (first author 7,108, last author 7,061); author overlap 7,771; exactly reported p-value 656. Impact factors and citation counts come from OpenAlex, journal ranks from SCImago, and author h-indexes from SciSciNet; all joins are by DOI or normalized journal name. Statistics on this page are verified against an independent Python implementation (scripts/check_logit.py).