For every causal claim, write the policy interpretation in LATE/ATT/descriptive terms:
If the policy sentence needs a broader population than the estimand supports, narrow the claim before submission.
For each causal design, pre-write the most damaging reviewer objection and the table or paragraph that answers it:
Threat | Why plausible here | Evidence already in paper | Missing evidence | Claim to narrow
This matrix prevents overclaiming. If the missing evidence cannot be produced, the correct repair is often to narrow the estimand or policy interpretation, not to add another robustness table. JHR referees will usually accept a precise local estimate more readily than a broad policy claim unsupported by the design.
| Design | First probe | Evidence that usually settles it |
|---|---|---|
| Staggered DID | Pre-trends and forbidden TWFE comparisons | Event study from a heterogeneity-robust estimator; honest-bounds sensitivity on the pre-period |
| RDD at an eligibility cutoff | Manipulation and sorting at the threshold | Density test at the cutoff, covariate smoothness, donut estimates |
| IV from policy variation | First-stage strength and exclusion stories | Per-instrument first stage with effective F; reduced form shown; weak-IV-robust CI |
| School/charter lottery | Differential attrition and re-application | Attrition by win/loss, balance within lottery strata, bounds for missing outcomes |
| Shift-share / Bartik | Shock vs. share identification | State which component is exogenous; exposure-robust SEs |
The cluster question cuts across all rows: inference must sit at the level where treatment was assigned, and the paper should say how many effective clusters remain after fixed effects absorb variation.
Illustrative RD: a selective public high school admits applicants scoring at or above 70 on a composite exam; outcome is college enrollment (numbers invented to show the decision rules):
Estimate and audit the design, don't only describe it. Full map:
execution-with-mcp. JHR is labor/education economics — program evaluation with selection; DiD/IV/RDD and the selection objection are central.
detect_design → recommend → fit with as_handle=true → audit_result.callaway_santanna / sun_abraham +
bacon_decomposition + honest_did_from_result); IV (effective_f_test +
anderson_rubin_ci); RDD (rdrobust + mccrary_test).romano_wolf for many-outcome control.oster_delta / sensemakr for observational claims.Report the magnitude in interpretable units; route the full battery to the appendix. A run end-to-end (synthetic data, real returns) is in the JF execution walkthrough.
[Claim] causal / descriptive / decomposition
[Design] RCT / DID / RDD / IV / event study / other
[Main threat] ...
[Design defense] ...
[Reconciliation needed] ...
[Next step] jhr-data-analysis