Skills Data Science Validate Research Replication Package

Validate Research Replication Package

v20260724
rt-replication-package
This tool assists researchers in assembling and rigorously validating a complete replication package for academic submission. It checks the master script, pinned environment, data plan, and methodology against specific journal policies, ensuring that every reported number is reproducible and deterministic, preventing 'my machine' errors.
Get Skill
476 downloads
Overview

Replication Package (rt-replication-package)

The step that now blocks acceptance at the top (AEA / JF / Management Science Data Editors re-run your code and check every number). Full manifest + validation checklist: shared-resources/empirical-methods/replication-package.md.

When to trigger

  • Approaching final submission, or a conditional accept with a data-and-code requirement.
  • A Data Editor / verification service needs the package.

What it does

  1. Read the venue policy from the target pack's resources/official-source-map.md (code at submission vs. acceptance? Data Editor? AsCollected page? restricted-data rule? hosting?). Match it exactly.
  2. Assemble the manifest: master script (raw → all exhibits), ordered steps (clean → construct → estimate → robustness → tables), pinned environment, README/roadmap, data (or pseudo/synthetic data for restricted access + access instructions), the script→exhibit output map, and the disclosure statement.
  3. Validate with the checklist: runs clean end-to-end on a fresh machine; every reported number reproduced (body + appendix); seeded/deterministic; restricted-data plan; output map complete; venue policy satisfied.

Hard rules

  1. Every reported number traces to a script step (body and appendix).
  2. Deterministic — pin versions, seed RNGs.
  3. Restricted data → pseudo/synthetic data + access instructions, never confidential data committed.
  4. Match the venue's exact policy from the source-map; re-verify before submission.
  5. Keep it in sync with the execution bridge — when a revision re-runs an estimate, regenerate the affected outputs.

Output format

【Venue policy】(code timing, Data Editor, restricted-data rule, hosting)
【Manifest】master · steps · env · README · data · output-map · disclosure — status each
【Validation】end-to-end ✓/✗ · all numbers reproduced ✓/✗ · seeded ✓/✗ · restricted-data plan ✓/✗
【Blocking gaps】what would fail a Data-Editor check, ranked
【Ready?】GO / NOT-YET — deciding items

Anti-patterns

  • "Works on my machine" (absolute paths, unpinned versions, manual steps); reported numbers the script doesn't reproduce; committing confidential data; leaving assembly to post-accept crunch.
Info
Category Data Science
Name rt-replication-package
Version v20260724
Size 2.8KB
Updated At 2026-07-29
Language