PLDI runs artifact evaluation after paper acceptance, deliberately separating paper review from artifact review, and it accepts "any artifact authors wish to submit, broadly defined" — compilers, proofs, benchmark suites, measurement infrastructure (PLDI 2026 research-artifacts track, read 2026-07-08; AE co-chairs for 2026 were Raphaël Monat and Qirun Zhang). Badges appear on the published PACMPL article, so the artifact is part of the paper's permanent record.
| Badge | 2026-cycle criterion | What actually earns it |
|---|---|---|
| Functional | Artifact supports the paper's claims and works as documented | An evaluator reproduces your headline flow from the README alone |
| Reusable | Only awarded to Functional artifacts judged especially well packaged, documented, and designed for future research | Extension points documented; someone could swap in a new benchmark or pass |
| Available | Automatic once the complete artifact is archived publicly in an archival location | A Zenodo snapshot with a DOI — a lab GitHub link does not qualify |
Two structural kindnesses in the 2026 rules: artifacts are requested only after acceptance, and there is no artifact camera-ready deadline because Zenodo accepts new versions at any time. Use that slack for polish, not procrastination — the evaluation itself still runs on the AE committee's timetable.
Your evaluator has a laptop, a deadline, and no institutional access to your cluster. Design for that:
artifact/
README.md # claims covered, est. runtimes, kick-the-tires steps
LICENSE
Dockerfile # pinned base image, pinned compiler versions
claims.md # paper claim -> script -> expected output, one row each
scripts/
smoke.sh # <10 min: builds, runs one benchmark, checks one number
reproduce_all.sh # full run with per-experiment time estimates
benchmarks/ # provenance + versions of every program measured
results/expected/ # our outputs, for diffing
Rules of thumb that decide badges:
make check
verify the exact theorem names cited in the paper.The evaluation protocol inside the artifact must match what
pldi-experiments put in the paper: same warmup discipline, same repetition
counts, same statistics. An artifact that reruns 3 iterations when the paper
reports 30-run confidence intervals invites exactly the doubt AE exists to
dispel.
Kick-the-tires/rebuttal phases, badge wording, submission format (VM vs container), and the AE timetable are reset per edition — reread the current research-artifacts track page before packaging.
[Badge target] Available / +Functional / +Reusable
[Smoke test] exists? runtime? passes in a fresh container?
[Claim map] paper table/claim -> script -> expected output (n rows)
[Pinning] compilers / benchmarks / base image / proof assistant
[Archive] Zenodo DOI minted? version matches camera-ready?