Use this after ISSTA reviews are released. ISSTA's response step feeds a decision that is not just accept/reject: a first-round paper can be told to submit a Major Revision, so the response and the revision plan are two connected artifacts. Reopen the current author-response and major-revision instructions before drafting, because the dates and mechanics are cycle-specific.
| Pushback | What it signals | ISSTA-ready reply |
|---|---|---|
| "Evaluated only on your own subjects" | External-validity doubt | Point to the labelled benchmark and its provenance, or commit to adding an established one (Defects4J, real CVEs) in revision |
| "Baseline is not configured fairly" | Comparison-validity doubt | Cite the equal-budget setup, or concede and re-run the baseline under the reviewer's configuration |
| "Improvement is within noise" | Statistical-significance doubt | Reference the non-parametric test and effect size, or add repeated runs in revision |
| "Cannot tell if the artifact reproduces this" | Verifiability doubt | Point to the entry script and logged outputs, or fix the README before the artifact deadline |
When the outcome is Major Revision, convert every review comment into a tracked row: the comment, the change, where it lands in the paper, and its status. The resubmission's cover letter walks this ledger; reviewers who see each of their points addressed in order revise upward.
[R2.3] "No comparison to reversed-order rerunning."
-> Added reversed-order baseline at equal budget; Table 2, rows 4-6. STATUS: done
[R1.1] "Shared-state model undefined for native calls."
-> Scoped native state out explicitly; Section 3.1, threats. STATUS: done
[R3.2] "Overhead not reported."
-> Added per-project overhead; Table 3. STATUS: in progress
[Priority issue] <reviewer concern>
[Decision dimension] soundness / evaluation / scope / related-work / verifiability
[Reply type] rebuttal correction / promised change
[Draft reply] <anonymous, evidence-anchored text>
[Evidence anchor] <table/figure/section/artifact path>
[Revision-ledger row] <comment -> change -> location -> status>