Skill UI

A comprehensive guide for designing, executing, and reporting high-stakes research studies in Software Engineering (SE), tailored to meet the stringent standards of top academic conferences like ICSE. It covers best practices for evidence-claim matching, advanced statistical rigor (non-parametric tests, effect sizes), handling AI model ablations, and implementing threats-driven design to ensure findings are robust, valid, and maximally reproducible.

This comprehensive guide provides best practices for designing, running, and reporting computational experiments for academic journals. It teaches authors how to make performance claims scientifically defensible by addressing potential referee criticisms (e.g., tuning artifacts, seed luck). It covers advanced statistical techniques (performance profiles, nonparametric tests) and details the full structure required for a complete, verifiable GitHub data deposit.

This guide provides comprehensive guidelines for designing, conducting, and auditing empirical studies in software engineering. It emphasizes achieving scientific rigor by matching every claim to specific, measured evidence, ensuring fair baselining (equal budget), and applying appropriate advanced statistics (non-parametric tests, effect sizes). Essential reading for researchers submitting papers based on performance evaluations.