Skill UI

This guide details the requirements for creating a comprehensive replication package for manuscripts submitted to Econometric Society journals. It ensures full reproducibility of all results, including code, data provenance, and parameter settings, adhering to the journal's Data and Code Availability Policy. Key components include a master script, reproducible data, and detailed documentation.

A comprehensive guide for researchers on packaging and documenting NLP artifacts, such as datasets, prompts, and evaluation code, for academic publication. It details best practices for ensuring reproducibility, managing provenance, adhering to licensing standards, and structuring evidence for top-tier conferences like EMNLP.

A comprehensive guide for researchers submitting NLP papers, detailing best practices for achieving high reproducibility. It instructs users to pin model versions, log data provenance, report compute costs, and adhere rigorously to the Responsible NLP checklist. Essential for ensuring academic integrity and enabling rigorous verification of ML experiments.

A comprehensive guide for structuring academic empirical research projects to ensure the full reproducibility of findings. This methodology covers standard folder layouts, master scripting, data provenance documentation, and best practices required for passing rigorous peer review scrutiny, thereby preventing data loss and calculation errors during revisions.

A comprehensive framework for ensuring rigorous scientific reproducibility in systems papers. It details how to establish a full provenance ledger (tracking hardware, software commits, and workloads), manage performance variance through repeated runs and statistical reporting, and implement automated pipelines so that all reported figures can be regenerated from raw data.

A comprehensive guide for establishing academic rigor in AI and data science research. It outlines best practices for achieving reproducibility, emphasizing the mandatory release and documentation of raw data, trained models, and qualitative findings. Key tools include datasheets, model cards, and detailed provenance logging, ensuring that research claims are fully auditable and consistent.

This guide details best practices for submitting research artifacts to academic conferences, following the ACM Artifact Review and Badging scheme. It covers everything from required packaging plans (Docker, README, data provenance) to achieving various levels of validation (Functional, Reusable, Reproduced). Learn how to structure your submission to ensure maximum reproducibility and pass rigorous evaluator checks on a clean machine.

This guide provides a rigorous framework for designing and auditing empirical studies in software engineering (SE). It emphasizes that evidence must proportionally match the claims made, covering topics like using real subject systems, establishing fair baselines, calculating SE-standard statistics (effect sizes, CIs), and ensuring thorough provenance, especially for LLM-based work. It helps researchers strengthen their research methodology before publication.

A comprehensive guide for researchers ensuring the highest level of open-science reproducibility for academic submissions (e.g., FSE). It details how to map every research claim (numbers, techniques) to verifiable evidence, pin technical provenance (SHAs, model IDs), manage data availability statements, and achieve defined degrees of reproducibility (Turnkey, Scripted).

This guide instructs authors on creating mandatory Data Availability Statements and building comprehensive replication materials for academic manuscripts. It covers best practices for data provenance, code management, and handling restricted data, ensuring research integrity and reproducibility.

A comprehensive guide for auditing and improving the scientific rigor of hardware architectural evaluations (e.g., HPCA style). It emphasizes establishing a clear 'fidelity contract' defining tools, scope, and configurations. It requires that all performance claims are backed by credible instruments, mandates capturing full machine state provenance, ensuring honest baseline tuning, and reporting results via per-workload distributions rather than single means.

This guide details the process of packaging artifacts for academic conferences (ICLR), ensuring that research claims are fully reproducible and reviewable. It emphasizes providing minimal reproduction paths, maintaining data provenance, rigorously anonymizing metadata to protect authors' identities, and creating robust, permanent records for the scholarly community.