Skill UI

A comprehensive guide for packaging and submitting artifacts for Information Retrieval (IR) conferences (like SIGIR). It details the strict requirements for reproducibility, including mandatory components like TREC-format run files, qrels, and index recipes. The process covers repository structure, anonymization for review, handling data provenance, and ensuring that results can be audited by any reviewer using standard community tooling.

A comprehensive audit checklist designed for authors submitting papers to the SIGIR conference. It guides users through critical pre-submission checks, covering adherence to specific full/short paper formats, mandatory anonymization techniques (e.g., stripping system names, log provenance), cross-track submission rules, AI disclosure requirements, and common pitfalls that lead to desk rejection.

A comprehensive guide for researchers developing systems or algorithms, detailing the necessary level of evidence required for performance claims (e.g., bounds, real-world performance, learning guarantees). It emphasizes scientific rigor, statistical reporting (confidence intervals), assumption testing, and data provenance to ensure findings are robust and reproducible in academic settings.

Provides comprehensive guidelines for achieving high levels of scientific reproducibility for academic papers, particularly those in systems metrics. It covers mapping claims to verifiable evidence, ensuring simulators are seeded and deterministic, and documenting measurement provenance to allow reviewers to independently verify all reported results.

A comprehensive guide for authors in data systems and related fields on achieving rigorous, verifiable reproducibility for academic papers. It mandates detailing the provenance of every claimed result by pinning five critical layers: code version, configuration parameters, dataset source, workload characteristics, and hardware specifications. Best practices include reporting distributions (e.g., percentiles, variance) and maintaining a detailed 'repro debt ledger' to withstand intense peer review.

This guide outlines the best practices for preparing system artifacts for academic evaluation, following the ACM Artifact Review and Badging scheme. It details how authors must ensure reproducibility, especially for tail latency and cost results, by providing scaled-down testbeds, comprehensive documentation, and clear provenance tracking to satisfy conference evaluators.

This comprehensive skill guides authors submitting research to The Burlington Magazine. It outlines the rigorous academic standards required for object-based art history, focusing on connoisseurship, attribution, and provenance. It ensures manuscripts are tightly argued using primary source evidence, technical findings, and close visual examination, helping authors align their work with the journal's specific evidentiary bar and house style.

A comprehensive guide for designing and auditing visualization experiments for high-stakes academic submissions (e.g., IEEE VIS). It details how to match evidence—such as controlled studies, power analyses, and benchmarks—to the specific type of claim (perceptual, algorithmic, system). Learn best practices for statistical rigor, CVD-safe encoding, and maintaining full provenance to ensure your findings are scientifically sound and trustworthy.

This guide details best practices for achieving high standards of reproducibility in visual analytics papers, focusing on open materials, verifiable code, and rigorous documentation. It covers mapping figures to data sources, preregistering studies, documenting data provenance, and defining the level of reproducibility achieved (e.g., turnkey, scripted).

This comprehensive guide outlines the rigorous standards required for achieving experimental reproducibility in systems research papers (e.g., VLDB). It instructs authors on disclosing critical context, including hardware specifications, software configurations, data provenance, workload patterns, and handling measurement variance. Adhering to these guidelines ensures that all performance claims are fully traceable from raw data to published figures, meeting the highest standards of academic review.

A comprehensive guide for authors submitting papers to web conferences. It details how to structure empirical claims, ensuring evidence matches the claim's scope (scale, freshness, generalizability). The guide covers avoiding methodological flaws like temporal or popularity leakage, selecting datasets with proper provenance, and performing rigorous ablations and statistical analysis for maximum academic rigor.

A comprehensive guide for preparing replication and transparency packages for World Development manuscripts. It provides checklists and best practices for both quantitative (code reproducibility, data provenance, de-identification) and qualitative (evidence-to-claim trail, methods appendix) research. It addresses ethical constraints and the unique challenges of double-anonymized review.