Skill UI

A comprehensive guide for researchers preparing their academic artifacts (code, data, results) for the SIGMOD Availability & Reproducibility Initiative (ARI). This document details the criteria for achieving reproducibility badges (Available, Evaluated, Results Reproduced), focusing on coverage, ease of reproduction, flexibility, and portability. It provides best practices for structuring artifact repositories and ensuring hardware honesty for verifiable research claims.

A comprehensive guide for authors in data systems and related fields on achieving rigorous, verifiable reproducibility for academic papers. It mandates detailing the provenance of every claimed result by pinning five critical layers: code version, configuration parameters, dataset source, workload characteristics, and hardware specifications. Best practices include reporting distributions (e.g., percentiles, variance) and maintaining a detailed 'repro debt ledger' to withstand intense peer review.

This guide details the structured process for preparing and evaluating research artifacts for major academic conferences like SOSP. It covers badge targeting (Functional, Reusable), cooperative evaluation expectations, and best practices for packaging system-level artifacts, including defining tiers (runnable, provided-hardware, documented-only) and conducting 'kick-the-tires' smoke tests.

A comprehensive methodology for rigorous scientific research in operating systems and complex systems. This guide details how to pin environments (OS, firmware, hardware), automate data capture, and script all figures to ensure that experimental results are auditable and fully reproducible, protecting findings against review challenges.

A comprehensive guide for authors submitting to high-impact scientific journals. It outlines the essential principles for structuring experiments and presenting evidence, emphasizing claims-evidence matching, fair benchmarking, differentiating result sources (simulation, hardware, theory), and ensuring statistical rigor. Covers best practices across computer science, control, communication, and microelectronics.

A comprehensive guide for researchers submitting academic papers, detailing the standards required for scientific reproducibility. It covers fixing experimental environments (toolchains, datasets, hardware), differentiating result sources (simulation vs. physical measurement), statistical reporting best practices, and providing clear guidelines for code and data availability statements.

A comprehensive guide detailing how to package and present research artifacts—including code, hardware designs, and datasets—for major academic conferences. It outlines best practices for both initial review-time evidence and long-term public reuse, ensuring reproducibility and maximizing the chances of adoption by future builders.

This guide provides a rigorous methodology for achieving scientific reproducibility in technical and user experience papers (e.g., UIST). It details three essential 'ledgers': the Build Ledger (pinning software, hardware, and environment dependencies), the Measurement Ledger (defining precise protocols for metrics like latency and accuracy), and the Study Ledger (documenting human evaluation tasks). Following these protocols ensures that external researchers can accurately rebuild the artifact and validate reported results.

This comprehensive guide manages the end-to-end project lifecycle for submitting a systems paper to the UIST conference. It structures the demanding timeline, detailing milestones from abstract submission and hardware development to video production, rebuttal, and final camera-ready submission, while also providing crucial risk mitigation strategies.

This comprehensive guide outlines the rigorous standards required for achieving experimental reproducibility in systems research papers (e.g., VLDB). It instructs authors on disclosing critical context, including hardware specifications, software configurations, data provenance, workload patterns, and handling measurement variance. Adhering to these guidelines ensures that all performance claims are fully traceable from raw data to published figures, meeting the highest standards of academic review.

This guide provides a comprehensive framework for designing rigorous and defensible evaluation sections for machine learning and information retrieval papers, particularly for top-tier conferences like WSDM. It covers the four critical evidence quadrants—Effectiveness, Validity, Efficiency, and Robustness—and details necessary protocols, such as temporal splits, proper baseline selection (including heuristics), ablation studies, and hardware-aware performance reporting. Use this when auditing or writing the experimental section of a research paper.

Cirq is a powerful, open-source Python framework developed by Google Quantum AI. It allows users to design, simulate, and execute complex quantum circuits. It supports advanced features including parameterized circuits, noise modeling, circuit transformation, and seamless integration with multiple quantum hardware backends (e.g., IonQ, Google Quantum, Pasqal).