Skill UI

This guide outlines essential strategies for ensuring that complex physical or scientific measurements are fully reproducible by external parties, long after the original testbed is decommissioned. It mandates capturing detailed provenance data, including energy measurement methods, hardware specifications, firmware toolchains, environmental conditions, and ground-truth protocols, thus transforming raw data into verifiable, sharable scientific artifacts.

A comprehensive guide for submitting networking research artifacts for SIGCOMM conferences. It dictates how to package code, downscale topologies, substitute traces, and pin environments so that reviewers can independently verify and reproduce the paper's headline results, establishing verifiable scientific claims.

A comprehensive methodology for rigorous scientific research in operating systems and complex systems. This guide details how to pin environments (OS, firmware, hardware), automate data capture, and script all figures to ensure that experimental results are auditable and fully reproducible, protecting findings against review challenges.

A comprehensive guide for researchers submitting academic papers, detailing the standards required for scientific reproducibility. It covers fixing experimental environments (toolchains, datasets, hardware), differentiating result sources (simulation vs. physical measurement), statistical reporting best practices, and providing clear guidelines for code and data availability statements.

This guide outlines the rigorous process for submitting and evaluating research artifacts for academic conferences (TACAS/ETAPS). It details the mandatory two-round evaluation system, covering tool-demonstration papers and general research case studies. Authors must package their work to achieve badges like Available, Functional, and Reusable, ensuring maximum reproducibility and environmental compatibility on a clean virtual machine.

This comprehensive guide outlines best practices for designing and auditing empirical evaluations of formal verification tools and algorithms. It stresses the use of shared, community-accepted benchmarks, ensuring fair comparison against the strongest baselines with equal resources, meticulous reporting of all metrics, validating results for soundness, and guaranteeing full reproducibility on clean virtual environments.

This guide details advanced best practices for achieving scientific reproducibility in probabilistic modeling and deep learning research. It mandates rigorous disclosure of seeds, hyperparameters, diagnostic traces (e.g., R-hat, ELBO), and compute environments to ensure that all claims made in academic papers are verifiably backed by traceable evidence. It elevates reproducibility from a simple code run to a comprehensive audit.

This guide models the entire lifecycle of academic conference submissions, detailing stages like bidding, review writing, author response, and decision-making. It helps researchers strategically plan their work to maximize impact, preemptively address potential weaknesses, and optimize their research communication strategy in a competitive academic environment.

This guide provides a rigorous methodology for achieving scientific reproducibility in technical and user experience papers (e.g., UIST). It details three essential 'ledgers': the Build Ledger (pinning software, hardware, and environment dependencies), the Measurement Ledger (defining precise protocols for metrics like latency and accuracy), and the Study Ledger (documenting human evaluation tasks). Following these protocols ensures that external researchers can accurately rebuild the artifact and validate reported results.

This guide covers the mandatory Open Science appendix for high-level security research. It instructs authors on enumerating all artifacts (code, datasets, configurations), documenting data withholding decisions, and implementing engineering controls (like environment pinning and version tracking) to ensure that published results are independently and reliably reproducible.

A comprehensive guide to vcpkg, a native C/C++ package manager. It covers project setup, dependency management, and cross-compiling for robust builds. The guide details best practices for Manifest mode, CMake integration, and Visual Studio setup, ensuring reproducible and isolated build environments.

A comprehensive guide detailing the standards for scientific reproducibility in visualization research, particularly for IEEE VIS submissions. It outlines the Graphics Replicability Stamp Initiative (GRSI) requirements, covering how independent volunteers must reproduce results using provided code, data, and environments (e.g., Docker containers). Learn how to package your work—including determinism aids, mapping tables, and licensed data—to pass rigorous, clean-machine testing.