Skill UI

A comprehensive guide for designing and auditing high-stakes networking experiments, particularly for top-tier academic submissions like SIGCOMM. It outlines the 'evidence ladder' from microbenchmarks to live deployment, emphasizing the need to select robust baselines, report full performance distributions (percentiles and variance), and accurately map the mechanism's operational break points.

A comprehensive guide for structuring research papers, particularly in computer graphics and visualization fields (like SIGGRAPH). It dictates how to match technical claims (e.g., 'higher quality,' 'faster') to verifiable evidence, including head-to-head comparisons against State-of-the-Art (SOTA), detailed metric reporting (PSNR, SSIM, Hausdorff), performance timing linked to specific hardware, and robust user studies. Essential for achieving academic rigor and pre-empting domain reviewer objections.

This comprehensive guide details the rigorous methodology for designing, executing, and reporting experimental results in Information Retrieval (IR). It covers best practices for selecting appropriate test collections, choosing defensible metrics, performing proper statistical significance testing (e.g., paired t-tests), ensuring baseline fairness, and proactively addressing modern pitfalls like LLM contamination. Essential for publishing high-quality, auditable research in IR/NLP.

A comprehensive guide for researchers developing systems or algorithms, detailing the necessary level of evidence required for performance claims (e.g., bounds, real-world performance, learning guarantees). It emphasizes scientific rigor, statistical reporting (confidence intervals), assumption testing, and data provenance to ensure findings are robust and reproducible in academic settings.

This comprehensive guide outlines best practices for designing data systems evaluations, ensuring results are robust, reproducible, and convincing to expert reviewers. It covers workload realism, baseline fairness, necessary ablation studies, and advanced metric visualization techniques (e.g., sweeping curves, failure analysis) required for top-tier data systems research.

This guide provides comprehensive best practices for designing and auditing cloud system evaluations, particularly for academic submissions. It mandates that claims must be supported by measured evidence from real or realistic deployments, moving beyond simple simulations. Key metrics emphasized include tail latency (p99), operational cost modeling, and verifiable behavior under scale and multi-tenancy.

A comprehensive guide for authors submitting to SODA (ACM-SIAM Symposium on Discrete Algorithms). It clarifies the rules for including computational evidence, distinguishing between proofs relying on computation, supporting implementations, and full experimental contributions. It emphasizes using verifiable certificates and structuring the submission for rigorous, independent review.

A comprehensive guide for authors submitting theoretical computer science papers to SODA. It clarifies the distinction between SODA (theory-led) and ALENEX (experiment-led), advising on when and how to incorporate numerical evidence. The protocol details best practices for honest numerics, including instance generation discipline, claim-class labeling (illustration, evidence, motivation), and proper submission routing to ensure academic rigor.

A comprehensive guide for writing the evaluation section of a systems paper for top-tier conferences. It teaches authors how to map every technical claim to verifiable experiments, establish fair state-of-the-art baselines, utilize mixed workloads (microbenchmarks, E2E applications, and traces), and honestly report critical metrics like performance tails, variance, and overheads. The goal is to maximize credibility and withstand rigorous peer review.

A comprehensive methodology for rigorous scientific research in operating systems and complex systems. This guide details how to pin environments (OS, firmware, hardware), automate data capture, and script all figures to ensure that experimental results are auditable and fully reproducible, protecting findings against review challenges.

This comprehensive guide helps researchers defend their complex study designs—whether they involve lab experiments, large-scale surveys, or qualitative observations—against rigorous methodological critique. It emphasizes establishing a clear, credible link between theoretical mechanisms and empirical evidence, focusing specifically on the structure-individual link. It covers advanced inference techniques, identifying potential confounding variables, and articulating the design's limitations.

A comprehensive guide for authors submitting to high-impact scientific journals. It outlines the essential principles for structuring experiments and presenting evidence, emphasizing claims-evidence matching, fair benchmarking, differentiating result sources (simulation, hardware, theory), and ensuring statistical rigor. Covers best practices across computer science, control, communication, and microelectronics.