Skill UI

A comprehensive guide for structuring research papers, particularly in computer graphics and visualization fields (like SIGGRAPH). It dictates how to match technical claims (e.g., 'higher quality,' 'faster') to verifiable evidence, including head-to-head comparisons against State-of-the-Art (SOTA), detailed metric reporting (PSNR, SSIM, Hausdorff), performance timing linked to specific hardware, and robust user studies. Essential for achieving academic rigor and pre-empting domain reviewer objections.

This comprehensive guide details the rigorous methodology for designing, executing, and reporting experimental results in Information Retrieval (IR). It covers best practices for selecting appropriate test collections, choosing defensible metrics, performing proper statistical significance testing (e.g., paired t-tests), ensuring baseline fairness, and proactively addressing modern pitfalls like LLM contamination. Essential for publishing high-quality, auditable research in IR/NLP.

A comprehensive guide for researchers developing systems or algorithms, detailing the necessary level of evidence required for performance claims (e.g., bounds, real-world performance, learning guarantees). It emphasizes scientific rigor, statistical reporting (confidence intervals), assumption testing, and data provenance to ensure findings are robust and reproducible in academic settings.

This comprehensive guide outlines best practices for designing data systems evaluations, ensuring results are robust, reproducible, and convincing to expert reviewers. It covers workload realism, baseline fairness, necessary ablation studies, and advanced metric visualization techniques (e.g., sweeping curves, failure analysis) required for top-tier data systems research.

This guide provides comprehensive best practices for designing and auditing cloud system evaluations, particularly for academic submissions. It mandates that claims must be supported by measured evidence from real or realistic deployments, moving beyond simple simulations. Key metrics emphasized include tail latency (p99), operational cost modeling, and verifiable behavior under scale and multi-tenancy.

A comprehensive guide for authors submitting theoretical computer science papers to SODA. It clarifies the distinction between SODA (theory-led) and ALENEX (experiment-led), advising on when and how to incorporate numerical evidence. The protocol details best practices for honest numerics, including instance generation discipline, claim-class labeling (illustration, evidence, motivation), and proper submission routing to ensure academic rigor.

A comprehensive guide for writing the evaluation section of a systems paper for top-tier conferences. It teaches authors how to map every technical claim to verifiable experiments, establish fair state-of-the-art baselines, utilize mixed workloads (microbenchmarks, E2E applications, and traces), and honestly report critical metrics like performance tails, variance, and overheads. The goal is to maximize credibility and withstand rigorous peer review.

This comprehensive guide helps researchers defend their complex study designs—whether they involve lab experiments, large-scale surveys, or qualitative observations—against rigorous methodological critique. It emphasizes establishing a clear, credible link between theoretical mechanisms and empirical evidence, focusing specifically on the structure-individual link. It covers advanced inference techniques, identifying potential confounding variables, and articulating the design's limitations.

A comprehensive guide for authors submitting to high-impact scientific journals. It outlines the essential principles for structuring experiments and presenting evidence, emphasizing claims-evidence matching, fair benchmarking, differentiating result sources (simulation, hardware, theory), and ensuring statistical rigor. Covers best practices across computer science, control, communication, and microelectronics.

This guide outlines the essential principles for structuring scientific manuscripts submitted to academic journals. It details how to correctly allocate core evidence, lengthy proofs, additional experiments, and multimedia content across the main text, appendices, and supplementary materials. The core principle emphasized is that all evidence crucial for judging the paper's scientific contribution must reside within the main body text.

This guide clarifies the role of computation in theoretical computer science submissions, particularly for venues that prioritize theorems over empirical data. It distinguishes between legitimate forms of evidence—such as certified proof steps, constructive object discoveries, and illustrative examples—and inappropriate performance benchmarking. It helps researchers structure their work to align with theorem-based research, ensuring computational elements enhance the core theoretical contribution.

This comprehensive guide outlines best practices for designing and auditing empirical evaluations of formal verification tools and algorithms. It stresses the use of shared, community-accepted benchmarks, ensuring fair comparison against the strongest baselines with equal resources, meticulous reporting of all metrics, validating results for soundness, and guaranteeing full reproducibility on clean virtual environments.