Skill UI

This guide provides comprehensive standards for designing, executing, and reporting systems experiments for top-tier academic venues (like ICDE). It emphasizes methodological rigor, including mandatory ablation studies, reporting variance and tail latency (p99), demonstrating scaling trends (curves over points), disclosing cost trade-offs, and ensuring full hardware and software reproducibility to withstand adversarial review.

A comprehensive guide for designing and reporting rigorous experimental results in robotics research. It details the 'evidence ladder' (from simulation to real-world deployment), statistical best practices (e.g., sample size, confidence intervals), ensuring baseline fairness, and quantifying sim-to-real transfer gaps. Essential reading for researchers submitting to top-tier robotics conferences.

A comprehensive methodology guide for designing and auditing empirical studies in software engineering, specifically focused on maintenance, evolution, and change impact. It outlines best practices for ensuring evidence is proportional to the claim, managing data provenance (SHAs, criteria), handling statistical confounds (survivorship, effect sizes), and integrating rigorous LLM evaluation protocols.

A comprehensive guide for designing and auditing rigorous experiments for sensing systems and information processing claims. It emphasizes the necessity of measuring performance on real hardware (not simulation), accurately instrumenting ground truth, quantifying energy consumption, latency, and physical footprint, and establishing fair, skeptical baselines. This ensures scientific rigor and reproducibility in research submissions.

A comprehensive guide for designing, auditing, and reporting reliable robotic experiments, particularly for major conferences like IROS. It moves beyond simple success rates, detailing how to map every claim to verifiable evidence (trials, simulations, field data). Key areas covered include defining pre-set success criteria, reporting failure taxonomies, ensuring fair baselines, and quantifying the critical sim-to-real gap using advanced small-n statistical methods.

This comprehensive guide provides best practices and an audit checklist for designing robust, real-world experiments in mobile and wireless communication systems. It emphasizes moving beyond simulations, detailing necessary RF setups, quantifiable energy metrics, and reporting statistical distributions to ensure high academic credibility.

A comprehensive guide for submitting research artifacts to the MobiSys Artifact Evaluation Committee (AEC). It details the process for achieving three independent ACM badges (Available, Functional, Results Reproduced). Emphasis is placed on creating fully reproducible, script-first workflows, designing for hardware-optional environments, and maintaining strict anonymity during review.

This comprehensive guide outlines best practices for designing rigorous and convincing technical evaluations for networked systems. It structures the process by requiring experiments to answer specific questions, advising authors to climb the evidence ladder from synthetic benchmarks to real-world deployment. Key concepts include reporting tail latencies (p99), establishing strong baselines, and maintaining meticulous provenance records to ensure reproducibility.

This comprehensive guide outlines the rigorous methodology required for designing high-impact evaluations of parallel systems and algorithms for top-tier conferences. It stresses that contributions must demonstrate the 'twin bar': both concurrency correctness (e.g., linearizability) and measurable scalability (e.g., strong vs. weak scaling). It covers best practices for establishing strong baselines, managing measurement hygiene (variance, pinning), and anticipating advanced reviewer scrutiny in HPC.

This guide provides a comprehensive audit checklist for designing and evaluating sensor-based systems (SenSys). It mandates rigorous, real-world measurements across five core axes: energy efficiency, on-device latency distribution, ground truth accuracy, deployment realism, and embedded AI footprint. It ensures that research results are based on measured hardware performance, not simulations or idealized baselines.

A comprehensive guide for designing and auditing high-stakes networking experiments, particularly for top-tier academic submissions like SIGCOMM. It outlines the 'evidence ladder' from microbenchmarks to live deployment, emphasizing the need to select robust baselines, report full performance distributions (percentiles and variance), and accurately map the mechanism's operational break points.

This comprehensive guide outlines best practices for designing data systems evaluations, ensuring results are robust, reproducible, and convincing to expert reviewers. It covers workload realism, baseline fairness, necessary ablation studies, and advanced metric visualization techniques (e.g., sweeping curves, failure analysis) required for top-tier data systems research.