Skill UI

A comprehensive guide for designing rigorous Monte Carlo simulation studies, particularly for methods-heavy academic submissions (e.g., SMR). This guide covers specifying the Data Generating Process (DGP) space, selecting non-negotiable competitor methods, choosing appropriate performance metrics (e.g., power, bias, coverage), and critically reporting the limitations and failure regimes of the proposed method to ensure maximum credibility and reproducibility.

This guide provides comprehensive best practices for designing and auditing cloud system evaluations, particularly for academic submissions. It mandates that claims must be supported by measured evidence from real or realistic deployments, moving beyond simple simulations. Key metrics emphasized include tail latency (p99), operational cost modeling, and verifiable behavior under scale and multi-tenancy.

A comprehensive guide for authors submitting technical papers to top-tier conferences, detailing how to effectively partition content between the main paper body and supplementary artifacts. It emphasizes the critical rule that all decision-making evidence (e.g., headline results, key performance metrics) must reside within the limited page budget of the main submission, ensuring the work is judgeable even without accessing raw data.

A comprehensive guide for writing the evaluation section of a systems paper for top-tier conferences. It teaches authors how to map every technical claim to verifiable experiments, establish fair state-of-the-art baselines, utilize mixed workloads (microbenchmarks, E2E applications, and traces), and honestly report critical metrics like performance tails, variance, and overheads. The goal is to maximize credibility and withstand rigorous peer review.

A comprehensive guide for authors aiming to submit manuscripts to 'Studies in Science of Science.' It details the journal's focus areas, such as science metrics, knowledge networks, and technology policy, emphasizing the need for deep theoretical and methodological contributions. The skill helps authors optimize structure, avoid common rejection traps, and ensure adherence to academic publishing standards.

This comprehensive guide outlines best practices for designing and auditing empirical evaluations of formal verification tools and algorithms. It stresses the use of shared, community-accepted benchmarks, ensuring fair comparison against the strongest baselines with equal resources, meticulous reporting of all metrics, validating results for soundness, and guaranteeing full reproducibility on clean virtual environments.

A comprehensive tool for assessing whether an econometric manuscript is suitable for submission to The Econometrics Journal (EctJ). It guides authors through evaluating methodological novelty, theoretical rigor, structural compliance, and adherence to high academic standards, covering everything from new estimators and inference procedures to house style and potential desk-reject risks. Use this skill when targeting a highly specialized economic journal or needing to re-frame an applied study into a theoretical methods contribution.

A comprehensive guide for designing, auditing, and reporting advanced statistical and causal inference experiments. It emphasizes moving beyond simple accuracy metrics to quantify uncertainty, model robustness, and causal mechanisms. Covers metrics like coverage, KL divergence, and regret, and outlines best practices for seeding, compute fairness, and creating standardized reporting blocks (e.g., UAI standards).

This guide provides a rigorous methodology for achieving scientific reproducibility in technical and user experience papers (e.g., UIST). It details three essential 'ledgers': the Build Ledger (pinning software, hardware, and environment dependencies), the Measurement Ledger (defining precise protocols for metrics like latency and accuracy), and the Study Ledger (documenting human evaluation tasks). Following these protocols ensures that external researchers can accurately rebuild the artifact and validate reported results.

A detailed guide for revising academic papers, specifically tailored for systems and database conferences like VLDB. It emphasizes establishing a strong 'page-one contract,' quantifying all performance claims with scoped metrics and baselines, and maintaining rigorous candor regarding design trade-offs and terminology throughout the manuscript.

A comprehensive guide for strengthening the reproducibility of academic papers, especially those involving systems claims (e.g., power usage, latency). It details maintaining a 'recipe ledger' for all reported metrics, ensuring constraint-aware measurement of environmental variables, and managing data/artifact consistency across the entire two-round revision process.

This guide provides a structured framework for conducting and presenting robust checks in econometric manuscripts, particularly relevant for studies using developing-country data. It moves beyond simple appendix dumping by organizing tests around potential identification threats and specific data-quality risks (e.g., measurement error, coverage gaps). It helps ensure the core findings are load-bearing against sophisticated academic scrutiny.