Skill UI

A comprehensive guide for ensuring the reproducibility of Information Retrieval (IR) research, essential for SIGIR submissions. It details best practices for pinning data collection versions, index settings, neural model hyperparameters, and managing non-determinism. It helps researchers document artifacts—not just promises—to validate reported gains.

This skill guides authors on positioning information retrieval (IR) work for the Text REtrieval Conference (TREC). It functions as a venue-selection and manuscript re-framing tool, ensuring the submission aligns with TREC's focus on shared evaluation tracks, test collections, and NIST-style benchmarking. It helps authors structure evidence, understand reviewer expectations, and distinguish the work from general CS conference submissions.

This guide provides a comprehensive framework for designing rigorous and defensible evaluation sections for machine learning and information retrieval papers, particularly for top-tier conferences like WSDM. It covers the four critical evidence quadrants—Effectiveness, Validity, Efficiency, and Robustness—and details necessary protocols, such as temporal splits, proper baseline selection (including heuristics), ablation studies, and hardware-aware performance reporting. Use this when auditing or writing the experimental section of a research paper.