Skill UI

A comprehensive guide for ensuring the scientific reproducibility of research papers that rely on dynamic web data, such as web crawls, platform APIs, or live system traffic. It defines three reproducibility regimes (Frozen, Decaying, Unreplayable) and provides detailed instructions on documenting data decay, temporal splits, environment configurations, and computational seeds to maintain the rigor of published findings.

This guide outlines the mandatory requirements for authors publishing quantitative or mixed-method research in academic journals. It details the process of creating a complete replication package—including raw data, master scripts, seeds, and extensive README documentation—and depositing it in a trusted data repository (like Dataverse). It ensures scholarly rigor by guaranteeing that published findings are fully reproducible and traceable.

A comprehensive framework for ensuring reproducibility in Web Search and Data Mining (WSDM) research. It addresses unique challenges associated with behavioral data, covering data provenance, temporal splitting discipline, bias assumption reporting, statistical rigor (seeds, variance), and the 'honesty ladder' for non-reproducible results, helping researchers publish robust and verifiable work.