Package artifacts for a venue where they are persuasion, not process. The pack found no formal artifact-evaluation track or badge system for current WSDM editions (待核实 each cycle) - the CFP-level expectation is the community norm of "practical yet principled": reviewers reward submissions whose claims a skeptic could re-derive. That means the artifact's job is to be inspectable during review and usable after publication, with no committee to certify it.
Because appendices count against WSDM's page budget and there is no rebuttal in which to hand over materials later, the anonymous repository referenced in the PDF is the only expansion space you get. Build it to be skimmed in ten minutes:
anon-artifact/
├── README.md # 1 screen: claim -> script -> expected output table
├── LICENSE # anonymized placeholder license during review
├── env/ # lockfile or container spec, exact versions
├── data/
│ ├── public/ # download scripts for public benchmarks
│ └── PROPRIETARY.md # honest statement of what cannot be shared and why
├── src/ # training/ranking/mining code, no company paths
├── configs/ # one config per reported table row
└── results/
└── seeds_1-5/ # raw metric dumps behind every table in the paper
Anonymity requirements are stricter than habit: fresh remote with no commit
history, no author handles in configs or notebook metadata, no internal package
registries, no dataset paths like /nfs/companyname/.... An Associate Chair
can see who you are; your reviewers must not.
WSDM's core subject matter - query logs, click data, user graphs, ad interactions - is often legally unshareable. The venue's reviewers know this; what they punish is unverifiable work, not industrial work. Choose a rung and state it explicitly in the paper:
| Rung | What is released | What the paper must then carry |
|---|---|---|
| Full release | Data + code + configs | Just the pointers |
| Sampled/anonymized release | A privacy-scrubbed sample + full code | The scrubbing procedure and how sample results track full-data results |
| Public-benchmark mirror | Code + experiments re-run on public datasets | Both result sets, with the deltas discussed, not hidden |
| Code-only | Pipeline code, no data | Enough dataset statistics that a platform-holder could replicate |
| Nothing sharable | - | A candid limitation; expect a proportional credibility discount |
The public-benchmark mirror rung is the WSDM workhorse: pair the proprietary result with the same method on public data so at least one full row of the evidence is end-to-end reproducible by anyone.
Anything here can go stale or change license - re-verify at packaging time.
Foundation-model-era WSDM papers ("Search with Foundation Models" is in-scope per the 2026 CFP) add artifact surfaces older guidance misses:
With no committee stamping artifacts, reviewers use fast proxies to decide whether the repository is real or decorative. Engineer the proxies:
| Signal | Reads as | Cost to provide |
|---|---|---|
| Config file per reported table row | The numbers came from these runs | Minutes |
| Raw seed-level metric dumps | Variance claims are checkable | Minutes |
A make reproduce-table2 entry point |
Someone actually reran this | An hour |
| Download script for each public dataset | The pipeline is end-to-end | An hour |
Honest PROPRIETARY.md |
The authors know what they cannot prove | Minutes |
| Commit dated one hour before deadline as the only commit | Decorative repo | - avoid |
The last row is about the squeeze: a repository assembled on deadline night
tends to mismatch the paper's numbers, and one mismatch found by a reviewer
discounts the entire artifact. Freeze the repo when experiments freeze
(wsdm-workflow sets this at one week out), not when the PDF does.
At camera-ready time the anonymous mirror becomes the citable artifact: move to the real organization/account, add the actual license, tag the release that matches the camera-ready numbers, and archive a snapshot (e.g., a DOI-issuing archive) so the URL in the ACM DL version outlives your CI. Update the README's claim-to-script table against final camera-ready table numbers - drift between repo and proceedings is the most common post-publication complaint.
One more conversion detail: if the paper used the public-benchmark-mirror rung, keep both pipelines in the public repo permanently - the mirror is what future papers will actually build on, and its issues page becomes your citation engine.
[Artifact tier] full / sampled / public-mirror / code-only / none + stated in paper?
[Ten-minute test] README claim->script->output table present: yes / no
[Anonymity sweep] history / handles / paths / registries: clean or leaks listed
[FM pins] model string+date+decoding params recorded: yes / no / n-a
[Post-acceptance plan] public home, license, tagged release, archival snapshot