A comprehensive strategic guide for authors targeting the Conference and Labs of the Evaluation Forum (CLEF). It helps frame IR/NLP research by defining the appropriate contribution type, evidence bar, and rigorous evaluation methods required for top-tier academic submission, ensuring the paper aligns with the venue's specific community expectations.
Conference and Labs of the Evaluation Forum (CLEF)
Conference positioning
Conference and Labs of the Evaluation Forum (CLEF) is a top computer-science conference venue for IR/NLP evaluation campaigns, shared tasks, multilingual retrieval, and benchmark methodology. It rewards a paper whose value comes from benchmark design, shared-task analysis, or rigorous evaluation. Treat this skill as a fit / venue-selection / re-framing tool for conference submission strategy, not as a substitute for the current year's CFP, author kit, ethics policy, or submission portal.
Because CS conferences change deadlines, templates, page limits, review workflow, artifact rules, AI-use policy, and rebuttal formats every cycle, always verify the live official instructions before making a submission-ready recommendation. Start from the official source anchor recorded for this venue in ../../resources/conference-roster.md and ../../resources/official-source-map.md.
When to trigger
The author names CLEF / Conference and Labs of the Evaluation Forum as the target venue.
A manuscript in IR/NLP evaluation campaigns needs a conference-fit read before being formatted or submitted.
The paper must be re-framed from journal style or arXiv style into a selective CS conference narrative.
The author needs an evidence-gap, anonymity, artifact, rebuttal, or re-routing diagnosis for this venue.
Best submissions make a precise contribution type visible: algorithm, theorem, system, dataset, benchmark, empirical finding, design artifact, tool, or socio-technical analysis.
The paper should explain why the result matters to CLEF's reviewers, not just why it is interesting to the authors' lab or product context.
Position related work against the most recent conference-cycle papers in this venue and its closest siblings; stale comparisons are a common early-review weakness.
If the contribution is interdisciplinary, state which part is CS research and which part is domain evidence.
Venue-specific calibration
Reviewer lens: Treat CLEF as a evaluation forum venue whose reviewers expect the scope and evidence to match its own community. Do not submit a generic CS paper until the introduction names the exact subcommunity, contribution type, and proof or empirical standard.
Contribution hook to foreground: the venue-specific contribution bar.
Scope vocabulary to use naturally in the abstract and introduction: IR/NLP evaluation campaigns, shared tasks, multilingual retrieval, and benchmark methodology.
Official anchor domain: www.clef-initiative.eu. Quote annual rules only after opening that source and the current-year CFP/author kit.
Close-neighbor routing guardrail
Use this profile only when the manuscript's central contribution is genuinely in evaluation
forum and the author can say why CLEF reviewers are the primary audience, not merely a
convenient deadline.
Closest roster neighbors to compare before final routing: acm-sigir-conference-on-human- information-interaction-and-retrieval (CHIIR), acm-ieee-joint-conference-on-digital- libraries (JCDL), acm-conference-on-designing-interactive-systems (DIS), acm- interactive-surfaces-and-spaces (ISS). Break ties by contribution type, evidence shape,
reviewer community, and the current official CFP from www.clef-initiative.eu.
Method & evidence bar
Use task-appropriate baselines, multiple datasets or languages when the claim is broad, and error analysis that explains model behavior.
For LLM work, control for data leakage, prompt sensitivity, evaluation contamination, and human-evaluation reliability.
For resources, document annotation, licensing, demographics, quality control, and intended use.
For CLEF, the evidence must support the venue-specific signature: a paper whose value comes from benchmark design, shared-task analysis, or rigorous evaluation.
Include limitations, negative results, compute/resource reporting, data provenance, and ethics details when they affect the claim.
Structure & house style
State the language phenomenon, task, or system behavior before the model name.
Connect examples to measured errors; reviewers dislike anecdotal examples presented as evidence.
Use the current official template exactly; do not guess page limits, font sizes, supplement rules, anonymity exceptions, or camera-ready requirements from old cycles.
The introduction should answer: problem, why now, what is new, why this venue, and what evidence proves the claim.
Put the strongest result in the main paper, not only in the appendix or supplement; reviewers should not have to reconstruct the contribution.
Re-check the current cycle's CFP, author kit, submission system, abstract/paper deadlines, page limits, supplementary-material rules, anonymity policy, dual-submission policy, ethics policy, AI-use policy, artifact/code/data expectations, rebuttal/author-response format, and camera-ready requirements.
Confirm the review workflow and portal: OpenReview / CMT / HotCRP / PCS / START or society portal, as specified for the current cycle.
Check whether accepted papers require in-person presentation, separate registration, artifact badges, proceedings copyright, or post-acceptance release forms.
If the live official instructions conflict with this skill, the official instructions win.
Pre-submission self-check
One sentence states why this manuscript belongs at CLEF, using the venue's scope rather than generic "top conference" language.
The claim is calibrated to the evidence: no broader than the datasets, proofs, systems, user studies, deployments, or threat model support.
Related work includes the nearest current-cycle evaluation forum papers and explains the technical delta.
The paper satisfies the current official template, anonymity, ethics, artifact, and rebuttal requirements.
The main paper is self-contained enough for reviewers to evaluate novelty and correctness without hunting through external links.
Common desk-reject triggers
Evaluation that is only a prompt table or cherry-picked generation examples.
Missing dataset documentation, licensing, or annotation reliability.
Claims of general language understanding from narrow English-only benchmarks.
Formatting, anonymity, dual-submission, external-link, or supplement violations under the current-year policy.
A contribution framed for a neighboring field while giving CLEF reviewers too little technical or empirical substance.
Re-routing decision
If the paper misses CLEF's bar, compare against annual-meeting-of-the-association-for-computational-linguistics / conference-on-empirical-methods-in-natural-language-processing / north-american-chapter-of-the-association-for-computational-linguistics / european-chapter-of-the-association-for-computational-linguistics. Re-route based on contribution type, not prestige: theory to a theory venue, systems to a systems venue, application-heavy work to a domain venue, and early ideas to workshops or shorter tracks when the official CFP supports them.
Output format
[Fit] High / Medium / Low (one-line reason)
[Target] Conference and Labs of the Evaluation Forum (CLEF)
[Contribution type] algorithm / theory / system / dataset / benchmark / empirical / design / security / other
[Main evidence gap] <single most important missing proof, experiment, study, artifact, or policy check>
[Official items to re-check] CFP / author kit / deadline / format / anonymity / ethics / AI-use / artifact / rebuttal / camera-ready
[Top rejection risk] <venue-specific risk>
[Re-route suggestion] <better-matched conference or journal if not a fit>
