技能 编程开发 ASE学术产物评估指南

ASE学术产物评估指南

v20260724
ase-artifact-evaluation
本指南详细说明了如何将已接受的学术研究工具和数据,转化为符合ASE会议要求的、高质量、可复现、可重用的展示包。它指导用户掌握获取“可获取产物”和“可重用产物”徽章的标准,确保研究成果具有持久性和可访问性。
获取技能
140 次下载
概览

ASE Artifact Evaluation

Convert the accepted paper's package into badges. ASE runs an Artifact Evaluation track offering the Artifacts Available and Artifacts Reusable badges (ACM scheme). Because ASE proceedings are indexed in both IEEE Xplore and the ACM Digital Library, an earned badge appears on the paper's front page in both. Evaluation happens on the track's own deadline, separate from the research-track notification — stage the package before then.

The two badges (verify the current call)

  • Artifacts Available — the artifact is placed in a publicly accessible archival repository with a DOI (Zenodo, figshare, Software Heritage, or an institutional/ACM repository). A personal GitHub link alone is not archival; mint a DOI.
  • Artifacts Reusable — the artifact significantly exceeds minimal functionality: it is carefully documented and well-structured so a third party can reuse the tool, not merely reproduce your tables. This is the higher bar and where automated-SE tools usually need the most work.
  • Whether Functional and Results Reproduced badges are also offered at a given edition is 待核实 — confirm on the current Artifact Evaluation call.

From the submission artifact to the badge artifact

The review-time (anonymized) artifact and the badge artifact are the same package matured. After acceptance you can de-anonymize it, but the substance should already be there if you followed ase-reproducibility.

[De-anonymize]  restore the real tool name, authors, repository, license.
[Archive]       deposit in a DOI-issuing archive; the DOI is what "Available" certifies.
[Document]      README with exact run path, expected outputs, and a small worked example.
[Environment]   container/lockfile pinning deps + the exact tool commit; note hardware needs.
[Reuse story]   show how to run the tool on a NEW input, not just replay your experiments.

Reusable is about strangers, not your tables

Evaluators judge reusability, so write for someone who wants to use your automation on their own code:

  • A clear entry point and documented inputs/outputs.
  • Instructions to run on a new subject, with a template config.
  • Sensible structure (source vs. data vs. scripts), an open license, and dependency pinning.
  • Removal of dead scripts, secrets, and machine-specific paths.

Evaluator-proofing checklist

[Runs clean]   fresh environment (container) -> documented command -> expected output, no manual patching
[DOI]          archival deposit with a DOI + open license (for Available)
[Docs]         README covers install, run, expected results, and reuse on a new input (for Reusable)
[Provenance]   subject SHAs, dataset version, seeds, model IDs/dates + cached outputs included
[Scope honesty] hardware/time requirements and known limitations stated up front
[No secrets]   API keys, tokens, private paths removed

Timing and scope

  • The Artifact Evaluation deadline follows research-track acceptance; treat it as a real milestone, not an afterthought — a strong tool with a weak package earns no badge.
  • Evaluators are often students and junior researchers on a schedule: an artifact that needs a live API key, unpinned dependencies, or your specific cluster will fail on setup regardless of the underlying quality.
  • Badges are recognition, not re-review of the science; the paper is already accepted. The goal is durable, reusable automation.

Output format

[Target badges] Available / Reusable (Functional/Reproduced 待核实 for this edition)
[Archive] DOI minted? open license?
[Runs clean] fresh-env command -> expected output, no manual fixes?
[Reusable] docs + run-on-new-input path present?
[Provenance] SHAs / dataset version / seeds / model IDs / cached outputs bundled?
[Blockers] <ordered fixes before the AE deadline>
信息
Category 编程开发
Name ase-artifact-evaluation
版本 v20260724
大小 4.21KB
更新时间 2026-07-28
语言