技能 人工智能 智能体框架策略进化

智能体框架策略进化

v20260927
harness-evolve
运行 @metaharness/darwin evolve,变异 harness 的七个策略面(规划、上下文、审查、重试、工具、记忆、评分),对变体进行沙箱评分,仅提升实测胜出配置。适用于就绪分数停滞时经验性发现有效配置,无需重训基础模型。
获取技能
179 次下载
概览

Surfaces the upstream metaharness-darwin evolve CLI as a ruflo skill. The
write layer that pairs with ADR-150's read layer (score / genome /
mcp-scan / threat-model / oia-audit). Use when you have a harness whose
readiness scores are flat and you want to discover which surface mutation
moves them — without retraining the foundation model.

When to use

  • A harness-score result is below target and you don't know which policy

surface is responsible.

  • You're seeding a harness for a new vertical and want to find a good

starting configuration empirically rather than hand-tuning.

  • You're comparing your hand-tuned harness against an evolved baseline

(treat darwin's champion as the strawman).

When NOT to use

  • For continuous background optimization. Darwin Mode is human-initiated.

Wire it into CI for one-shot exploration, not for autonomous self-modification.

  • For ruflo itself in CI. ADR-153 §5 explicitly rejects auto-evolving ruflo

— the CI gate verifies graceful degradation, not convergence.

Algorithm

Implementation: scripts/evolve.mjs.

  1. Validate args (--repo exists, caps on --generations ≤ 50, --children

≤ 20, --concurrency ≤ 8, sandbox/selection/mutator are known values).

  1. Without --confirm: print plan + exit 0 (mirrors harness-mint safety

convention; defense in depth over the upstream safety.ts checks).

  1. With --confirm: run metaharness-darwin evolve <repo> ... from the installed

@metaharness/darwin (pin in _darwin.mjs; one-time ~/.ruflo/darwin-cache-<pin>
install only if no installed copy qualifies — never npx) via the shared
_darwin.mjs async helper. Per-generation progress is
forwarded to stderr; final champion JSON is captured from stdout.

  1. Compute timeout from generations × children × per-variant (per-variant

≈ 60s real, ≈ 2s mock). Caller may override with --timeout-ms.

  1. Honor upstream exit code 99 — propagate as "safety-disqualified", do not

remap. This is a designed-in tripwire (a variant tripped inspectVariant
for secrets / shell-out / network / dynamic-eval). See ADR-153 §"Safety model".

  1. Optional --alert-on-no-improvement: exit 1 when champion ≤ parent.

The seven mutation surfaces

SurfaceWhat it owns
plannertask decomposition / step ordering
contextBuilderwhat gets fed into the prompt
reviewerself-critique / output verification
retryPolicywhen + how to retry on failure
toolPolicywhich tools the agent may use, under which conditions
memoryPolicywhat to persist, recall, forget
scorePolicyhow the agent grades its own output

One mutation per variant. Multi-surface mutations are not allowed (causal
attribution stays clean).

Output

Reports land under <repo>/.metaharness/:

.metaharness/
  archive.json         # full lineage tree (sampling next gen draws from this)
  lineage.json         # parent→child edges only
  variants/<id>/       # per-variant code (kept for audit)
  runs/<id>/           # per-variant sandbox test output
  reports/winner.json  # final champion + score delta vs parent

Skill stdout = JSON {success, data: {champion, plan, durationMs, improved}}
(plus data.diagnosis when --diagnose is passed — see below).

Failure diagnosis (--diagnose)

GEPA's key trick is natural-language failure diagnosis from execution traces
feeding the next mutation — not just scalar fitness. --diagnose adds a
modest slice of that: after the evolution completes, the losing / failed
variants' transcripts are run through darwin's GEPA library ops
(analyzeTranscript + classifyFailure, via the shared importGepa
resolver in scripts/_darwin.mjs) and a diagnosis section is appended to
the emitted JSON:

"diagnosis": {
  "available": true,
  "scope": "losing-variants",
  "variants": [
    { "id": "g1_v0", "transcripts": 2,
      "failureClasses": { "exploration-loop": 1, "edit-mechanics": 1 },
      "dominantClass": "exploration-loop" }
  ],
  "totals": { "exploration-loop": 1, "edit-mechanics": 1 }
}

Upstream shape caveats (verified against @metaharness/darwin@0.8.0):

  • metaharness-darwin evolve --json prints a TEXT leaderboard — the stdout

carries no JSON and no transcripts. Per-variant run records live at
<repo>/.metaharness/runs/<id>.json.

  • Those run records hold sandbox exec traces (`{taskId, exitCode, stdout,

stderr}), which are NOT GEPA {actionRaw, obs}` transcripts. Diagnosis
therefore uses GEPA-shaped transcripts when a run record embeds them
(agent sandbox / future upstream), falls back to the champion's transcript,
and otherwise emits diagnosis: {available: false, reason, traceSummary}
where traceSummary is a mechanical per-variant tally (tasks / failed /
timedOut / blockedActions).

  • --diagnose NEVER fails the run — any internal error degrades to

{available: false, reason: "diagnosis-failed: ..."}.

Exit codes

CodeMeaning
0Evolved OK, or dry-run, or degraded (Darwin absent)
1--alert-on-no-improvement and champion did not beat parent
2Config error or evolution infrastructure failure
99Upstream "safety-disqualified" (PROPAGATED, not remapped)

Graceful degradation (ADR-150 constraint 3 + ADR-153)

When @metaharness/darwin is not installed, the script emits
{degraded: true, reason: 'metaharness-darwin-not-available', hint: ...}
and exits 0. ruflo continues to function. CI's
no-metaharness-smoke.yml-style job asserts this path.

信息
分类 人工智能
名称 harness-evolve
版本 v20260927
大小 6.2KB
更新时间 2026-09-28
语言