正在打开每日关注
HarnessBank：让智能体自进化可验证
用基因库与门控验证闭环，区分智能体改进是真提升还是偶然过拟合。
来源摘录
Our new paper. Self-evolving agents have a measurement problem. The hard part isn’t generating changes to an agent harness. It’s knowing which changes genuinely helped rather than got lucky, overfit, or never activated at all. LLMs diagnose failures and propose changes across prompts, knowledge, runtime, tools, and configuration. But deterministic code owns the credit: validity checks, activation checks, paired significance testing, and a sealed test. Across 7 domains, the evolved harnesses gained +9.0 to +15.5pp on the 6 statistically credited sealed tests, retaining 86–147% of the training gain. What transfers is not one universal harness. It’s the diagnose-and-credit loop. https://t.co/w56EndGp3n
深度解析
是什么 HarnessBank把智能体自进化拆成“提出改动”和“证明改动有效”两件事：任务智能体执行基准任务，独立的 evolver agent 诊断失败，修改提示词、知识、运行时、工具、控制循环和配置；系统维护按语义坐标组织的 Harness Gene Bank，对候选进行重造、重组、筛选与选择。
核心设计 LLM 不负责自我宣布成功，确定性代码负责 credit：先做有效性与激活检查，再以成对显著性检验和 sealed test 对照改动。多样性、质量、一致性三道门分别约束探索广度、验证严谨度与执行可靠性。
关键洞察 摘要称七个 benchmark 提升 5.1%–15.4%；推文强调六个具统计归因的 sealed tests 达到 +9.0–+15.5pp。跨模型实验说明可迁移的不是万能 harness，而是“诊断—生成—验证—归因”闭环。工程上应记录版本、激活证据、配对样本和留出测试集。
边界与风险 结论依赖密封测试的代表性、统计功效和门控实现；语义基因库仍可能过拟合，候选评估成本也会上升。材料未提供代码，复现难度暂无法判断。