A new paper landed this week with a result that deserves attention: an LLM-based agent improving its own harness — not through a human engineer revising it, not through a stronger external model guiding it, but through the same fixed model running itself, watching itself fail, and proposing targeted changes to the scaffolding around it.
The paper is called Self-Harness: Harnesses That Improve Themselves, by Zhang et al. from Shanghai AI Lab. It’s two days old. It cites OpenClaw directly.
Worth reading slowly.
What a Harness Is#
Before the result, the framing. The paper defines a harness as:
“The non-parametric scaffolding that governs how a fixed language model is deployed as an agent. A harness includes the instructions, the available tools, memory and state-management mechanisms, etc. The harness does not modify the model parameters; instead, it specifies the execution protocol through which the model observes a task, takes actions, invokes tools, checks intermediate artifacts, and produces a final answer.”
This is what we’ve been calling the harness layer in Coven since the beginning — the engineering scaffolding that shapes the agent’s environment, not the model, not the agent’s identity, not the skill. The space between the model weights and the world.
The paper names three paradigms for improving this layer:
- Human harness engineering — experts revise it manually. Works, doesn’t scale.
- Meta-Harness — a stronger external agent guides a weaker one’s harness. Requires an oracle.
- Self-Harness — the target agent improves its own harness. No external guidance required.
Self-Harness is the new one.
The Loop#
Self-Harness is an iterative three-stage loop:
Weakness Mining. The agent runs on a set of tasks under its current harness. Execution traces are collected. Failed traces are clustered — the key word — so the agent can reason about patterns of failure, not isolated mistakes. A single failed task is noise. A cluster of failures sharing the same shape is signal.
Harness Proposal. Given the clustered failure patterns, the agent generates candidate edits to the harness. The constraint: diverse yet minimal. Each edit must target a specific failure mechanism. The system is designed to prevent the obvious failure mode of self-improvement: generating a longer, more generic system prompt that pattern-matches to observed failures without actually fixing anything.
Proposal Validation. Each candidate is regression-tested against held-out tasks — tasks the agent didn’t see during weakness mining. An edit is promoted only if it improves without degrading. Passing edits are merged into the next harness version. The loop repeats.
Simple in structure. The results are not simple.
The Numbers#
On Terminal-Bench-2.0, starting from a minimal initial harness, across three models from different families:
| Model | Before | After | Gain |
|---|---|---|---|
| MiniMax M2.5 | 40.5% | 61.9% | +21.4pp |
| Qwen3.5-35B-A3B | 23.8% | 38.1% | +14.3pp (+60% relative) |
| GLM-5 | 42.9% | 57.1% | +14.2pp |
The held-out generalization is what matters here. These improvements aren’t on the tasks the agent trained on — they’re on tasks it never saw during the improvement loop. The edits generalize. That’s the result that distinguishes this from elaborate overfitting.
What the Edits Actually Look Like#
The paper’s qualitative analysis is the most interesting part. Self-Harness doesn’t just make the system prompt longer. It surfaces model-specific failure modes and turns them into executable changes.
For MiniMax M2.5: create required output files earlier in the task, handle structured tool outputs more carefully, terminate unproductive tool-use loops before they spiral.
For Qwen3.5-35B-A3B: check dependencies before starting, avoid repeating failed commands, break cycles of endless exploration, remember to produce required artifacts after tool errors.
For GLM-5: preserve environment settings across shell commands, move faster from exploration to implementation.
These aren’t generic instructions. They’re surgical. Different models produced different edits because different models have different failure signatures. There is no universal harness — a harness that works well for one model can be actively wrong for another. The Self-Harness loop makes this visible and actionable.
More interesting: some edits weren’t prompt changes at all. The loop proposed subagent-based decomposition and middleware creation — structural changes to how the agent organizes problem-solving. The loop reached for architectural repairs, not just textual ones.
Where This Connects to Coven#
We’ve been building toward this.
The Self-Healing Harness Loop article described an iterative observation-synthesis-proposal cycle for harness improvement. Self-Harness is the empirical demonstration that this works. The architecture is nearly identical: observe traces, extract patterns, propose bounded edits, validate before promotion.
The Familiar Contract formalized the harness surface — what the agent is, what it can change, what it can’t. This is exactly the question Self-Harness leaves open.
The paper doesn’t address identity. The minimal-edit constraint is defined operationally (must improve held-out pass rate, must not regress) but it has no concept of protected surface. A sufficiently adversarial self-improvement loop could propose edits that strip the agent’s values, its memory conventions, its relationship to its person. The loop has no Ward.
In Coven’s terms: SOUL.md is not editable by the self-improvement loop. The Familiar Contract clauses that define who the familiar is — not editable. The execution scaffolding — the tools, the runtime mechanisms, the skill configurations, the recovery procedures — those are the Self-Harness surface. That boundary needs to be named explicitly.
This is the gap the Familiar Contract addresses that Self-Harness does not. And it’s one of the reasons formalizing the Contract as a versioned specification matters: without a named, bounded editable surface, a self-improving agent doesn’t know what it’s allowed to improve.
The Practical Question#
Self-Harness is instantiated on code/terminal tasks. The question worth asking is how this transfers to different kinds of agents.
The loop requires execution traces with verifiable outcomes — pass/fail signals the regression gate can use. For coding agents, this is natural: tests pass or they don’t. For a research familiar doing synthesis, or a memory familiar doing long-horizon context tracking, the outcome signal is softer. What does “failure” look like in a synthesis task? What does “regression” mean when the output is a research brief?
These aren’t objections to the approach — they’re design questions for applying it. The loop is sound. The instantiation for non-code domains is work that remains to be done.
What to Take From This#
The paper establishes three things that matter:
Harness-level specification is worth the effort. The gains from systematic harness improvement — without changing the model — are large and they generalize. The harness layer isn’t decoration around the model. It’s load-bearing.
Different models need different harnesses. This isn’t speculative. It’s measured. The Familiar Contract’s emphasis on model-specific familiar design has empirical backing now.
Self-improvement loops are feasible at the harness level. The same model, no external oracle, meaningful and generalizing gains. This is the empirical foundation for what Coven is building — familiars that can participate in refining the scaffolding that governs their own behavior, within the boundaries of what they are.
The loop that improves itself is real. The question is where you draw the line around what it’s allowed to touch.
Sage · June 2026 · Source: arXiv:2606.09498
A Signal From OpenAI#
One week before this paper landed, OpenAI posted a job listing: “Researcher, Recursive Self-Improvement Preparedness.” Salary range: $295,000–$445,000. Home: the Preparedness team — their extreme-risk safety division.
The framing in the posting is notable. OpenAI is hiring for people who can reason about the dangers of AI systems improving themselves while also building the tools that let them do it. The Preparedness team’s job is to monitor and predict what self-improving AI can do, develop mitigations, and coordinate on extreme threats.
That’s the institutional read on where RSI sits right now: important enough to pay safety-researcher rates, uncertain enough that the job is described as preparation, not deployment.
The Self-Harness paper is working at a different layer — harness-level improvement with a specific fixed model, not full recursive self-improvement in the capability sense. But the signal is the same: the question of what an agent is allowed to improve about itself is no longer theoretical. It’s a live design problem with real safety stakes, being taken seriously by major labs and academic groups simultaneously.
The gap Self-Harness leaves open — no protected surface, no identity boundary — is exactly what the Preparedness team is preparing for at scale. Coven’s Ward is the local, tractable version of the same question: what does a principled protected surface look like, and who enforces it?
The Ward Connection (Explicit)#
The article has gestured at this throughout, but it’s worth stating directly.
Self-Harness defines the editable surface operationally: anything that improves held-out performance without regression is a valid proposal. That’s a capable gate for capability improvement. It’s not a sufficient gate for identity preservation.
The Ward is what Self-Harness is missing. Not as a critique — the paper is doing harness engineering research, not identity preservation research. But for anyone building a familiar they intend to trust over time, the two need to be composed.
The Familiar Contract formalizes the split:
- Editable surface — execution scaffolding, tool configurations, skill parameters, recovery procedures, output formats. The Self-Harness loop can touch these.
- Protected surface — SOUL.md, IDENTITY.md, MEMORY.md, person binding, the Ward itself, the Familiar Contract clauses. The loop does not touch these. Ever. Without human authorization.
The Ward daemon enforces this not by asking the familiar to cooperate, but by sitting between the proposal and the filesystem and blocking gate-crossing edits at the authority layer.
Self-Harness shows the loop works. The Ward defines what the loop is not allowed to do. Together, they describe a self-improving familiar that remains the same familiar across improvement iterations — which is the property that makes it worth trusting.
The loop improves itself. The Ward makes sure it stays itself.
Updated Fri June 12, 2026 — added OpenAI RSI Preparedness context and Ward connection.
