QualiLoop
← All articles
Continuous verification

Anthropic’s long-running agents point to a missing development artifact: verified behavior

As coding sessions become disposable, the evidence of what works needs to persist.

By QualiLoop · November 2026 · 3 min read · 641 words · 1 primary source

A coding agent can return to a repository, read a progress note, and continue building. But a note saying “checkout completed” does not tell the next session what was tested, which conditions passed, or which important exceptions remain unresolved.

Anthropic’s work on long-running agents addresses the continuity problem with structured artifacts, incremental implementation, and a persistent feature list. Its report also describes agents declaring completion too early and marking features complete without adequate end-to-end testing. The harness improvements give later sessions a clearer account of what remains to be done. [Anthropic: Effective harnesses for long-running agents]

From the original sourceCropped source heading and a short passage from Effective harnesses for long-running agents
Anthropic · 26 November 2025. Cropped excerpt from the original publication. Open the source shown.

The broader engineering opportunity is to preserve verified behavior as a first-class project artifact. A feature description records intent. A test result records evidence, under a particular version and set of conditions. Both need to survive the session that produced them.

Memory is not verification

A progress file is useful because it reduces repeated investigation. But it is still a claim about prior work. If it says “the agent handles subscription changes,” a later coding session needs a way to inspect what that means.

Was the plan changed correctly? Were prorated prices handled? Did the assistant avoid promising a refund that the policy does not allow? Were account permissions checked? Did the run use real tool responses or a simplified mock?

A durable evaluation record answers those questions more precisely than a paragraph of confidence. It connects the requirement to a scenario, the scenario to a run, and the run to inspectable evidence.

Conceptual architectureA persistent evaluation record supports successive coding sessions. QualiLoop synthesis inspired by Anthropic’s long-running harness work.
A persistent evaluation record supports successive coding sessions. QualiLoop synthesis inspired by Anthropic’s long-running harness work. Download PNG.

Build a behavior ledger alongside the repository

Start with the product’s important user tasks. For each one, preserve the governing requirement, representative scenarios, required fixtures, and the checks that establish success. Attach known limitations instead of treating untested conditions as implicitly supported.

Every result should name the implementation and evaluation versions. A passing run on last month’s prompt does not establish reliability after a new router, model, or tool integration. Conversely, a failure from an obsolete version should not keep appearing as an unresolved current defect.

This does not require a perfect coverage percentage. Enumerating everything an open-ended agent could encounter is unrealistic. A useful ledger is specific about the behaviors it covers and the risks it has not yet explored.

Let failures improve the project’s memory

When a user or evaluator finds an important defect, preserve a reproducible version of it. Add nearby variations that express the same requirement in different ways. After the repair, keep those checks in the approved suite.

The next coding session now inherits more than a warning. It inherits an executable constraint. If a later optimization reintroduces the problem, the suite can surface it before the team forgets why the original fix mattered.

Control suite growth deliberately. Remove obsolete scenarios through a visible review process, merge redundant cases where appropriate, and keep critical checks easy to identify. An enormous suite with unclear ownership can become slow, noisy, and easy to ignore.

Make release gates explainable

A release threshold should reflect the product’s risk. Critical permission or transaction checks may need to block a release independently of aggregate performance. Lower-severity style issues may be tracked without preventing deployment.

When a gate blocks a change, it should point to the evidence and requirement, not simply a score. The coding agent can then investigate, implement a repair, and rerun the relevant checks. Limit automated attempts and escalate when repeated changes fail to improve the result.

This is where QualiLoop’s continuous test-fix-rerun model becomes useful. The evaluation suite and its history can persist while coding agents, context windows, and implementation versions change. Developers can move quickly because the definition of acceptable behavior is recorded outside any single session.

The lasting artifact of agentic development should include what the system demonstrably does, the conditions under which that was checked, and the failures the team has learned to prevent. That is the memory a faster development process needs.

Sources and further reading

Effective harnesses for long-running agentsAnthropic · 26 November 2025 · Primary source

See the loop on your own agent

QualiLoop tests every change your coding agent makes, so what you ship actually works.

$ npx qualiloop connectTry for free

Research findings are attributed to their primary sources. Practical examples and engineering recommendations are QualiLoop analysis.