Research and practical guidance on testing AI agents.
What the latest work from OpenAI, Anthropic and Google DeepMind means for building agents that work, and how to put it into practice.
Research notes
Ten short essays on what recent research says about testing and verifying AI agents.
OpenAI’s agent improvement loop reveals the problem with green tests
Five passing evaluations can still leave a weak validator. The next development bottleneck is the quality of the feedback loop.
3 min readAnthropic’s coding agent tested its app. QA still found core features missing.
An impressive interface can hide incomplete behavior. Independent acceptance testing makes the difference visible.
3 min readOpenAI found that the perceived goal of a test can change agent behavior
Evaluation awareness is a measurement problem. The result can move in either direction.
3 min readWhat Anthropic’s alignment-faking research means for evaluating agents
A model’s beliefs about training can influence its answers. The practical response is stronger evidence, not speculation about intent.
3 min readGoogle DeepMind’s honeypot research makes the case for realistic agent tests
An artificial test can change the behavior it tries to measure. Realism is part of the evaluation method.
3 min readAnthropic’s reasoning research explains why an agent’s explanation is not an audit
Readable reasoning is useful evidence. It is not a complete record of why a model acted.
3 min readOpenAI’s reward-hacking research: protect the test, not just the code
When the implementer can weaken the acceptance check, a green result loses meaning.
3 min readOpenAI’s hallucination research shows why your agent’s score can reward the wrong behavior
A system that guesses more can look more accurate while making substantially more mistakes.
3 min readAnthropic’s agent-evaluation guide: “done” is a state, not a sentence
Why testing a helpful answer is different from testing a completed workflow.
3 min readAnthropic’s long-running agents point to a missing development artifact: verified behavior
As coding sessions become disposable, the evidence of what works needs to persist.
3 min readEngineering series
Practical guides for teams building with coding agents.
Your coding agent can write the tests and still miss the failure
What Anthropic’s engineering experiments tell us about self-testing, and how to build a stronger definition of done.
4 min readThe fastest route to a working agent runs through better feedback
Why evaluation can shorten development, how to measure the effect, and what the research does not prove.
4 min readHow to build an agent improvement loop you can trust
A practical design for connecting simulated users, failure evidence, coding agents and release decisions.
4 min readSee the loop on your own agent
QualiLoop tests every change your coding agent makes, so what you ship actually works.
$ npx qualiloop connectTry for free