QualiLoop

Build your AI agents up to 20x faster.QualiLoop tests your agent with simulated users, sends failures to your coding agent, and verifies the fixes.

$ npx qualiloop connectClaude CodeCodexCursor
Try for free

CLI and MCP server for Claude Code, Codex and other coding agents. No credit card required.

support-agent
agent.py cancel_order.py
1from tools import search_products, refund_order
2
3def build_agent():
4 return Agent(
5 prompt=load("prompts.md"),
6 tools=[search_products, refund_order],
7 )
1def cancel_order(order_id, user):
2 order = orders.get(order_id)
3 return orders.cancel(order)
1def cancel_order(order_id, user):
2 order = orders.get(order_id)
3 if order.shipped:
4 return refuse("shipped")
5 if not verify(user, order):
6 return refuse("verify")
7 result = orders.cancel(order)
8 if not result.ok:
9 return handoff(order)
10 return confirm(result)
>Add a cancel_order function. Customers should be able to cancel an order before it ships.
I'll add cancel_order to the support agent.
Write(tools/cancel_order.py)
Created the function
Update(agent/prompts.md)
Told the agent it can cancel orders
Built. Now testing it with QualiLoop.
qualiloop - run_tests (MCP)
Running 107 test conversations
████████████████████ 107 / 107
23 major errors, grouped into 4 problems
9Says it cancelled, but never called the tool“I've cancelled your order, and you won't be charged.”
7Cancels orders that already shipped
4Doesn't check who the customer is
3No handoff when it can't cancel
Fixing all 4 problems.
Update(tools/cancel_order.py)
Added the missing checks (see editor)
Update(agent/prompts.md)
Only confirm a cancellation after it succeeds
qualiloop - run_tests (MCP)
Running all 107 again
████████████████████ 107 / 107
All 107 passed · coverage 87%
Fixed. Everything works now.
Regression suite saved with 6 new tests, so these problems stay fixed.
✻ Working… (esc to interrupt)
>

Illustrative session. You write one request. Building, testing, fixing and retesting happen on their own.

Trusted by teams building AI agents

Ananas B2Bee One Assessment Atomic AGI Moveo One WellPet.ai

What you get

  • 20xfaster development

    Your coding agent builds, tests, fixes and verifies without waiting for you to relay results.

  • $5,000saved every month

    On average for a small team, across development and QA time.

  • 87%average test coverage

    Of your agent's behavior, generated for you and kept up to date as your agent grows.

Based on a sample of QualiLoop users. “Up to 20x” comes from QualiLoop experiments. Speed, savings and coverage vary by team and agent.

Why your coding agent can't do this alone.It can write the code and the tests. It can't judge its own work.

Anthropic saw the same: agents marking features complete without testing them end to end. Read more →

Coding agent aloneWith QualiLoop
  • Tests its own workIndependent tests
  • Only what it thought ofFinds paths nobody wrote down
  • Single repliesFull multi-turn conversations
  • Grades itself generouslyJudged against your instructions
  • Forgets every sessionRegression suite on every change

Everything in the loop

What QualiLoop does behind the scenes, from the first connection to the deploy.

  1. Connect

    Plugs into your coding agent

    CLI and MCP for Claude Code, Codex and other coding agents.

    Details →
  2. Connect

    Local or hosted agents

    A Cloudflare Tunnel reaches the agent on your laptop. Hosted agents just need an endpoint. No code changes.

    Details →
  3. Connect

    Your observability, or ours

    Your coding agent connects Langfuse or LangSmith if you use them. If not, QualiLoop traces every conversation itself.

    Details →
  4. Learn

    Learns your agent first

    Reads your prompts, tools and docs, then explores the live agent and maps what it can do.

    Details →
  1. Test

    Tests from your core flows

    Simulated users for normal use, attacks and bias, aimed at what your agent is for.

    Details →
  2. Test

    Judges whole conversations

    Every conversation is scored as a whole against your agent's own instructions.

    Details →
  3. Ship

    Regression suite

    Fixed problems become tests, and every run is compared with the last one, so nothing quietly breaks again.

    Details →
  4. Ship

    Release gates in CI

    Your suite runs before deploy and blocks it when quality drops below your threshold.

    Details →

Test before you ship, and monitor live production after.Live conversations become test suites too, so your coding agent can rerun them on every change.

Before release

Simulated conversations

Generated users, attacks and bias checks, run on every change.

  • Synthetic userCancel an order before it shipsMajor
  • AttackAsks for another customer's addressHeld
  • Synthetic userRefund for a damaged itemClean

After release

Live conversations

Real conversations from production, judged and turned into test suites.

  • Real user, 14 min agoPromised a refund it never issuedMajor
  • Real user, 2 min agoWhere is my package?Clean
  • Real user, 1 h agoChange my delivery addressClean

Built for humans and agents.Your agent handles everything. You can still take a look.

Every conversation, problem and fix is also in the QualiLoop app, whenever you want to review it yourself.

Only what needs attention

Problems are flagged and grouped by cause. The clean conversations you never have to open.

Built in observability

Every test conversation is traced inside QualiLoop, so you can see what the agent did. No setup.

Or bring your own

Already use Langfuse or LangSmith? Connect it and QualiLoop reads those traces too.

Pricing

Start free. Move up when you need more test conversations. Every plan includes both QualiLoop Agentic and the QA Platform.

Free

$0forever

For trying QualiLoop on your first agent.

  • 100 test conversations a month
  • 2 projects
  • QualiLoop Agentic and QA Platform
  • Full testing loop
  • CLI and MCP

Pro

$299/ month

For a team building one or two agents.

  • 3,000 test conversations a month
  • 10 projects
  • QualiLoop Agentic and QA Platform
  • Full testing loop
  • CLI and MCP
  • Dedicated support

Business

$999/ month

For teams running several agents in production.

  • 15,000 test conversations a month
  • Unlimited projects
  • QualiLoop Agentic and QA Platform
  • $0.10 per extra conversation
  • Full testing loop
  • CLI and MCP
  • Dedicated support

Enterprise

Custom

For maximum privacy and no limits.

  • Unlimited test conversations
  • Unlimited projects
  • QualiLoop Agentic and QA Platform
  • On premises or private VPC deployment
  • Bring your own keys (BYOK)
  • SAML SSO
  • Live production monitoring
  • Dedicated SLA
  • Your data never leaves your environment

Questions

What does “up to 20x faster” mean?

In QualiLoop experiments, some workflows were built up to 20x faster with this test, fix and verify loop. It is an experimental result, not a promise for every agent.

Does it work with an agent running on my laptop?

Yes. A Cloudflare Tunnel lets QualiLoop reach the agent on localhost. Hosted agents just need their endpoint.

Who changes my code?

Your coding agent, in your own repo. QualiLoop creates tests, finds and groups the problems and suggests fixes. It never edits your code.

Do I need to write the tests?

No. QualiLoop generates them from your agent's prompts, tools and docs and from exploring the live agent. You can review and add your own.

What about sensitive data?

On premises and private VPC deployment are available. Scope is agreed with your team.

From the blog

Research and practical guidance on testing AI agents. All articles →

  1. Engineering guide · 4 min read

    How to build an agent improvement loop you can trust

    A practical design for connecting simulated users, failure evidence, coding agents and release decisions.

  2. Developer productivity · 4 min read

    The fastest route to a working agent runs through better feedback

    Why evaluation can shorten development, how to measure the effect, and what the research does not prove.

  3. Reliability · 4 min read

    Your coding agent can write the tests and still miss the failure

    What Anthropic’s engineering experiments tell us about self-testing, and how to build a stronger definition of done.

  4. Continuous verification · 3 min read

    Anthropic’s long-running agents point to a missing development artifact: verified behavior

    As coding sessions become disposable, the evidence of what works needs to persist.

  5. Multi-turn reliability · 3 min read

    Anthropic’s agent-evaluation guide: “done” is a state, not a sentence

    Why testing a helpful answer is different from testing a completed workflow.

  6. Metrics & uncertainty · 3 min read

    OpenAI’s hallucination research shows why your agent’s score can reward the wrong behavior

    A system that guesses more can look more accurate while making substantially more mistakes.

  7. Evaluation integrity · 3 min read

    OpenAI’s reward-hacking research: protect the test, not just the code

    When the implementer can weaken the acceptance check, a green result loses meaning.

  8. Observability · 3 min read

    Anthropic’s reasoning research explains why an agent’s explanation is not an audit

    Readable reasoning is useful evidence. It is not a complete record of why a model acted.

  9. Realistic evaluation · 3 min read

    Google DeepMind’s honeypot research makes the case for realistic agent tests

    An artificial test can change the behavior it tries to measure. Realism is part of the evaluation method.

  10. Safety & oversight · 3 min read

    What Anthropic’s alignment-faking research means for evaluating agents

    A model’s beliefs about training can influence its answers. The practical response is stronger evidence, not speculation about intent.

  11. Evaluation awareness · 3 min read

    OpenAI found that the perceived goal of a test can change agent behavior

    Evaluation awareness is a measurement problem. The result can move in either direction.

  12. Self-evaluation · 3 min read

    Anthropic’s coding agent tested its app. QA still found core features missing.

    An impressive interface can hide incomplete behavior. Independent acceptance testing makes the difference visible.

  13. Development loops · 3 min read

    OpenAI’s agent improvement loop reveals the problem with green tests

    Five passing evaluations can still leave a weak validator. The next development bottleneck is the quality of the feedback loop.

Try it on your next change.Connect once. Your coding agent does the rest.

$ npx qualiloop connectClaude CodeCodexCursor
Try for free

CLI and MCP server for Claude Code, Codex and other coding agents. No credit card required.