Skip to content

Blog

Breaking the Loop: Why agents never finish on their own

Agents in a software factory always find another problem, another ticket, another improvement. Only a second, much slower loop can say when the work is finished: the one that reaches the market.

Written by

Specification and review

Published: 24 October 2026 22 min read

More output, and none of it shows up in revenue

“Our teams produce far more with agents. Why don’t we see it in revenue?” It is a question we hear in our own client work.

A team starts its factory at the end of the day. Agents pick up tickets, write code, run the tests, review one another and fix whatever the checks reject. By the next morning, pull requests are waiting, most of them green. Alongside them sit new tickets the agents opened along the way.

You can measure the increase. In 2025, Faros AI (opens in a new tab) analyzed more than 10,000 developers across 1,255 teams. On teams with high AI adoption, developers merge 98% more pull requests, and review time goes up by 91%. Most developers in that dataset only used autocomplete, and agents barely featured. With agents, the jump is bigger: Anthropic reports (opens in a new tab) merging around 2,500 code changes a day on average in May 2026 and more than 30,000 in September, so many that it replaced git with a system of its own for its monorepo.

+98% merged pull requests, and 91% more review time, on teams with high AI adoption (Faros AI, 2025).

Little of this makes it to the company level. At that level, the Faros study found no significant correlation between AI adoption and better throughput, better DORA metrics or better quality. It puts this down to bottlenecks downstream of coding, above all in review, and to uneven adoption across teams. If the gain does not even show at company level, it is hardly surprising that revenue does not reflect it.

We see another reason in teams that work with agents: agents do not stop by themselves. A review agent finds something in every round. Each finding gets implemented, and the implementation creates the next one.

A human calls it “good enough” after the second round and leaves the rest. Much of what mattered comes back anyway, as a bug or as an alert from production. An agent has no notion of “good enough” unless someone gives it one. Another finding costs it no attention, so it keeps looking: another abstraction, another optimization for a load that does not exist.

Jin and Chen (opens in a new tab) demonstrated one building block of this pattern in a 2026 lab study: language models often rate correct code as defective, and they do it even more often when asked to justify the verdict and propose a fix. Yoko Li of a16z (opens in a new tab) measured an agent loop chasing an unreachable goal and found that two thirds of the cost bought no further progress. Her conclusion: loops have “no idea how to stop”.

The same is true at both scales. A single run always turns up one more finding, and the factory as a whole always turns up one more ticket. Both have brakes: a maximum number of rounds and a token budget. We need them, but all they limit is effort. Neither defines what finished means.

Inside the factory, the only criterion for finished is green, which means the change passes its checks. No test tells you whether it was the right change, and code can pass a check without getting better. So a factory without an impact hypothesis turns out features quickly, and nobody has checked whether they are worth anything. It is an endless loop that burns tokens and produces output.

The title is borrowed from Judas Priest: “Breaking the Law”, 1980. Anyone who writes code knows break, the statement that ends a loop. The factory has no break for “that is enough”. Someone has to give it one.

The slow loop sets the pace

The inner loop is the factory. hackers&wizards defines it as the system that produces software changes. It takes in an intent, breaks it into bounded work, supplies agents with context and accepts results after defined checks. A round of writing, checking and fixing takes minutes.

The outer loop is product work, all the way out to the market. Strategy and customer knowledge shape a hypothesis about impact. The team ships, the market responds, and what comes back changes what the organization knows about its customers. A single pass takes weeks at the very least. One of us, Markus, has written at length about this loop and its pace in “AI Flow Dynamics” (opens in a new tab).

The slow loop sets the pace
Inner loop: the factory Outer loop: product and market
Question Is it built right? Is it the right thing?
Pace Minutes Weeks
Checked against Specification and acceptance criteria Success model and market data
Result Output Outcome
Ends never on its own with a decision the deadline forces

The inner loop delivers efficiency, the outer one effectiveness. Building code fast is no help when the direction is wrong, and the right direction is no help when the code for it comes too slowly. Success means optimizing both at once.

Once a team can build more than it can evaluate with customers, the outer loop is the bottleneck, and adding agents does not change that. A team can produce several options in an afternoon. What it cannot do is put them in front of customers at the same pace and still expect a clean signal from each. So the number of features a team can have in testing with customers at once depends on how much feedback it can evaluate, not on how much it can build.

Four breaks where a human decides

Hence our rule: the inner loop runs only as far as the outer loop allows. At four points, nothing goes further without a human decision. We call these points breaks.

  1. Explore further: Is the idea plausible enough to earn more attention?
  2. Build: Is it plausible enough to be built, and what is it allowed to cost?
  3. Ship: Is what was built good enough to go out to customers?
  4. Keep: Has it delivered in the market what the impact hypothesis promised?
The factory runs as the inner loop inside the outer loop's build step, and at four breaks on the outer loop a human decides whether work moves on.

Figure 1: The factory runs as the inner loop inside the outer loop’s build step, and at four breaks on the outer loop a human decides whether work moves on.

The example

We go through the four breaks with a single example. It is made up, and its numbers serve only to illustrate the sequence.

In the example: the need

A B2B product helps teams manage their projects. In interviews, team leads explain that every Friday they assemble the project status from tasks, comments and conversations. The team gathers seven ideas for this need.

First break: no idea moves on without an impact hypothesis

A lot of features start in the shower in the morning. The long haul used to begin after that, because software was expensive and the team had to identify the right version in advance.

Today, any of these ideas could be in the factory by the same morning. That is why the first decision comes before the factory starts, and it starts with very little: one sentence.

If this feature succeeds, this is what changes for these people in this situation.

That sentence is the impact hypothesis. Only after it come the number the team will watch for the impact and the deadline by which the impact should show. Together, impact hypothesis, number and deadline form the success model, which lives in a feature file next to the need.

The order is intentional, because usage is not yet value. The seat belt is the extreme case: its value lies in the rare emergency.

Writing the impact hypothesis takes product knowledge the team can get at: strategy, customer knowledge, KPIs, research, past features and how they performed. Markdown files will do to start, as long as people and agents can find and read them. We call this a product context system. It captures what the organization knows, what it assumes and where its knowledge runs thin.

That single sentence already filters. We often see features being built without anyone having written it down first.

In the example: the success model

The product manager manages to write the impact hypothesis for six of the seven ideas. For the editable weekly summary, the feature file reads:

Impact hypothesis: On Fridays, team leads revise a draft instead of assembling the status by hand.

Number: At least 20% of team leads open the draft in at least three different weeks, and at least half of those team leads edit and share it. Opening on its own would just be usage. Only a shared draft suggests that it has replaced a hand-written status.

Deadline: eight weeks.

For the seventh idea, a dashboard for senior management, the product manager cannot find any change for the team leads. The idea stops here, without having cost a single token.

Second break: build several options, each with a price tag

Estimate first, then cut

Each idea that makes it through the first break is an option: one possible way to serve the same need.

Before the second decision comes a forecast. A skill, a reusable procedure for agents, gets the feature file and access to the product context system. Several models estimate, each from a different angle, how likely the option is to hit the targets in its success model. The skill returns a range along with its sources, the contradictions it found and the gaps. Where the estimates are far apart, the range widens.

The forecast is a rough approximation. Two ranges are not directly comparable either: a small improvement that is almost certain can be worth less than a large one that is uncertain. A human has to make the call.

In the example: the forecast

Estimate first, then cut
Option Forecast: chance of reaching the targets Assessment
Editable weekly summary 45 to 60% Need is visible, trust remains critical
Slack/Teams digest 55 to 70% Fits an existing routine
Questions about the project status 30 to 55% Usage situation still unclear
Risk alerts 50 to 75% High value possible, depends on data quality
Audio summary 15 to 40% Few signs of demand for this format
Presentation deck 20 to 30% Relevant only for a small part of the target group

The product manager drops the audio summary and the presentation deck. Four options get the go-ahead to be built.

A test drive instead of a brochure

When building was expensive, this is where the team would have sent one option into development and possibly thrown out the better one. It would never have found out, because it would only have compared brochures. Afterwards we often hear: “We cannot know what would have come of it, there was no A/B test.” Building every option would have been too expensive.

Cheap code generation now allows a different sequence. The factory builds several options for the same need, and building them commits nobody to a release. Markus calls this Option Storming (opens in a new tab). The buying decision comes after the test drive.

This sounds like the opposite of our complaint above. There, the factory burns tokens, and here we deliberately have it build options that nobody ships. Compute is cheap, and what is scarce is human attention and signal from the market. Option Storming spends what is cheap to protect what is scarce: in the example, three of the four options reach neither the trunk (the product’s main branch) nor the market.

Even so, the number of options has a limit. Every test drive takes the team’s time, and building more options than the team can seriously try out does not make the choice better. That is how Option Storming serves the outer loop: rather than cutting options early on gut feeling, we try them out as builds and send only the best into the narrowest bottleneck there is, the market.

The brakes get a yardstick

Every run begins with the intake, which receives the feature file and first checks whether it can build from it. In our experience, an unclear requirement becomes an unclear plan, and an unclear plan becomes code the checks reject over and over. A good intake therefore asks the product team clarifying questions before any agent writes a line. Skip this step, and the factory risks building the wrong thing perfectly.

Then the agents get going, with one run per option. Here the brakes from the start of this article come back, this time with a yardstick from the outer loop.

A budget per option. The product manager and engineering agree on it together, in euros for tokens and in correction rounds. How big it is depends on what the decision between the options is worth to the team. Once it is used up, the run ends, and that is a result too, one the team looks into. The option may have been scoped too large or described too vaguely, or it may lack data.

Acceptance criteria decide which findings count. The review agent still finds something every round. Only a finding that violates an acceptance criterion or a check, security checks included, gets implemented. When the agent demonstrates a defect no check covers yet, such as a security hole or wrong data, it gets fixed and turns into a new criterion or a new check.

Everything else goes on a list, which only a trigger reopens: a bug, a monitoring alert, a measured latency. Until then, the code is good enough. This is also how a single run ends: once the criteria are met or the budget is gone.

Every check states its purpose. A linter message like function is too complex (11 > 5) tells the agent something is wrong but not how to improve it. Often the agent responds by just moving branches into helper functions.

Liina Suoniemi (opens in a new tab) measured this in a small test: when the guidance asked what the function is for, the share of genuine simplifications from Sonnet 4.6 went from 11% to 78%.

Tools like Habit Hooks (opens in a new tab) swap each message for short guidance.

Options end before the trunk. Each option is a throwaway build on its own branch, with a preview environment and test data, plus evidence: screenshots, a recorded click path and test results. Builds like these cost tokens plus the cost of running a preview.

The lasting cost arises on the trunk. Someone has to own the code there, and options that were never meant to coexist get in each other’s way. Whatever the team does not pick, the factory removes, and the feature file records what was built and why it ended.

Measurement is part of the acceptance criteria. If the success model says team leads open, edit and share the draft, the factory builds the events for that as well, and a check makes sure they arrive. Leave this out, and the agreed number is missing when the deadline comes.

In the example: the build

Each option gets 150 euros for tokens and no more than six correction rounds, which caps four options at 600 euros in tokens.

The intake asks what the weekly summary should show for a project with no activity that week. Every option runs against test data from three typical teams, one of them with patchy task data.

The risk alerts burn through their six correction rounds, because the patchy data keeps throwing up new edge cases. The run stops there, with an incomplete preview. So far, not one line has reached the trunk.

Third break: built is not shipped

Now comes the test drive. The product manager, design and engineering click through the previews and, where it helps, show them to a few customers. The forecast skill rates the options again, this time with the evidence the factory produced.

Selection stays strict, because market feedback is scarce and slow. It simply happens later and with better information.

In the example: the test drive

Third break: built is not shipped
Built option Before After Observation
Editable weekly summary 45 to 60% 60 to 75% Cited sources and the ability to edit raise the expected trust
Slack/Teams digest 55 to 70% 50 to 60% No control and no way to correct it inside the routine
Questions about the project status 30 to 55% 45 to 60% The actual interaction is more useful than its description
Risk alerts 50 to 75% 25 to 45% Patchy data produces too many false alerts

Before the build, the digest and the risk alerts had the highest forecasts. A team going by the brochure would most likely have built one of those two. After the test drive, the weekly summary is in the lead and the risk alerts are at the back. The digest lacks exactly what the weekly summary has: a way to correct it.

On paper, the risk alerts’ problem took up half a sentence. In the preview, it fills a page with false alerts. The factory learns from this: from now on, for every feature that generates content from project data, the intake asks how it behaves when the data is patchy.

What gets accepted is the option, not the diff

A pull request is where someone checks a single change. Agents produce such changes faster than humans can read them. Smaller pull requests, with agents approving some of them, take off some of the pressure, as at PostHog (opens in a new tab). The real decision is made one level up: what the product accepts is the option, not the diff.

The build of the chosen option does not get merged. The throwaway build existed for the decision, not to last. It is the template from which the factory rebuilds the feature on the trunk in small steps, with the same checks as any run and its own budget. It counts as accepted once it passes the option’s acceptance criteria and click paths. Engineers read the risky steps, and a review agent checks the rest. No human reads the code of the options that were dropped. Rebuilding rather than merging, and clearing out rather than keeping, both depend on one assumption: with token costs of a few hundred euros, rebuilding is cheaper than keeping code and maintaining it.

Shipping with an expiry date

On the technical side, the third break depends on separating deploy from release. The factory puts the feature into production behind a feature flag and gives it an expiry date, the deadline from the success model. If error rates or latencies climb above the agreed thresholds, the factory rolls the change back by itself.

Whether the impact materializes only becomes clear over the weeks up to the deadline, eight in the example. The product manager decides who sees the feature in that time. This is where the outer loop sets the pace and the inner loop waits.

In the example: shipping

The test drive costs three people two hours each. The team ships the weekly summary to a limited group of customers, with an eight-week deadline. The questions about the project status were the pleasant surprise of the test drive, and the team keeps them in mind for later. The feature files, with their forecasts and evidence, stay, and the builds of the discarded options are removed.

Fourth break: a feature has to earn its place

In our experience, once something has shipped, it stays in most products until someone musters the energy to remove it. Without a deadline, the outer loop has the same problem as the inner one: it does not end. Yet the experience of the big experimenters (opens in a new tab) speaks against making staying the default. Microsoft reports that roughly one third of the ideas it tests have a positive effect. At Booking.com, nine out of ten such ideas fail to improve the metrics.

The expiry date turns the default around. When the deadline passes, the factory removes the feature again unless a human decides to keep it. That default is itself a decision, one the team made in advance when it set the deadline. This way the outer loop gets a limit too: the inner loop has a budget in tokens and correction rounds, the outer loop a deadline in weeks. In this sequence, the market does the final review, and what it reviews is a whole feature.

Basecamp uses the circuit breaker (opens in a new tab) for projects: anything not finished within the set time gets no extension by default. We apply it to shipped features. The team decides how strict the deadline is.

  • The hard form removes whatever does not show its impact.
  • A middle form allows an extension.
  • The mildest form removes nothing and only forces the decision on the due date.

In every case, the team appends the actual data to the feature file. The data exists because the factory built the measurement along with the feature.

In the example: the result

After eight weeks, 28% of team leads in the customer group have opened the draft in at least three weeks, beating the target of at least 20%. But only 38% of these team leads edited and shared it, short of the expected half or more. Conversations show one reason: team leads would rather copy the text into their own email.

The opening target is met. Whether the draft replaces the hand-written status remains open for now, because someone who copies the text into their own email does not register as sharing in the measurement. The success model defined sharing too narrowly. The product manager grants one extension of four weeks, and the factory now also counts everyone who copies the text. If the second target is still missed after that, the feature is removed.

In the four weeks of the extension, the figure is 44% of these team leads, up from 38%, copying included. The second target is still missed. The conversations reveal what team leads copy: single paragraphs. They rewrite the rest, so the status takes them as long as before. The impact hypothesis therefore does not hold: the draft does not replace assembling the status by hand. The product manager decides to remove the weekly summary. The factory takes it out, and the customer group hears the reason beforehand.

Fourth break: a feature has to earn its place
Target After eight weeks In the extension
Opened in at least three weeks at least 20% 28%
Of those, edited and shared at least half 38% 44%, copying included

The test drive had moved this option to the front, and it took the deadline to show that it misses its second target. The feature file now holds something nobody knew before: team leads want building blocks for their status, not a finished draft. That speaks for the questions about the project status, the pleasant surprise of the test drive. They enter the four breaks as the next option, with a sharper impact hypothesis.

Even so, the work on the weekly summary is finished: the market has answered, and a human has decided. Left to itself, the factory would have kept on building.

Of seven ideas, one reaches the market in the example, and after the deadline and one extension the product manager decides to remove it again.

Figure 2: Of seven ideas, one reaches the market in the example, and after the deadline and one extension the product manager decides to remove it again.

What this means for revenue and time

So what about revenue? It depends on the outer loop, and in our experience agents have not made that loop any faster. More output helps there mainly when it leads to a better selection of what reaches the market.

And the time saved, the time writing code no longer takes up? It does not materialize on its own.

Aruna Ranganathan and Xingqi Maggie Ye (opens in a new tab) published interim results of a study in early 2026. Over eight months they followed a technology company with about 200 employees. With AI, people there worked faster, took on a wider range of tasks and put in longer hours, often without anyone asking them to. The study covers several functions, not just engineering.

Our reading: saved time fills up with more work by itself. In teams with a factory, we see that this extra work is mostly more building, which is the inner loop. Teams that do not plan this time lose it there.

That time belongs in the outer loop, in impact hypotheses, test drives and conversations with customers. Working this way has always made sense. In our experience, there was little time left for it as long as building tied up the team.

What this model cannot do

The benefit of the test drive is an assumption. It seems plausible to us that teams choose better after a test drive than after a brochure, but we have not measured it. What is more, the team does most of the test drive itself, and we suspect it goes easier on its own preview. Showing it to a few customers helps.

The forecast is not calibrated. Whether language models can usefully estimate a feature’s impact is an open question. The forecast does a first sort and forces people to write down their assumptions. For now, nobody should trust it with more than that.

Four decisions can become the new queue. The strongest objection to any human decision in the flow is that humans do not scale. But the four breaks apply per idea, not per change, and the factory prepares each decision with evidence. Anyone who still brings more features to a decision than they can handle has only moved the bottleneck.

Maintainability does not raise its own alarm. Findings about structure and maintainability go on the list, and no bug or alert reopens it. Teams that want to keep maintainability from decaying turn it into a check.

Removing has a price. Taking away a feature customers already use can cost trust, which is why shipping starts with a limited group. Only something built to be removed can be removed cleanly: its own flag, its own data, no interface that others depend on. That is why the removal path belongs in the acceptance criteria.

The deadline can be too short. In our experience, small groups of B2B customers often do not produce a clean signal within a few weeks. So the team sets the deadline feature by feature.

Not everything has options or an expiry date. Nobody builds a migration or a regulatory change four times, and nobody removes one automatically. The impact hypothesis and the measurement in the acceptance criteria apply there just the same.

What you can do on Monday

You do not need a factory for the first three steps.

  1. Write the impact hypothesis for the next three features in your queue. If you cannot, you have found a feature that should not move on.
  2. Add the measurement to the acceptance criteria of your next feature.
  3. Give your next feature an expiry date, and on the due date decide whether it stays.

Steps 4 and 5 require agents.

  1. Have your review agent work only against acceptance criteria and checks. Defects it demonstrates become new criteria, and everything else goes on the list.
  2. Have two options built for one single need, with a budget, and decide only once you see the previews.

When the work is finished

Once a success model is in place, the fact that a factory cannot stop on its own is no longer a flaw. Both loops then have their break: the inner loop ends at acceptance criteria and budget, the outer loop at the deadline from the success model. The work is finished when the market has answered and a human has decided whether the feature stays.

Sources and limits

  1. Verification in a Software Factory When generating code gets cheap, a Factory's throughput depends on how well the system can establish that a change is acceptable, safe, and worth shipping. 3 min read
  2. Why faster coding does not guarantee faster delivery Productivity research measures task time, perceived savings, pull requests, and production flow, each in its own way. Read side by side, the measures show where the constraint moves. 4 min read

All 5 articles on Specification and review

ClientsClient work

Scaling needs named owners

Coding agents scale across teams when named people own the platform, the team practice, the guardrails, and the measurement. Rollout & scale is where that ownership is built and handed to internal owners, in your stack.

Training on this topic: Agentic Engineering for Product Managers and Product Owners
See Rollout & scale

Want to check one team first? The self-check: twelve yes or no questions, three minutes, instant result, no account. Take the self-check