Why faster coding does not guarantee faster delivery

The studies measure different things

Developer productivity research often seems to contradict itself. The reason is that the headline numbers answer different questions.

Task completion time is how long one person needs for a bounded task. Self-reported time savings capture perception. Pull-request throughput counts accepted changes. CI activity counts attempts to integrate work, and main-branch success tells you whether those attempts pass. None of these, taken alone, tells an organization whether customers got value any sooner.

A fair comparison needs the population, the task, the measurement window, and the quality guardrail stated next to every number.

What METR found, and what changed

In METR’s early-2025 randomized study, 16 experienced open-source developers took 19 percent longer when they could use the products available at the time. They had expected to be faster, and once the work was done they still believed they had been.

The setting matters here. The developers worked in mature repositories they knew well, and the tasks demanded the kind of context that small coding benchmarks usually remove.

METR went on to run a larger follow-up with newer products. In February 2026 it reported that the new design could no longer deliver a reliable estimate of current speed. More and more developers turned down tasks that barred agent use, and running agents concurrently made time harder to measure. The raw data pointed to improvement, but METR described the evidence for its size as weak.

An honest reading keeps both results. Early products slowed this group of specialists in one realistic setting. Newer products look more useful, yet the current effect is still hard to estimate with the same method.

The delivery system can absorb local gains

DORA’s 2025 research treats AI-assisted development as an amplifier of the system around it. Teams still need a healthy platform, clear policies, a focus on users, and reliable feedback. Writing code faster adds little when the work then waits for review, integration, a product decision, or a release window.

CircleCI’s 2026 telemetry shows where the constraint sits downstream. Across more than 28 million workflows, average activity was up 59 percent year over year. For the median team, activity on feature branches went up while main-branch throughput went down. Main-branch success stood at 70.8 percent, the lowest in five years in that dataset.

The top performers looked different: their change volume and main-branch throughput grew together. The report links this to validation and recovery capacity that kept up with generation.

Adoption is not a result

In its Q1 2026 company sample, DX reports industry adoption of 93 percent. At that level, comparing users with non-users tells you little. The better question is what changes inside a team as its usage deepens.

DX found throughput uneven and quality volatile. Some teams got better. Others saw more defects or weaker maintainability. Self-reported time savings outran the longitudinal changes in PR throughput.

An adoption target can show that a product is in place. It cannot show that the organization delivers sooner or with more confidence.

A measurement set that holds up

Begin with the value stream, not the product license, and record a baseline before you change anything in the system.

Put lead time from idea to first users next to review time, rework, failure rate, and recovery time. Add developer confidence: a system can look healthy while the people running it lose track of how it works. Name the business outcome separately.

The measures should show where work waits and whether the constraint has moved. That tells you more than any search for a single universal productivity multiplier.

Two leading indicators from a practitioner interview

In a practitioner interview (opens in a new tab), Patrick Debois suggests two leading indicators for teams that run coding agents. The first is the number of human interventions a team still needs before the agent gets it right. As the harness, context, and guidelines improve, that number should drop. Break it down by risk or change type: a high-risk migration and a copy tweak attract different intervention counts, and a single average hides the pattern. The second is how far shared context, skills, and harness components are adopted and reused. If one team’s setup spreads and others reuse it, the investment compounds. If every team starts from scratch, it does not.

Neither indicator is an outcome, and neither is safe to use as a target on its own. Turn interventions into a target and a team can ship unreviewed risk. Force reuse and teams can standardize on the wrong pattern. Read both next to the flow, review, rework, failure, recovery, confidence, cost, and business-outcome measures above. Debois’s account tells you where to look. It is one practitioner’s view, not independent evidence that these signals predict delivery.