Skip to content

Knowledge base

Quality and productivity metrics

Under agentic coding, volume metrics such as lines of code or pull requests become harmful, because rewarding volume floods review with unchecked output. The measures that hold are first-pass acceptance, rework rate and the DORA delivery metrics, read for agent changes against the human baseline.

Written by

Updated:

Agentic Engineering 5

Our position. We count a change once it runs in production (opens in a new tab) and customers use it, pay for it or tell us it helped. Merged pull requests are output. AI is great at productivity and neutral on value (opens in a new tab), so the measures we keep track what reaches production and what breaks there.

Every measure below is read against a baseline: the team’s own numbers before adoption, or the human change failure rate for agent changes. A metric that becomes a target gets optimized and stops measuring the capability behind it.

Which metrics still mean something with AI?

Counting lines of code or pull requests, always a poor proxy, becomes actively harmful under agentic coding. Thoughtworks puts coding throughput as a measure of productivity in its Caution ring, because rewarding volume floods review with insufficiently checked output, lengthens cycle times through repeated reviewer back-and-forth, and hides the residual effort of adapting generated code to a team’s conventions (Thoughtworks Technology Radar Vol 34, April 2026 (opens in a new tab)). The metric that replaces it is first-pass acceptance rate, how often AI output is usable with minimal rework. A low rate exposes hidden effort and is actionable, since better priming documents and design conversations raise it over time, and it connects to delivery outcomes because low acceptance drives up change failure rate while repeated iteration extends lead time.

The DORA program’s metrics remain the durable backbone, and the 2025 edition adds a fifth, rework rate, the share of the delivery pipeline consumed by unplanned fixes to work previously called done; a rising rework rate is an early warning that AI-assisted speed is buying instability rather than throughput (Thoughtworks Technology Radar Vol 34, April 2026 (opens in a new tab)). For the human-agent interaction itself, Thoughtworks recommends measuring collaboration quality at the team level, first-pass acceptance, iteration cycles per task, post-merge rework, failed builds, and review burden, rather than speed alone, since shorter feedback cycles and fewer failed builds are what actually signal effective work with an agent. This is the same conclusion the 2025 DORA report reaches from the adoption side: with AI adoption near universal, adoption is the wrong thing to count, and measurement has to pair existing DORA and DX baselines with AI-specific signals to show whether outcomes improve (DORA 2025 Report and AI Strategy).

DORA also warns against a subtler failure: targeting the four key delivery metrics directly is a known pitfall, because teams optimize the number rather than the underlying capability and lose the outcome; the durable move is to build the capabilities (version control, continuous integration, small batches, loose coupling) that the metrics only measure. This matters more under AI, where DORA’s early findings show developer happiness rising while software delivery performance falls, more bugs reaching production and worse code quality, so the metric that would have flattered the team most (they feel productive) is exactly the one that misleads (WeAreDevelopers 2025 talk on DORA and modern software development (opens in a new tab)).

For agent-generated changes the continuous delivery community proposes four measures, each read against the human baseline. Agent deployments are tagged, and their rollback and incident rates are compared over 30-day windows, with the target at or below the human change failure rate. Review time for agent changes should match that for human changes, and a much faster review signals rubber-stamping rather than efficiency. Test coverage on agent-written files should exceed the team baseline. Every agent change should carry its full artifact set: intent, test specification, feature description and provenance such as agent identity and prompt. A monthly audit checks that set against a 100% target. The same page traces its adoption failures back to two gaps, a skipped per-change delivery contract and missing pipeline enforcement (MinimumCD.org and DojoConsortium.org - Pitfalls and Metrics (opens in a new tab), as of September 2026).

Does AI-coauthored code draw more review findings?

CodeRabbit, a code-review vendor, published an observational study of 470 open-source pull requests, classifying 320 as AI-coauthored from commit and content signals and treating the remaining 150 without such signals as human-only; it applied a structured review-finding taxonomy and normalized finding rates (CodeRabbit – State of AI vs Human Code Generation Report (opens in a new tab), 17 December 2025). Read it as a vendor observational study of automated review findings that were not independently adjudicated and do not establish production quality.

The reported gap is 10.83 automated review findings per AI-labelled pull request against 6.45 per human-labelled one, about 1.68x overall, and the AI-labelled side carries a heavier tail of high-finding pull requests rather than a uniform shift. The category profile is uneven: readability, business logic, formatting consistency, and error handling weigh heaviest on the AI-labelled side, while spelling and testability favored the AI-labelled pull requests (CodeRabbit report, 17 December 2025 (opens in a new tab)). No finding category was unique to AI; the report’s claim is that frequency differed, not kind.

The practical read, which partly supports CodeRabbit’s own product category, is to aim review attention at local business rules, exception paths, concurrency and dependency ordering, configuration, credentials, and I/O, while keeping deterministic policy, test, and security gates in place. The limitations bound the claim tightly. Authorship is inferred from explicit signals, so the human-labelled pull requests may themselves contain AI. The sample covers only open-source CodeRabbit users. The vendor owns the taxonomy and the reviewer and sells the remedy. There is no raw reproducible dataset, no independent human adjudication, and no controls for pull-request size, language, repository, author, task, or complexity. It counts the findings an automated reviewer raises, which measures review load, and without those controls it cannot show that AI authorship causes lower quality (CodeRabbit report, 17 December 2025 (opens in a new tab)).

Resources

Delivery practice

Quality research