Evidence
The research on AI-assisted engineering finds that AI amplifies the practices a team already has: strong delivery systems get faster, weak ones ship more defects. Individual speed gains are real but uneven, and they reach the business only where review, testing and the operating model change with them.
Written by
-
Benedikt Stemmildt Founder & Co-CEO
Updated:
Evidence 7
Our position. A study about AI that does not account for model evolution has a shelf life measured in weeks, not years (why these results don’t hold up (opens in a new tab)).
Read a study for what it measured and on whom. The headline numbers differ by what each one counts: time saved per developer, pull-request throughput, defect rates or cost to serve.
What do the studies and benchmarks measure?
- AI Productivity Research Evidence — public, cited findings from METR, Stanford, DORA, and Multitudes, with guidance for comparing unlike measurements.
- DX Q1 2026 AI-Assisted Engineering Impact Report — DX’s quarterly benchmark across 400+ companies: adoption, time savings, PR throughput, and the Core-4 measurement framework.
- Amazon Cost to Serve Software (CTS-SW) — Amazon’s framework quantifying a 15.9% YoY cost reduction from developer-experience improvements.
- DORA 2025 Report and AI Strategy — the finding that AI amplifies whatever engineering practices already exist, and the telemetry that disputes it.
- The State of AI-Native Engineering 2026 — 219 engineering leaders: half the code is AI-generated, comprehension is the top fear, and the org has not changed around it.
Where are the trends heading?
- Moving away from Agile: What’s Next — McKinsey on the operating-model shifts that separate top AI performers from the rest.
What signals come from the market?
- OSS Trust-Gating in the AI-Contribution Era — the Vouch model as the first production answer to AI-generated contribution flooding.
Related
- AEMM industry evidence: how the same studies map onto maturity levels for an assessment.
- Productivity evidence and quality: what the evidence means for code quality in daily work.
- Measuring impact: how to measure your own team instead of borrowing another company’s number.
- Coding benchmarks: what model benchmarks measure, and why they differ from field studies.
- Leadership: the decisions that turn these findings into a changed operating model.
Resources
Posts
- Why these results don’t hold up (opens in a new tab) - LinkedIn post arguing that a study of AI-assisted engineering which ignores model evolution has a shelf life of weeks, not years.
In this section
- AI productivity studies: METR, Stanford, DORA and Multitudes
No single AI productivity number holds. METR measured experienced developers 19% slower in repositories they knew, Stanford found a median gain near 10% that shrinks as tasks and codebases grow, DORA finds AI amplifies the existing system, and Multitudes ties gains to learning time.
- Amazon Cost to Serve Software (CTS-SW)
Amazon's numbers on developer experience: what it measured, how the savings make a business case, what drives cost down and which metrics it rejects.
- The DORA report on AI-assisted development
The DORA 2025 report finds that AI amplifies the engineering system it lands in: strong review, pipelines and small batches gain speed, weak ones gain instability. Telemetry from Faros AI disputes that a strong foundation protects a team from downstream deterioration.
- The DX AI-assisted engineering impact report
DX's Q1 2026 benchmark of 400+ companies found 92.7% AI adoption and 3.9 hours saved per developer each week, yet quality swung both ways and CI waits and meetings still cost more time than AI saves. Enablement and measurement, not tool choice, decided the outcome.
- Moving away from Agile: McKinsey's AI operating-model research
McKinsey's survey of about 300 enterprises found AI lifts company-level productivity only 5 to 15 percent, because teams keep agile-era sizes, sprints and roles. Top performers were 7x likelier to run AI-native workflows and 6x likelier to have AI-native roles in smaller pods.
- Vouch in Ghostty: trust-gating open-source contributions
Ghostty merged the Vouch model in February 2026: only contributors a maintainer has explicitly vouched for may open pull requests, and denounced users are blocked. It answers floods of AI-generated contributions with explicit social trust rather than an accumulated reputation score.
- The State of AI-Native Engineering report
Augment Code's survey of 219 engineering leaders found that about half their code is AI-generated, yet only 57 changed code review and fewer changed roles, onboarding or measurement. Losing shared understanding of the codebase worried 55%, the top concern and the least addressed.