Skip to content

Blog

Clean codebases decide who gets value from AI

Stanford research on 120,000 developers, presented in a talk, found codebase cleanliness predicted AI value better than token spend. Teams with weak codebases regress past a certain spend.

Written by

Measuring AI impact

Published: 28 April 2026 Updated: 2 min read

Two engineering teams. Same model, same budget, same agentic stack.

One ships faster every month. The other regresses past a certain spend.

The variable that decides the split is not the model. It is what the codebase does to the model.

The signal from 120,000 developers

Stanford’s software engineering productivity research covers 120,000 developers. In a talk on that work (opens in a new tab), Yegor Denisov-Blanch matched 46 teams using AI with 46 similar teams that were not. Two findings broke the prevailing narrative.

First, the median productivity gain from AI in that cohort was about 10 percent.

Second, the strongest predictor of who got value was not which model, how many tokens or which IDE. It was how clean the codebase was: an R-squared of 0.40 against codebase cleanliness. Token spend showed a correlation of about 0.20, and that number is r, not R-squared. Squared, it is 0.04, a tenth of the 0.40.

The death valley past 10 million tokens

Around 10 million tokens per engineer per month, the talk saw a dip: more AI spend, worse output. The agent runs out of trustworthy context. It generates plausible code that breaks against conventions the codebase never wrote down.

The agent is not the bottleneck. The codebase is the contract.

Codebase cleanliness is a trust score

A clean codebase tells the agent what good looks like, what it must never do, and what gets verified before it ships. Conventions are gates the agent can pass on its own:

  • Tests that catch what reviewers cannot
  • Naming patterns the agent can imitate without being told
  • Architecture boundaries that fail loud, not silently
  • Linting and fitness functions that block bad output before it lands

The teams above the death valley did not buy more AI. They paid down debt the agent can read. R-squared 0.40 is the number on that ledger.

Buying tokens first makes the curve worse

Buying tokens before paying down structural debt makes the curve worse, not better. The teams that compound their gains over the year are the ones who treated codebase health as the operating budget for AI, not as something separate from it.

The rich do not get richer because they had more money. They get richer because their codebase is something an agent can be trusted inside.

Are you spending on tokens, or on the contract the codebase makes with the agent?

Further reading

  1. Boards want 10x, and most companies are not measuring The 10x number keeps showing up in board decks while few companies measure anything. The pressure rolls downhill, and the gap between what gets reported and what happens keeps widening. 2 min readShort post
  2. Measure AI impact, not AI usage Most companies track how much their developers use AI tools. The DX AI Measurement Framework adds impact and cost, and the metrics that survive measure what reaches production. 2 min readShort post
  3. AI inflates the metrics that don't measure value AI pushes up every number that is easy to measure: code volume, pull requests, velocity. Four metrics show whether the value went up with them. 1 min readShort post

All 16 posts on Measuring AI impact

ClientsClient work

Test the constraint in your system

The Factory Readiness Assessment connects this research to your value stream, your data, and the next decision.

Training on this topic: Agentic Engineering for Engineering and Product Leaders
See the assessment

Want to check one team first? The self-check: twelve yes or no questions, three minutes, instant result, no account. Take the self-check