Skip to content

Knowledge base

Roles and economics

In a software factory the engineer builds the system that builds the product, and success is the share of changes shipped automatically at a known cost. That moves software production from an R&D expense to cost of goods sold, where every dollar of tokens faces a return test.

Written by

Updated:

Software Factories 5

Our position. We count a change once it runs in production and customers use it, pay for it or say it helped; a merged pull request is output (A swarm is not throughput (opens in a new tab)). A token budget is an investment decision: it belongs with the people who decide what gets built, and goes first where faster delivery moves a business number.

That count sits one step past the factory metric practitioners use, the share of changes shipped automatically at a known cost.

How does the factory change the engineer’s job?

The factory model reframes the engineer’s job: you are no longer writing the product, you are building the factory that builds it. Fleets of agents, each with a task, a toolbelt, context, and a feedback loop, run in parallel while the human reviews outputs, refines specs, and redeploys. Taking the analogy seriously points at the real investments: quality control, process documentation, precisely specified inputs, and reliable environments, because generation is no longer the bottleneck and verification is (The Factory Model (opens in a new tab)).

How do you measure a factory’s success?

Warp’s CEO turns the reframe into a metric and a P&L line. Feature count becomes “a failure metric”; the success measure of a factory is the percentage of all changes shipped automatically and at what cost, with factory efficiency defined as shipped product over inference cost plus human time cost. The ten-step workflow bookends implementation with a triage agent upstream and a monitor agent that files issues downstream, closing the loop, and treats any fallback to an interactive human-in-the-loop agent as “a failure to learn from”; self-improvement agents that mine the factory’s own conversations for token waste and eval alternative harnesses and prompts are “the golden path”.

The stated economic consequence is that software production stops being an R&D expense and shows up as COGS on the P&L, ending unlimited token budgets and subjecting every dollar of automation to a return test; the existence proof for latent capacity is Warp’s own public factory instance holding 1,300 issues in a ready-to-implement state (We Are Now Factory Engineers, Not Product Engineers (opens in a new tab), 24 June 2026).

A podcast conversation with the same founder, published in October 2026, starts one step earlier. A good factory has high yield, and the speakers find no general definition of yield, because it differs by business. Their ladder climbs from pull requests to shipped features, then to customer outcomes, and finally to business impact. A customer outcome is a workflow that took 20 minutes and now takes 10. Each rung is more meaningful and harder to measure. Before product-market fit the ladder collapses. The only signal comes from a handful of users, few enough that a person can watch every session replay. A fully automated factory then has no yield to optimize against. Once usage is large, telemetry makes the factory measurable: a regression shows up after a share of users has upgraded, and the change can be reverted. The speakers place the blocker in the org chart. Where engineering owns one part of the loop and product the other, two separate organizations share the yield, and the speakers see no fast factory there. They call that a structural problem of people rather than of tooling (Boundary - How to Build an AI Software Factory That Ships Faster (opens in a new tab), October 2026, practitioner and vendor).

The founder’s own platform shows what such a measure looks like by default, as a vendor demo. Its automation index counts human interactions per pull request, a number to drive down rather than a yes-or-no state. Beside it sit DORA lead time and volume, and cost broken down by engineer and by factory stage. Recording every run is the stated prerequisite, because it allows online evals over a sample of past runs. There an LLM judge scores, for example, the share of runs that write redundant tests. Other judges flag dead ends, verbosity and an agent working out again what a skill should already hold, and the findings feed skill improvements. Split by model, such a score replaces opinions about a model with a measurement. The onboarding advice qualifies the earlier rule that every interactive fallback is a failure. Little work can be fully automated yet. So teams first route all work through the shared cloud loop, prompts from chat, the issue tracker or an MCP client included. That makes the work tracked and measurable, and a growing share of those prompts then becomes automations. A co-host argues the opposite order: first take people out of the existing loop, then encode the best engineers’ way of working as a pipeline. Both sides reject redesigning the whole process at once (same source).

Where do practitioner reports converge?

The “third age” framing (a term for a golden era where the developer’s job shifts from writing instructions to defining intent) had by March 2026 become the common vocabulary of a wave of practitioner reports that converge on the same shape. StrongDM defines a software factory as “non-interactive development where specs and scenarios drive agents that write code, run harnesses, and converge without human review”, with two hard rules: code must not be written by humans, and code must not be reviewed by humans (Software Factories and the Agentic Moment (opens in a new tab), 6 February 2026). The “dark factory” account borrows the manufacturing term for the same lights-off operation, and a widening adopter-laggard gap is its recurring warning: Block cut headcount from over 10,000 toward under 6,000 citing “a new way of working”, and Cursor reports 35 percent of internally merged PRs now come from agents in cloud VMs (The Dark Factory (opens in a new tab), 5 March 2026).

Ona’s public Memo experiment states the thesis most plainly, “you don’t build Memo, you build a factory to build Memo”, and reports the counterintuitive result that the operations side needed more automations than the build side (Ona – Software Factory: the Memo experiment (opens in a new tab)). The practitioner framing lands in the same place from the individual end: the developer-evolution ladder, originally eight stages topping out at “building your own orchestrator”, has since extended to eleven as operators deployed standing agent crews, adding shepherd (tending a flock of agents), garden (a deployed multi-agent crew running a business process 24x7), and factory builder (building a full custom orchestrator as architect, curator, and shepherd).

The Gas City account’s sharpest factory claim is economic: a three-to-five-engineer team running agent “packs” can credibly replace seven-figure SaaS bills by in-housing the 20 percent of features a company actually uses, keeping the capability as a compounding asset with a git-versioned audit trail instead of paying recurring rent. It reframes “dark” as merely background work where observability is a design choice, not a mandate to keep the lights off (Welcome to Gas City (opens in a new tab)); the orchestration mechanics themselves belong to multi-agent coordination.

Resources

Factory measurement

Articles

Posts

Tools and repositories