Skip to content

Knowledge base

Harness foundations

The components and control layers that turn a model into a dependable engineering agent.

Written by

Updated:

Software Factories 5

The components and control layers that turn a model into a dependable engineering agent.

What a harness is

The word “harness” carries three meanings, and only one is the subject of this page. An eval harness benchmarks a model’s raw capability, feeding thousands of standardized questions (MMLU, HumanEval) to score accuracy; EleutherAI’s lm-evaluation-harness is the canonical example, and it is irrelevant to application work. A client harness is the application that gives an otherwise stateless text predictor hands: Cursor, Windsurf, Claude Code, and OpenCode manage the context window, make the API calls, and grant the model permission to read files and run commands. You buy or install that one; it is the vehicle you drive. The project harness is the one you build: the custom, repository-specific infrastructure that constrains, guides, and verifies the agent’s work, made of the test suites, type-checking configs, centralized docs, and agent-config files the tool reads on every task.

If the model is the engine and the client harness is the vehicle, the project harness is the paved road, the lane markers and guardrails without which a powerful agent drives into a ditch. Harness engineering is the practice of building those guardrails, which moves the engineer from writing implementation logic to designing the environment that holds the agent to writing correct logic on its own.

A practitioner interview draws the same line with different terms. It calls the tools and integration points a coding client exposes the inner harness, and the repository-specific environment and customizations a team builds around it the outer harness (Context engineering interview (opens in a new tab), July 2026, practitioner and vendor). Outer-harness engineering is where the team’s own work lands, and it raises the quality floor of every turn the agent takes (Context engineering interview (opens in a new tab), July 2026, practitioner and vendor).

A second practitioner account widens the frame by placing the harness inside a stack of four layers, each carrying its own build-or-buy decision: the compute the agents run on, the development environment they act in, the harness itself, and a control plane above it that dispatches work, keeps session traces, and holds budgets. The value of the split is that the decision is taken per layer rather than for the whole stack. A team can rent compute and adopt an off-the-shelf client harness while building its own control plane, and the layer worth owning is the one where its constraints are unusual (Boundary - How to Build a Software Factory for AI Coding Agents (opens in a new tab), August 2026, practitioner and vendor).

The same conversation names the interfaces between those layers as the weak point. Between the control plane and the harness the open candidates are narrow. ACP connects a coding harness to an editor or web app, and AG-UI broadcasts a harness’s interface events. Neither supports hooks, the lifecycle events a control plane needs to react to. Hooks are also where harnesses differ most: Claude Code, Codex, pi and OpenCode each expose a different model, so the hosts doubt a shared abstraction is possible yet. Their complaint about today’s products is bundling. Buying one layer usually means buying every layer below it, so a team that only wants orchestration adopts a vendor’s harness and sandboxes too. Composition would let it choose per layer. In their account, enterprise buyers keep compute and parts of the development environment. They want the harness swappable, because harnesses change faster than the rest of the stack. They expect no shared open-source control plane soon. That layer encodes each company’s own definition of work, such as what counts as an issue, and is now cheap to write (Boundary - How to Build a Software Factory for AI Coding Agents (opens in a new tab), August 2026, practitioner and vendor).

In the loop, on the loop, above the loop

A project harness changes the developer’s cognitive relationship to the code, and that relationship evolves through three phases. In the loop is legacy agentic work: the human is the primary actor, prompting each move, reading every diff, running every check by hand, so their physical reading speed stays the hard bottleneck on team velocity (the velocity trap). On the loop is modern harness work: the developer stops typing implementation and stops running the disciplines by hand, wires context, spec, tests, and gates into the harness once, then supervises the result one level of abstraction up.

The defining move is that when the agent fails, the developer fixes the harness rather than the code; a hallucinated schema means the harness is missing a constraint, so they add the pinning test or tighten the context and let the agent retry, which makes that whole class of mistake far less likely to recur because the constraint now lives in the harness rather than in the developer’s memory. Above the loop is the future state where humans direct fleets of specialized agents (an architect agent designs the harness, a coder agent implements, an auditor agent tries to break the code); it is still experimental for most enterprise teams, and mastering on-the-loop work is its prerequisite.

This progression is the individual half of a ladder that continues into the organization: on-the-loop mastery peaks at the Fleet Commander who directs parallel agents but is still the engine, so removing them cools the factory floor, and the next rung up is the Factory Operator who builds the system that runs the agents without a human in every loop.

Industry framings converge on the same ladder. Ona’s AI-SDLC framework names three stages, in the loop (the human is the primary actor and the gains stay bounded), on the loop (a developer orchestrates parallel agents “the way a manager checks in on direct reports”), and autonomous loops (a trigger such as an incident or ticket kicks off a chain where one agent decomposes, one fixes, one reviews, one tests); the stages are not sequential gates, and the framework’s operative claim is that the bottleneck shifts at each stage: at stage one to review (3 to 5x the PR volume, needing automated quality gates), at stage two to everything upstream and downstream of code (planning rigor, CI/CD and release throughput), and at stage three to the organization itself, “the quality of human judgment at the top” (Ona – The AI-SDLC Framework (opens in a new tab), 2026).

Ona’s field CTO states the factory commitment in the same terms, “incrementally moving the human out of the loop within the SDLC”, explicitly rejecting the weaker reading of one engineer running many parallel agents (Ona – The Missing Primitive for Agent Swarms (opens in a new tab), 2026).

In Chatto is Robots, a practitioner sorts a codebase into squishy areas that change easily and sturdy areas that carry weight for everything around them, and lets that split decide where attention goes. They read the changed-file list before any code, establishing what the diff touches before judging it. In the sturdy places, protocol definitions, public APIs, and routing, they read closely and review by hand. In the squishy places they lean on tests and conventions to catch mistakes, because a wrong call there is cheap to undo. The orchestrator follows the same logic: it keeps a human in the loop at the task, review, and merge boundaries instead of aiming for full automation, on the reasoning that those decisions are where judgment still pays off. This is one practitioner’s account of how the workflow operates, not a controlled study or a general finding. (Chatto is Robots (opens in a new tab), July 16, 2026)

The six components of a harness

A complete project harness is the three disciplines (context, intent, feedback) made physical, plus the capability layer an agent needs to act. Six components do the holding, and they group by the job they cover: what the agent should know before it acts, what it can do while it acts, and how its work gets verified before it lands.

  • Know: Rules and Context management. Rules are the things the agent cannot work out on its own, the conventions and pointers neither baked into the model nor reliably derivable from a brownfield codebase that is inconsistent or wrong in places. They carry the architectural boundaries that keep the agent’s blast radius small (force all database access through a repositories/ folder, mark a module off-limits during a rewrite, point at the one service that gets a pattern right) and are injected through an agent-config file the tool reads on every task, sometimes a shared AGENTS.md. Context management controls retrieval (dense, discoverable maps so the agent loads the relevant pieces without overflowing the window) and memory (actively pruning what no longer holds).
  • Do: Tools and Skills. Tools give the bare text predictor hands: run a test command, read a file, query a database, call an API. The Model Context Protocol is one open standard for wiring a capability once so any MCP-aware client can reach it, reached for when the client harness lacks a capability out of the box. Skills are reusable, model-invoked packages of instructions and scripts the agent pulls in on demand; the most important ones are the process steps engineers used to run by hand (write the specification, then red-then-green tests), now packaged so the harness invokes them in order and authors the procedure once instead of re-explaining it per prompt.
  • Verify: Hooks and Verification gates. Hooks are the harness’s reflexes, deterministic actions that fire at points in the loop (format a file on save, block a command touching a forbidden path) that the agent cannot skip or argue with. Verification gates are the checks every change must clear before it lands: linters, formatters, strict static type checkers, and the test suite. The gates make a result necessary, not sufficient; an agent can pass every gate with a multiply() that returns the wrong product, so they catch the mechanical layer (type mismatches, a violated schema, a failing assertion) but cannot judge whether the logic is right.

The per-task specification is not a seventh component; it is the input the harness consumes. A vague ticket in human English is a suggestion, and the specification is what the agent is held to.

The artifacts these components ship as recur across every published implementation, and the recurring ones are worth copying rather than reinventing. Distributed AGENTS.md or CLAUDE.md files (one per major module, read at the start of every session, containing architecture, conventions, and current state) are the universal pattern under different names. Anthropic’s JSON feature list doubles as project spec and progress tracker, one entry per feature with its verification steps and a pass/fail status, which the agent reads at session start to pick the highest-priority failing feature; the file is JSON rather than Markdown for a specific reason, that models are less likely to inappropriately overwrite JSON, which matters when the agent runs unattended for hours.

The same account describes agentic development built on project-specific skills rather than a single shared setup. Each project keeps its own set: a glossary that fixes the domain terms, an architecture inventory that catalogs the system’s components, and decision records that capture the reasoning behind them, both architecture decision records (ADRs) and feature decision records. These skills hand the agent the project’s own vocabulary and history to work from, so its output stays aligned with how the codebase was actually designed. (Chatto is Robots (opens in a new tab), July 16, 2026)

The workflow keeps a short current-status block at the top of the project’s AGENTS.md, and it does real work. It records whether Chatto is public yet, whether real users and real data exist, what compatibility the project owes them, and which version comes next. Those few lines change how an agent decides: the same refactor reads as safe on a private prototype with no users and as reckless once people depend on the data, and stating that difference up front is what keeps the agent from shipping an accidental breaking change. This is one practitioner’s account of what worked on one codebase, not a controlled result, but the mechanism reuses easily: an agent that knows the project’s current reality makes better calls than one working from the code alone. (Chatto is Robots (opens in a new tab), July 16, 2026)

A practitioner series on building a software factory from the bottom up sets a reproducibility bar for all of this: the setup is a versioned artifact. Repository instructions, project-scoped skills, hooks and plugin versions belong to it; skills installed only on one engineer’s machine and the agent’s personal memory do not, so a workflow that works only because the agent remembers an earlier session has not been made reproducible yet. The same account keeps the root instruction file small, states each convention in exactly one place, skips what the agent can read cheaply on its own such as package scripts, and uses pointers that load a convention only when the task needs it. Its one measurement, self-reported without method or sample size, is that repository guidance cut token use by roughly 10 to 20 percent in its planning scenarios and did not improve the plans where the code already made the pattern clear. The larger effect came where the code holds two competing patterns and the guidance names the one new work follows, because that changed which approach the agent chose. Observability in this setup is collected by lifecycle hooks rather than reported by the agent, recurring failures in the traces become eval cases for the skills, and formatting, linting, types and dependency boundaries are enforced by commit hooks or required CI instead of an instruction to remember them (How I’d Build a Software Factory Today: Part I (opens in a new tab), October 2026).

The series stacks its factory in five layers, foundations, capabilities, orchestration, task recipes and automations, and requires each to pay off on its own, so a team can adopt guidance and evals without ever running a scheduler that picks up tickets (same source).

The session initialization routine is the boot sequence that makes a fresh context window productive: confirm the working directory, read git log and progress files, pick the next incomplete feature, start the dev server, verify end to end, implement one feature, commit. Without it the agent burns its first twenty minutes rediscovering what is already done. Red Hat’s structured task template analyzes the real codebase with language-server tooling first and emits a task with real file paths, real symbol names, and concrete acceptance criteria, so nothing is hallucinated. And the sprint contract has the generator propose what it will build and how success gets verified, with the evaluator reviewing the proposal before any code is written (Harness Engineering: What Every AI Engineer Needs to Know in 2026 (opens in a new tab), April 2026).

Every one of them answers the same question: what does the agent need to know before it writes a single line? The answer is where it is, what is already done, what “good” looks like, what is off limits, and how to check its own work. That is not intelligence, it is context.

Which harness you need depends on the problem you actually have, and the three camps solve different ones. A solo developer or small team gets most of the value from AGENTS.md files plus a solid CI pipeline, which is the OpenAI pattern at its simplest and mostly already installed. A team whose quality failures are visible to users should add an evaluation loop, though not necessarily a full three-agent architecture, because the principle that scales down is separating the doer from the judge. An organization rolling agents out across many teams should invest in the ThoughtWorks taxonomy, mapping existing linters, tests, and CI into the feedforward/feedback and computational/inferential grid to find the gaps, and building harness templates for common service topologies so the per-service cost falls. And in a regulated industry, treat the harness as the control framework, because that is what an auditor will eventually ask about: the append-only event log is an audit trail, and the structured task templates produce documentation that maps to compliance requirements (same source).

A practitioner demo adds a building block the list leaves implicit: a cheap model that only classifies. It answers a yes-or-no question, picks from a fixed list, or returns scores with a confidence, and the question and options arrive as structured data. Its familiar use is a pre-tool hook asking whether a shell command is irreversible, which also catches destructive commands that no deny list names. The demo moves the same call into the agent loop. At a turn boundary it decides whether a task switch makes this the moment to compact, and in front of a fleet it routes a request to the agent or model that should take it. Exposed as a tool, it answers the main agent’s question about a file, such as whether it validates tokens or which layer it belongs to, without the file entering the main context, and it runs one question across a whole glob in parallel. The demo promotes one product, and its cost multiples are the presenter’s own and unverified (10 Levels of Jev For Agentic Engineers (opens in a new tab), September 2026, practitioner). The vendor’s documentation limits the gate use. The model does not treat its input as hostile, so an injected instruction or text that argues for its own classification can move the answer, and the vendor expects to improve on this later. A classifier in front of a destructive command therefore adds a filter and does not replace the deny rules (TypeSafe - Jev 1.13 jaggedness (opens in a new tab), reviewed October 2, 2026).

The control view: guides, sensors, and free layers

The six components sort by job; a second, orthogonal cut sorts the same machinery by how it acts on the agent. The article frames a harness in control-theory terms as two kinds of regulator: guides (feedforward controls) that anticipate and shape behavior before the agent acts, and sensors (feedback controls) that observe after it acts and drive self-correction. Each is either computational (deterministic, run by the CPU, like a type checker or a formatter) or inferential (semantic, run by a model, like AI code review or LLM-as-judge). The highest-leverage sensors are written for the agent to read: a linter message that embeds the fix is “a positive kind of prompt injection”, correcting the next attempt instead of merely failing it.

The two kinds of sensor also have an order. The continuous delivery community runs the computational ones first, as hooks for linting, type checks, secret scanning and static analysis, and invokes a review agent only after they pass. A mechanical failure then stops the change before anyone pays for an inferential review (MinimumCD.org and DojoConsortium.org - Getting Started: Where to Put What (opens in a new tab), as of March 2026).

A web book on delegation calls the fast half of this feedback backpressure and gives it a budget. A check supplies backpressure when it is attached to a step the agent already takes, such as saving a file, building or committing, and answers soon and cheaply enough in time, tokens and context to be repeated during the work. Its examples are a pre-commit hook that rejects a name breaking the project’s rules, the build step of a strict language, an architecture fitness test run on every build, and a check on saved interface files that lists the approved palette next to the offending color. A small model-based check qualifies too, once it has been evaluated against reviewed examples and its cost measured; anything slower belongs at a review gate. The output counts against the budget: return the failing case with enough detail to investigate and keep success messages short, because a check that takes longer than the work or floods the window spends the context the agent needs to act on it (The Elastic Loop: Harness (opens in a new tab)). The book also keeps four mechanisms apart that a failed run tends to blur. A document of acceptance criteria is context, running a check against it is backpressure, a permission boundary limits an action, and a gate decides whether work may proceed, so diagnosing a failed run starts by asking which of the four was missing (The Elastic Loop: Harness (opens in a new tab)).

The article’s harder point is that regulation splits into a maintainability harness and an architecture-fitness harness, both tractable, and a behaviour harness that is still “the elephant in the room” because it leans on AI-generated tests that are not trustworthy yet (approved fixtures are a partial fix). Codebases also differ in harnessability: strong typing and clean module boundaries hand you sensors for free, which produces the legacy paradox that the harness is most needed where it is hardest to build. Committing to a fixed service topology is itself a control, a variety-reduction move in the sense of the Law of Requisite Variety, and the open problem is that there is still no coverage metric for a harness the way code coverage measures tests (Harness Engineering, martinfowler.com (opens in a new tab)).

An information-theoretic account sharpens the same intuition into a rule and a shopping list. An LLM is a noisy channel; the specification is the signal and the harness is the error correction, so the deciding property of any layer is whether it is closed-loop, meaning the agent can read its output and self-correct, rather than out-of-loop like a PDF audit no agent will ever see. Layers sort economically by where their rules are defined: extrinsic ones come from outside the project (CVE scanners, WCAG, OWASP, type checkers) and cost almost nothing to switch on, while intrinsic ones (project tests, ADRs) must be authored. Roughly twenty free extrinsic layers exist and most repositories run fewer than a third of them, so the pragmatic floor is to enable every free layer before paying tokens to chase errors a linter would have caught.

Language strictness is error correction in this frame: strict modifiers raise the channel capacity, a concrete reason to prefer Rust, Kotlin, or F# as agentic-coding substrates, and mandatory layers should scale with risk from prototype to safety-critical. What no harness catches is also explicit: wrong or missing requirements, logic vulnerabilities, and the “theory of the program” (the 1985 programming-theory account) that lives only in developers’ heads; automated accessibility checks, for instance, find only 30 to 50 percent of WCAG problems (The Harness Inventory: Semantic Anchors (opens in a new tab), 2026).

Agent equals model plus harness, and the harness decays

ThoughtWorks reduces the discipline to one equation, Agent = Model + Harness: the harness is everything that is not the model, so the same model wrapped in a different harness is a different agent. The claim is measurable. An agent-framework vendor ran one fixed model on Terminal Bench 2.0 under its old harness and its new one and moved the score from 52.8 to 66.5 percent; Vercel deleted roughly 80 percent of its agent’s tools and performance rose (Harness Engineering: What Every AI Engineer Needs to Know in 2026 (opens in a new tab), 2026). Three teams reached the discipline from different problems and converged on the same five principles: context beats instructions, planning is a separate step from execution, feedback loops are non-negotiable, work proceeds one atomic unit at a time, and the codebase is the documentation.

OpenAI came at it environment-first, designing the repo so thoroughly that agents produce reviewable output, then letting them loose: the Sora Android build shipped with four engineers in 28 days at a 99.9 percent crash-free rate, Codex authoring 70 percent of weekly PRs. Anthropic came at it multi-agent, and ThoughtWorks built the taxonomy (same source). Anthropic’s answer is a GAN-inspired split (a planner expands a one-to-four-sentence prompt into a full spec, a generator implements one feature at a time, an evaluator drives the running app through Playwright and grades it, with a sprint contract negotiated between generator and evaluator before code is written), which exists because self-evaluation fails: agents confidently praise their own mediocre work, and tuning a standalone evaluator to be skeptical is far more tractable than making a generator critical of itself.

Context resets (clearing the window and handing off through a structured artifact) also beat in-place compaction when a model suffers “context anxiety” and wraps up prematurely, though a strong enough model removes the need (Harness Design for Long-Running Application Development (opens in a new tab), 2026).

That last caveat is load-bearing. Every harness component encodes an assumption about what the model cannot do on its own, so as models improve those assumptions expire and the component becomes overhead, a phenomenon Anthropic names harness decay. The trend line is clean: the same build cost 200 dollars over six hours under the Opus 4.5 harness and 124.70 dollars in under four under the Opus 4.6 one, purely from dropping the sprint decomposition and per-sprint grading the newer model no longer needed. Anthropic’s launch post for Opus 4.7 claims the model devises ways to verify its own outputs before reporting back, a vendor self-claim that, if it holds, eats into the evaluator’s remaining job (Anthropic - Introducing Claude Opus 4.7 (opens in a new tab), April 2026, vendor). Agent builders report the same churn from their side. Manus wrote in July 2025 that it had rebuilt its agent framework four times, each time after finding a better way to shape context (Manus - Context Engineering for AI Agents: Lessons from Building Manus (opens in a new tab), July 2025), and by October a webinar with the team put the count at five since the product’s March launch (Context Engineering in Manus (opens in a new tab), October 2025). The builder of a widely used open-source deep-research agent says it had to be rearchitected three or four times in a year to keep up with improving tool calling (Delphina - From Context Engineering to AI Agent Harnesses: The New Software Discipline (opens in a new tab), November 2025).

The counterintuitive rule, from the article’s reading of the bitter lesson, is build to delete: give every component a kill switch, disable it periodically, and remove it the moment output quality holds without it, because a stale component taxes every run in tokens and latency for zero quality (Harness Engineering (opens in a new tab), 2026). An Anthropic workshop reads the same curve off METR’s time-horizon chart, the task length at which an agent on a minimal scaffold completes half the tasks: about one hour for Claude 3.7 Sonnet and about twelve hours for Opus 4.6 a year later, a twelvefold rise, and this is the capability curve that keeps expiring harness components (AI Engineer - Anthropic Workshop: Build Agents That Run for Hours (opens in a new tab), May 2026, vendor).

Operationalizing build-to-delete means treating every “don’t do X” rule as decaying inventory: pair it with an eval that re-checks whether the next model still needs it, and run a scheduled sweep that re-evaluates every rule, or they accumulate forever and drown the ones that matter; the mirror-image sweep keeps “pocket evals”, hard tasks the current best model fails, to catch the moment a new model makes a workaround removable (Inside a Software Factory (opens in a new tab), 2026). The mental model that survives the decay is context itself: the talk frames a harness as an LLM-as-fuzzy-compiler where the rules and docs are optimization passes on which code is acceptable and swapping models is swapping code-generation backends, so the durable work is surfacing the right text at the right time; even source structure becomes a harness lever, with tests that assert files stay under a line count or that a Zod schema keeps one canonical definition, and its bar is that “every time you have to type continue is a failure of the harness” to carry enough context to finish (Harness Engineering talk (opens in a new tab), OpenAI, 2026).

Which components survive is not random, and a practitioner account supplies the selection mechanism: models become agentic through reinforcement learning on coding sessions, which are mechanically evaluable (did the tests pass, did it commit) in a way “write a poem” is not, so capabilities overrepresented in that training data are durable harness substrate (Unix workflows, files and diffs, incremental edits, permission prompts, tmux, the Chrome DevTools protocol) while underrepresented ones stay weak regardless of harness effort (clicking proprietary UIs, tasks that resist textual decomposition, low-volume observability interfaces). Building the harness out of what the models are being trained toward is what makes it survive model swaps; a minimal agent like pi runs on four tools and still works.

The same lens names a sharp failure mode: state changes that never enter the session data confuse the model, which is why Claude Code injects a “this file changed, reread it” message on external edits, and why a reloaded agent, unaware it had been restarted, fabricated having re-validated a fix it never executed (talk on how coding agents work (opens in a new tab), 2026).

Resources

Practitioner accounts

Vendor accounts