Harness maintenance

How to prevent configuration decay, remove stale machinery, and recognize harness anti-patterns.

Garbage collection is not optional

The memory half of context management is the half teams skip, and it is where a harness’s context rots. LLMs have zero intuition and treat every file as absolute truth, so a stale v1_architecture.md or fifty lines of commented-out legacy code gets read as currently valid, and the agent hallucinates an implementation that integrates with a database that no longer exists. Garbage collection is therefore a continuous, non-optional job the harness runs on its own, not a reminder to tidy up by hand. Three strategies build it in: a pre-flight cleanup packaged as a skill and triggered by a hook, which runs static analysis (like vulture in Python) for dead code and also checks the harness config itself for rules that no longer apply and maps that have drifted; context archiving that moves an obsolete ADR into an archive/ directory so it stays in the repo but out of the set the agent searches; and a post-flight retrospective where the agent reviews which iterations reached their goal in a couple of feedback loops and which churned through many, feeding the answer back into the rules, context, or tests so the next run of that kind needs fewer loops.

The retrospective is how the harness learns from its own history instead of repeating expensive corrections. At factory scale this same ritual generalizes into the Compound Factory, where the system runs retros on every cycle and mines its own logs to get better month over month. The discipline has a theoretical floor: forgetting is an intrinsic part of any functioning memory store, so a factory whose context only accumulates eventually degrades, and it has to forget as deliberately as it grows (Inside a Software Factory (opens in a new tab), 2026).

OpenAI’s Codex team reports the same ritual from a fully agent-generated codebase: they replaced a standing Friday cleanup that ate 20 percent of the week with background “doc-gardening” agents and codified “golden principles” that scan for drift and open small refactor PRs on a cadence, treating technical debt as a high-interest loan paid down continuously rather than in painful bursts. Two of their context lessons reinforce the map-over-manual rule. A single giant AGENTS.md rots into “a graveyard of stale rules”, so they keep it to a roughly 100-line table of contents pointing into a structured docs/ system of record and let agents load deeper sources on demand (progressive disclosure). And anything an agent cannot reach in-context effectively does not exist, so aligning it on a decision made in a Slack thread means writing that decision into the repository, the same way a new hire only learns what the codebase records (Harness Engineering: Leveraging Codex in an Agent-First World (opens in a new tab), OpenAI on its own Codex, 2026).

A continuous delivery guide adds the change-control view. System prompts, session templates, agent configuration and skills are delivery artifacts and live in version control next to the code they produce. An untracked prompt or stale skill is an unreviewed change to the delivery process, so the configuration is reviewed on the pipeline’s cadence and checked first when output turns strange, before blaming the model. Each skill carries an updated date so stale ones are visible, and a skill whose output needed heavy correction is fixed before its next run. Otherwise every later session repeats the same corrections. Context documents per workstream are reviewed quarterly: summaries of finished work are archived and the current-state description is rewritten to match the code (MinimumCD.org and DojoConsortium.org - Pitfalls and Metrics (opens in a new tab), as of September 2026).

Removing a component needs the same evidence as adding one. A web book on delegation works through retiring a reminder that once sent an earlier model to the approved component examples before it touched an interface. Compare the new model with and without the reminder on representative tasks, including those that needed it before, keep the acceptance criteria and independent review unchanged, and read quality, correction effort and total cost together. A reminder that rarely fires may be redundant, or the task mix may have shifted, so confirm that the log records its use before reading anything into the silence. Remove it through the release process and keep the previous configuration to restore if later work degrades. The approved components themselves stay, because they are product context rather than a workaround for a model (The Elastic Loop: Harness (opens in a new tab)).

A conference account adds a selection test for what earns a place in that always-loaded file. Admit only facts that stay true across months or years, the example given being the layers of the team’s own compiler, so the file reads as a fixed invariant whatever model or coding tool an engineer happens to use rather than a style guide that drifts. The team pairs the written invariant with a social rule instead of more prose: changing it requires talking to at least one other person first, which slows the risky edits down without spending an extra token (fighting slop with slop (opens in a new tab), July 2026, practitioner).

Charlie Labs makes the standing-maintenance idea explicit: its “daemon” is a role with an ongoing responsibility, not a one-off task, watching systems such as GitHub, Linear, Slack, and Sentry and turning recurring signals into PRs, issue updates, reports, escalations, fixes, or deliberate no-op decisions (Charlie Labs – Daemons work 24/7 (opens in a new tab), vendor, 2026). The role is declared once in a repository DAEMON.md, with watch conditions, routines, deny rules, schedules, policy, and output expectations; Charlie describes the format as open and portable across providers that support it (Charlie Labs – Daemons work 24/7 (opens in a new tab), vendor, 2026).

Its execution model is a useful checklist for any background maintenance loop: intake, context gathering, scoped planning and boundaries, execution or delegation, verification, reporting, and continuation from saved state. The model places human decisions at explicit approval gates for high-impact or irreversible changes, while keeping routine work automatic and its artifacts durable at each checkpoint (Charlie Labs – How Charlie Daemons Work (opens in a new tab), vendor, 2026). The reliability claims are equally concrete: duplicate events reconcile to one outcome, retries are bounded with backoff, interruptions resume from the last saved stage, and incomplete context or failed checks trigger an explicit escalation rather than silent completion (Charlie Labs – How Charlie Daemons Work (opens in a new tab), vendor, 2026).

Harness anti-patterns

Four failure modes are specific to building a single harness, distinct from the factory-scale failures in Dark Factory Patterns:

• Making the agent do a script’s job: letting the agent decide things a deterministic tool should settle. Formatting is the clearest case; a formatter on save fixes quote style for free every time, where asking the agent burns tokens reasoning about it and still errs. Push everything mechanical down to a linter, formatter, or hook, and reserve the agent’s reasoning for the logic and data integrity no script can settle.
• The overly restrictive harness: cranking every linter to maximum so the agent fights the config instead of the problem, burning its calls on alphabetical imports and line lengths and failing to merge because it used double quotes where the config wanted single. Delegate stylistic formatting to an automated tool, and reserve strict constraints (like mypy) for logic and data integrity.
• The neglected context: a harness that never addresses context decay, whose architecture map still says authentication is a local database six months after the move to OAuth, so the agent reads the stale map and hallucinates against a database that is gone. Treat context maintenance as part of the definition of done, so a change that moves a boundary updates the map in the same change.
• Relying on review instead of building loops: falling back on reading every diff for things a test could catch, which keeps you in the loop forever. Every recurring class of bug caught by eye is a loop not yet built; when you correct the same mistake twice, add the check rather than reviewing harder, and assert public behavior rather than implementation detail so a refactor does not break the test for the wrong reason.

Resources

Context maintenance

• fighting slop with slop (opens in a new tab) - Conference talk on keeping an architecture file to months-stable invariants and gating edits to it on a conversation.