Skip to content

Blog

An automation that audits the factory must not rewrite its own record

The automation auditor in the software-factory.dev factory inspects every other automation on the line. Its most useful rule: it may never overwrite last week's audit.

Written by

Software Factory

Published: 18 August 2026 Updated: 2 min read

There is one automation in the software factory built in public at software-factory.dev whose entire job is to inspect the other automations. This is the part I keep pointing people to.

It is called the Automation Auditor. Most factory parts build, fix or ship something. This one reviews the line itself.

What it reads every Monday

The trigger is a single weekly cron: 0 8 * * 1. Monday, 8am, before the week’s work starts piling up.

When it runs, it pulls a week of activity:

  • pull requests opened and merged in the last seven days
  • issues that automations filed
  • CI fix commits, found by searching commit messages
  • bugs raised by the verifiers and the incident responder

How it grades the line

For each automation it asks four plain questions. Did it finish its job? Did it fail repeatedly on the same pull request? Did it do unnecessary work, like reviewing an already-reviewed change? Are there patterns the feature builder or bug fixer keeps getting wrong?

It checks label hygiene too. Issues stuck in progress for more than 24 hours get flagged as stalled. So do escalations that have sat unanswered for over a week.

It can also edit the convention files the other automations read, though only on a clear, repeated problem.

The write rule worth copying

The one detail worth copying is smaller than any of that: the write rule for the audit file itself. If an audit already exists for the current week, the auditor must not overwrite it. It saves the new one under a dated name instead. Two audits for one week sit side by side, and both stay readable.

That sounds like housekeeping. It is the practice that makes the rest work. A factory that improves itself needs a part that watches the whole, and that part needs a memory it cannot quietly rewrite. An audit that overwrites the last audit is just the latest opinion.

You can read the full definition in the Automation Auditor configuration on GitHub (opens in a new tab).

When something inspects your own automations, can it edit the rules they run on, and can you still read every past verdict it gave?

Further reading

Written by

Factory skills, part 14 of 18

  1. Escalated work needs a return path back onto the line The smallest automation in the software-factory.dev factory puts escalated issues back on the line once a human has answered. The return path from human to machine needs its own owner. 2 min read
  2. A metric without a red line is decoration The weekly Performance Monitor in the software-factory.dev factory works because every metric it watches has a hard threshold written into it. 2 min read

All 18 parts

ClientsClient work

Scaling needs named owners

Coding agents scale across teams when named people own the platform, the team practice, the guardrails, and the measurement. Rollout & scale is where that ownership is built and handed to internal owners, in your stack.

Training on this topic: Agentic Software Factory: Advanced Team Training
See Rollout & scale

Want to check one team first? The self-check: twelve yes or no questions, three minutes, instant result, no account. Take the self-check