Skip to content

Knowledge base

Context as a versioned artifact

For teams with dozens of skills and rules: giving each one an owner, testing context with tiered evals, and the supply-chain risks of sharing it.

Written by

Updated:

Agentic Engineering 5

Our position. We give every rule and every skill one owner (opens in a new tab), the role whose work it describes. Anyone can use it, and a change goes to that owner and in as a commit through the same checks as code. The file was never the hard part, the owner was.

Each stage borrows a practice from engineering. Tests become tiered evals with an error budget, and shared skills bring supply-chain risks such as leaked credentials and prompt injection.

What changes when context is treated like code?

Once teams manage many pieces of context instead of hand-tuning one file, the artifact starts to behave like code and the software lifecycle reappears around it: context is generated, tested, distributed, observed, and adapted, each stage borrowing a practice from engineering (Context is the New Code - AI Engineer talk (opens in a new tab)). Even project setup becomes a living document that executes, run by a harness setup hook before the session starts so environment prep never spends the window (Claude Code setup hooks and just (opens in a new tab)). Owning context that way, as first-class code rather than something outsourced to a framework’s black box, is the baseline (AOE - Context Engineering (opens in a new tab)).

Testing context means evals, tiered like a test pyramid: a cheap linter checks mechanical properties such as whether a skill’s description is present and within length, a costlier readability check asks a model whether the written context is even understandable, and behavioral evals run the agent against known cases (Context is the New Code - AI Engineer talk (opens in a new tab)). Because those runs are nondeterministic, the unit is an error budget rather than pass or fail, run the case several times and require a success rate. A hard limit sits underneath: the field still lacks a shared way to measure whether a context change actually helped, and agent variability can swamp the effect, the same agent on identical context can produce very different results, which fits the broader finding that AI is producing more code but not more production-ready code (Faros - Context Engineering for Developers (opens in a new tab)).

One loop closes the gap by feeding evaluation back into the rules: an LLM-as-judge writes an English explanation of why a run failed, and a meta-prompt turns that explanation into an appended rule, reported to lift a coding agent’s resolved-issue rate by 5 to 15 points (talk on eval-driven prompt learning (opens in a new tab)).

Distribution imports supply-chain concerns. Reusable skills and rules get versioned and shared through registries, which brings dependency management, scanning for leaked credentials and prompt-injection payloads, and an AI SBOM recording who and which model authored a skill, the more necessary because most shared skills are low quality and, once loaded, nothing blocks a malicious one (Context is the New Code - AI Engineer talk (opens in a new tab)). Commercial platforms package the same idea for enterprises, capturing scattered team standards into a versioned playbook, distributing it as AI context across repos and agents, and catching violations pre-commit with automatic rewrites (Packmind (opens in a new tab), vendor).

The authoring craft has its own repeatable loop, distinct from the artifact lifecycle. A model’s untuned output gravitates to convergent defaults, the statistically safe choices that dominate its training data and offend no one, the same distributional pull that makes a coding agent emit generic boilerplate; overriding them means identifying the specific defaults you dislike, tracing the root cause (asking the model in a “debug mode” turn why it produced that output surfaces the faulty assumption behind it), then supplying concrete alternatives at the right altitude. Altitude is the operative constraint: step-by-step instructions that over-specify one scenario overfit and leave the agent brittle on longtail cases, while vague guidance fails to shift behavior at all, so the durable form articulates the reasoning behind the wanted behavior rather than enumerating steps.

Add only the sections that measurably change output, because improving one aspect tends to cascade to others and every added section can perturb what already worked, which keeps the token budget from bloating as the artifact grows. Reasoning models sharpen the discipline: a fully packed prompt can degrade them, since they respond best to direct, concise instruction and will overanalyze an over-complex one (Prompting reasoning models and the convergent-defaults method (opens in a new tab)).

One caution runs against reflexive compression. Stanford’s ACE framework argues agent context should read as a comprehensive, evolving playbook rather than a terse summary, because unlike a human an LLM can filter relevance at inference time but only when the detail is present; it splits Generator, Reflector, and Curator roles and merges incremental delta updates so repeated rewriting does not erode detail (Faros - Context Engineering for Developers (opens in a new tab)). That tension with the minimize-tokens instinct resolves the way ordering does, through relevance and structure rather than blanket brevity. And some behaviors resist rules outright: vivid, concrete stakes can make a convention stick where abstract phrasing slips, while a model’s eagerness to agree is stubborn enough that a dedicated rule barely dents it (Current Prompting Practices (opens in a new tab)).

Resources

Articles

Posts

Talks and videos

Tools and repositories

  • Packmind (opens in a new tab) - Commercial platform that captures team standards into a versioned playbook, distributes it as AI context and catches violations pre-commit.