Strategy and architecture

Our position. We hold that context should be federated in layers that mirror the org chart (opens in a new tab). One file means one owner for knowledge that has many owners, and it goes stale the day it is written.

What does a federated context platform consist of?

The architecture distinguishes four layers:

• Platform context supplies shared policies, capabilities, and infrastructure.
• Domain context captures business concepts, rules, and decisions.
• Technical context makes code, architecture, and runtime behavior legible.
• Agent context carries the task state and operational memory of an individual agent.

Three platform components connect those layers: an LLM gateway governs model access and policy, a context catalog makes context products discoverable, and interoperable interfaces such as MCP let agents retrieve them. Storage remains polyglot because each retrieval problem has a different shape:

• Enterprise search supports relevant document retrieval.
• Knowledge graphs support relationship-heavy and multi-hop questions.
• Memory systems retain evolving interaction history when persistence is useful.
• Version-controlled files provide auditable context for engineering and long-running work.

The design rests on five pillars:

1. Treat context as a product with an owner, consumers, quality expectations, and a lifecycle.
2. Separate platform, domain, technical, and agent context instead of building one undifferentiated store.
3. Choose search, graphs, memory, and files according to the retrieval problem rather than forcing one database onto every use case.
4. Federate discovery and policy through shared gateway and catalog services.
5. Assemble only the context a task needs through deliberate context engineering.

Governance is part of the architecture. Access control, freshness, provenance, and retirement determine whether the context fabric remains safe and useful after its first release.

Why does centralized context fail at enterprise scale?

Context quality limits what an agent can do, and enterprise context is spread across dozens of systems. The fix is organizational before it is technical: it starts with named owners for context, then a “Context as a Product” paradigm and a layered architecture.

The Context Challenge in Enterprise AI

The effectiveness of Large Language Models (LLMs) and the AI agents built upon them is directly proportional to the quality and relevance of the context they are provided. In a complex enterprise environment, this context is vast, distributed, and dynamic. The failure to manage it effectively is the primary source of common AI shortcomings.

Problem Statement

Current approaches to providing context to AI agents often treat it as a monolithic data ingestion problem, leading to systems that are brittle and difficult to scale. LLMs, limited by their training data cutoff dates and finite context windows, frequently “hallucinate” or provide syntactically valid but logically incorrect responses when operating without sufficient external knowledge (Enhancing LLM Code Generation with RAG and AST-Based Chunking (opens in a new tab)). This problem is exacerbated in enterprise settings where context is fragmented across dozens of systems: source code in Git repositories, documentation in Confluence, requirements in Jira, conversations in team chats, and business logic embedded in legacy applications.

Scaling Limitations

As an organization scales its AI initiatives—deploying more agents across more teams and connecting to more data sources—a centralized context architecture becomes an untenable bottleneck. A single team responsible for managing all context sources cannot keep pace with the rate of change across the enterprise. This centralized model mirrors the challenges of monolithic data warehouses, which struggle with agility and domain-specific expertise. A federated architecture, which allows for decentralized management and collaboration between independent components, is crucial for achieving scalability and fault tolerance (GeeksforGeeks - Federated Architecture - System Design (opens in a new tab)). Without such a model, the context platform itself becomes the single point of failure, hindering innovation rather than enabling it.

The Cost of Poor Context

The consequences of an inadequate context strategy are not merely technical; they have direct and significant business impacts. A substantial portion of an analyst’s or developer’s time—up to 70% in some cases—is spent on “data wrangling” to prepare data for use, a cost that is directly mirrored in the effort required to prepare context for AI agents (Deloitte - Treating data as a product in the era of GenAI (opens in a new tab)). Inaccurate or outdated responses from AI agents erode user trust, diminishing adoption and undermining the business case for AI investment. Furthermore, a poorly governed context system presents a severe security risk. Without robust controls, agents can inadvertently access and expose sensitive or personally identifiable information (PII), leading to compliance violations and reputational damage (LaunchDarkly - AI application development best practices: From prototype to production (opens in a new tab)).

A New Paradigm: “Context as a Product”

To overcome the challenges of scale, quality, and governance, a fundamental shift in perspective is required. Instead of viewing context as a raw byproduct of business operations, it must be treated as a first-class product. This “Context as a Product” paradigm, drawing heavily from the principles of Data Mesh, provides the organizational and architectural foundation for a scalable federated system (Deloitte - Treating data as a product in the era of GenAI (opens in a new tab)). This shift is primarily an organizational and cultural one; its success hinges on establishing clear ownership and treating internal data consumers, including AI agents, as valued customers. The initial step in this journey is not technological but organizational: the assignment of “Context Product Owners” to key domains.

Core Principles

Under this paradigm, each unit of context is managed and delivered as a product with a defined lifecycle and quality standards. A Context Product must be:

• Discoverable: Registered and documented in a central, searchable Context Catalog.
• Addressable: Accessible through a stable, well-defined interface or API.
• Trustworthy: Accompanied by metadata defining its source, lineage, quality metrics, and ownership, ensuring users and agents can rely on its accuracy.
• Secure: Governed by explicit access control policies.
• Interoperable: Conforming to a standardized metadata schema that allows for easy integration by various agents and systems.
• Inherently Valuable: Designed to be directly useful for a set of target use cases, such as code generation, business analysis, or customer support.

Federated Ownership

A core tenet of this model is decentralized, domain-oriented ownership. The team that creates and understands a data source is best positioned to own the corresponding Context Product. For example, the engineering team owns the “Codebase Dependency Graph,” the product team owns the “Feature Requirements” from Jira, and the legal team owns the “Contract Clause Library.” This federated ownership model mirrors the structure of successful decentralized organizations and distributed computing architectures, clustering responsibility with expertise and ensuring that context quality is maintained at the source (GeeksforGeeks - Federated Architecture - System Design (opens in a new tab)).

Types of Context Products

Context Products can be categorized based on their degree of transformation and readiness for AI consumption, providing a clear value chain from raw data to actionable intelligence:

• Raw Context Products: Data provided in its original format from a source system, with minimal cleansing. An example is a raw JSON dump of an API specification.
• Conformed Context Products: Data that has been cleaned, standardized, and enriched with consistent metadata. An example is a collection of Confluence pages that have been parsed, chunked, and tagged with standardized keywords.
• AI-Ready Context Products: Highly curated and structured data specifically designed for consumption by LLMs. This includes vectorized documents for RAG, structured knowledge graphs for complex queries, or fine-tuning datasets.

Architectural Blueprint: A Multi-Layered Model for Federated Context

The “Context as a Product” paradigm is realized through a multi-layered architecture that organizes context by its nature and purpose. This model establishes a clear separation of concerns and a dependency hierarchy that guides how agents assemble and utilize information. These layers are not merely stacked; they form a dependency graph where an agent’s specific task context relies on the technical context of its environment, which in turn must adhere to broader domain and platform constraints. An effective orchestration engine must resolve this dependency graph to ensure that higher-level constraints, such as security policies, are always respected.

Layer Definitions

The architecture is composed of four distinct layers, each providing a different facet of the total context required by an AI agent.

This layered model provides a structured framework for organizing, owning, and serving context across the enterprise, forming the blueprint for the federated platform.

---

Related

• The platform layer: the gateway and catalog that connect the four layers.
• Engineering context products: how a domain turns its sources into a product.
• Storage and retrieval architectures: which store serves which retrieval problem.
• Governance and operations: how access, freshness and ownership hold after launch.
• The Federated Factory at Enterprise Scale: the same federation idea applied to software factories.

Resources

Articles

• AI application development best practices: From prototype to production (opens in a new tab) - LaunchDarkly blog post on taking AI applications to production, including controls that stop agents exposing sensitive data.
• Enhancing LLM Code Generation with RAG and AST-Based Chunking (opens in a new tab) - Blog post on why models without sufficient external context hallucinate code, and how AST-based retrieval supplies it.
• Federated Architecture - System Design (opens in a new tab) - GeeksforGeeks explainer on federated architecture as decentralized management of independent components, the scaling answer to central bottlenecks.

Papers and reports

• Treating data as a product in the era of GenAI (opens in a new tab) - Deloitte paper on data as a product for generative AI, the data-mesh model behind Context as a Product.

Posts

• Federated in layers that mirror the org chart (opens in a new tab) - LinkedIn post arguing that one context file means one owner for knowledge with many owners, stale the day it is written.