Skip to content

Knowledge base

The platform layer

The platform layer is the shared part of a federated context platform: an LLM gateway that governs model access, cost and guardrails, and a context catalog that makes every context product discoverable. Agents ask the catalog what exists and the gateway for the content, so they never depend on where it is stored.

Written by

Updated:

Agentic Engineering 5

Our position. We hold that an agent is a new kind of platform user (opens in a new tab): if delivery paths only work with a senior engineer in the loop, the agent inherits every bottleneck and runs into it faster. The platform has to become self-service and owned as a product (opens in a new tab), with approvals turned into rules.

These central services own none of the context. Domain teams own their context products.

Which shared services make federated context work?

The Central Nervous System: The LLM Gateway as Orchestrator

The LLM Gateway is the single, mandatory entry point for all interactions between AI agents and the underlying LLMs. Its primary function is to decouple agents from the complexities of the LLM landscape while enforcing platform-wide governance and providing critical operational insights. Tools like LiteLLM offer a robust foundation for building such a gateway, providing many necessary features out-of-the-box (LiteLLM (opens in a new tab)). However, it is important to recognize that while such a tool is a powerful enabler, it does not constitute the entire platform; it must be integrated with other custom-built services like the Context Catalog.

Core Features

A production-grade LLM Gateway must provide the following capabilities:

  • Unified API: It offers a standardized, OpenAI-compatible API to developers and agents, abstracting away the heterogeneous APIs of more than 140 LLM providers, by LiteLLM’s own count in October 2026. This simplifies development and allows the platform team to swap or add new models without impacting downstream applications (LiteLLM (opens in a new tab)).
  • Observability & Cost Management: The gateway is the ideal location to implement comprehensive logging. Every request and response is captured, providing detailed metrics on latency, token counts, and error rates. This data is essential for performance monitoring and enables accurate cost tracking, allowing for spend to be attributed and charged back to specific users, teams, or projects (LiteLLM (opens in a new tab)).
  • Security Guardrails: As the central chokepoint, the gateway enforces non-negotiable security policies before any data is sent to an external LLM. This includes capabilities for detecting and redacting secrets, scanning for PII, and applying content moderation filters. These guardrails are a critical implementation of the platform’s guardrails (LiteLLM (opens in a new tab)).
  • Intelligent Routing & Load Balancing: The gateway can dynamically route requests to the most appropriate model based on a set of rules. This could involve routing complex reasoning tasks to a powerful model like GPT-4o, while sending simpler summarization tasks to a more cost-effective model. This strategy optimizes both performance and cost across the platform (LiteLLM - Getting Started (opens in a new tab)).
  • Authentication & Authorization: The gateway manages a system of virtual API keys for users, agents, and teams. These keys are used to enforce access controls, rate limits (e.g., requests per minute), and hard budget caps, preventing runaway costs and system abuse (LiteLLM (opens in a new tab)).

The Marketplace: Designing the Context Catalog

If the LLM Gateway is the nervous system, the Context Catalog is the brain’s memory map. It is the central registry where all “Context as a Product” offerings are published, documented, and made discoverable. It does not store the context data itself but rather the metadata that describes it and points to its authoritative source (Data.world - About the AI Context Engine - Documentation (opens in a new tab)). This “catalog of catalogs” is the linchpin of the federated model, transforming a collection of disparate data sources into a searchable, understandable marketplace of AI-ready assets (DEV Community - Catalogs as Context: How Metadata is Powering the Next Wave of AI (opens in a new tab)).

This catalog and the gateway work in tandem to create a powerful abstraction layer. Agents are decoupled from the underlying context sources; they no longer need to know if context is stored in a vector database, a graph, or a simple file. An agent’s workflow becomes: 1) query the catalog to discover the contextProductID for the needed information, and 2) request that context from the gateway, which then uses the catalog’s metadata to retrieve it from the correct source. This decoupling allows context producers to evolve their storage and retrieval implementations without breaking the agents that consume their products, a fundamental principle of scalable, distributed architecture.

Proposed Metadata Schema

To be effective, each entry in the Context Catalog must adhere to a rich, standardized metadata schema. The following schema provides a comprehensive starting point:

  • contextProductID: A unique, immutable identifier for the product (e.g., urn:context:engineering:code-style-guide).
  • productName: A human-readable name (e.g., “Frontend Code Style Guide”).
  • description: A clear, concise summary of the context provided by the product.
  • ownerTeam: The name of the team responsible for maintaining the product’s quality and availability.
  • sourceSystem: The type of the authoritative source system (e.g., Confluence, Git, Neo4j, Elasticsearch).
  • accessAPI: The endpoint, query template, or method required to retrieve the context data (e.g., https://confluence.mycorp.com/rest/api/content/{pageId} (opens in a new tab)).
  • dataFormat: The format of the returned data (e.g., text/plain, application/json, text/cypher).
  • version: The semantic version of the context product (e.g., 2.1.0), allowing for non-breaking changes.
  • freshness: Metadata describing the data’s timeliness, such as update frequency (e.g., daily) or a time-to-live (TTL) in seconds.
  • accessControlTags: A list of key-value pairs used for governance and security filtering (e.g., {“role”: “engineering”, “confidentiality”: “internal”}).

API Design

The catalog must expose a simple, robust API to enable programmatic interaction by both human developers and AI agents. The core API should support three fundamental operations:

  • POST /products: To register a new Context Product or update an existing one. This would be used by CI/CD pipelines of the context owners.
  • GET /products/search?q={query}: To discover Context Products. The search should query across all metadata fields, allowing users to find context with queries like “API specs for payments” or “owned by finance team.”
  • GET /products/{contextProductID}: To retrieve the detailed metadata for a specific, known Context Product.

Standardizing Access with the Model Context Protocol (MCP)

To achieve true interoperability and prevent the proliferation of bespoke APIs, the platform should adopt a standardized communication layer for context access. The Model Context Protocol (MCP) is an open protocol for connecting AI applications to external tools, data, and systems with a consistent interface (Model Context Protocol (opens in a new tab)).

Instead of each Context Product exposing a unique accessAPI, it would be exposed via a standardized MCP server (HPE - Part 3: Model Context Protocol (MCP): The protocol that powers AI agents (opens in a new tab)). The accessAPI field in the catalog would then point to an MCP tool call. For example, a documentation search tool would be registered as ref_search_documentation, and the agent would invoke it through a standard MCP client (ref-tools - Ref MCP (opens in a new tab)). This approach allows a “Domain Context Service” to act as an MCP gateway, routing requests to the appropriate backend service without the agent needing to know the underlying implementation details (HPE - Part 3: Model Context Protocol (MCP): The protocol that powers AI agents (opens in a new tab)). This makes the entire context ecosystem plug-and-play; new tools and context sources can be added and consumed by any MCP-compliant agent without requiring custom integration code (Sider - Is the Model Context Protocol the Missing Link for AI Agents? A No-Fluff Review (opens in a new tab)).

Resources

Articles

Documentation

Posts

Tools and repositories