Agent Governance & Evaluation9 min readยท

Managing Many AI Agents Across a Company: Shared Context, Ownership and Governance

Your support agent tells a customer returns are accepted within 30 days, the shopping agent quotes 45, and nobody can say which prompt or source is wrong.

Your support agent tells a customer returns are accepted within 30 days, the shopping agent quotes 45, and nobody can say which prompt or source is wrong. The fix is to centralize five layers (identity and access, shared knowledge, instructions, MCP tools, and evaluation) in one governed system and give each layer a named owner. In Sanity, an agent's system prompt is a Content Lake document whose fields map to owners, and changes ship through Content Releases behind an eval gate.

The stakes grow with every agent you add. Each new team copies a prompt string into its own repo, points a vendor crawler at the website, and connects its own tools. Within a quarter you have five versions of the brand voice, three copies of the refund policy, and no record of who changed what or why an agent acted the way it did.

This guide treats agent sprawl as a problem with how content is run, not with the model. It covers which layers to centralize, an operating model for ownership and release, why one governed context layer beats separate context for each agent, and a checklist for auditing the agents you already run.

What goes wrong when a company runs many AI agents?

Running many AI agents tends to go wrong in four ways: instructions get duplicated, knowledge sources drift apart, ownership is unclear, and there is no audit trail when something breaks. Each failure is small inside one agent. Across twenty agents, they add up.

Duplicated instructions start innocently. In most teams the system prompt is a string in the codebase. The support agent has one, the onboarding agent copies it and changes the tone, and a sales assistant copies that copy. Marketing can't read any of them, and compliance can't review them. When the brand voice changes for a campaign, the prompts fall out of date and nobody notices for a sprint. When an agent says something embarrassing in production, the fix is a pull request against whichever copy happened to be live.

Knowledge drift is quieter. The same fact appears in several places and the copies diverge: a help center says returns are accepted within 30 days, while a product page says 45. One agent retrieves from the help center and another from a vendor's crawl of the website, so customers get different answers depending on which agent they talk to. Agent platforms that bundle their own retrieval make this worse, because each one holds a separate index of your content.

Unclear ownership comes next. When an agent misbehaves, the cause could be the prompt, the retrieved document, a tool description, or the model. If nobody owns each of those pieces, the incident bounces between engineering, support, and legal until it goes stale.

The missing audit trail is the failure that reaches executives. If an agent issued a refund or disclosed account details, someone will ask whose identity it acted under and which version of its instructions was live. If agents call backends with a shared service account and prompts change without a release record, nobody can answer.

A better model won't fix any of this. These are governance failures in the content and configuration agents depend on, and they have to be fixed there.

Which layers should you centralize when you run many AI agents?

Five layers should be centralized across every AI agent a company runs: identity and access, shared context or knowledge, instructions and system prompts, tools exposed over MCP, and evaluation and observability. Models and harnesses can vary by team. These five should not.

Identity and access decides whose permissions an agent uses. The strongest pattern is auth-forwarded tools, where the user's session token passes through the agent runtime and tool layer to the backend API. This gives you personalized retrieval (the agent sees what the user can see), personalized action, and a traceable audit, because actions are logged against the user rather than the model. The agent inherits your existing row-level permissions, rate limits, and regulatory boundaries instead of needing a new set.

Shared context or knowledge is the content agents base their answers on: product facts, policies, documentation, and support material. Centralizing it means every agent queries one structured source instead of each vendor keeping its own index.

Instructions and system prompts control how agents behave with customers. They need versioning, attribution, scheduled release, rollback, and role-based editing, which are the same controls a company already applies to its website.

Tools over MCP need central review because tool descriptions are trusted text. Every Model Context Protocol server you connect adds descriptions to the prompt on every turn, so a central registry of approved servers is cheap insurance.

Evaluation and observability close the loop. Tracing standards such as OpenTelemetry and scoring platforms such as Braintrust give you the signal. A shared eval bench of frozen conversations with rubrics gives you the verdict. With a central bench, a change to a shared prompt section is tested against every agent that uses it.

A useful rule for deciding what to centralize: if two agents could give a customer contradictory answers because a layer differs between them, that layer belongs in the shared center.

Illustration for Managing Many AI Agents Across a Company: Shared Context, Ownership and Governance
Illustration for Managing Many AI Agents Across a Company: Shared Context, Ownership and Governance
โ„น๏ธ

Every MCP server adds trusted text to your prompt

An MCP server's tool descriptions are added to an agent's context window on every turn, before the user types anything. A sloppy or malicious description can prompt-inject the agent. When dozens of agents each connect their own servers, nobody knows what trusted text is running. Approve and review MCP servers centrally, the same way you review third-party code.

Who should own each AI agent and its context?

Every AI agent needs one accountable product owner, and every layer of its context needs its own named owner, because several parties write a production agent's prompt at once. When ownership covers only "the agent" as a single thing, incidents end up with no owner.

A production agent's context window is a stack of six layers: the model provider's system prompt, MCP server tool descriptions, your application system prompt, per-turn runtime context, an optional layer that customers can customize, and prompts inside tools. Different parties write these layers, and each can be edited to a different degree. You can't edit the provider layer, platform engineering owns the tool layer, and the application prompt is shared across business functions. Each owner has to know what the others are doing.

The application prompt is where most companies get ownership wrong, so split it. A practical division: Brand owns voice, Product owns how the agent uses user context, Support owns escalation language, and Compliance owns the list of things the agent must never say. Engineering owns the runtime that combines those fields into one final system prompt, plus the harness, the tools, and the eval bench.

For shared knowledge, ownership follows the source. The team that owns the returns policy owns the returns policy document, and every agent that cites it picks up their edits. When two sources disagree, someone has to decide which one is ground truth, and that decision needs an owner too.

Write down an ownership map for each agent: agent name, accountable owner, harness type, knowledge sources and their owners, prompt fields and their owners, approved MCP servers, and eval bench. If a cell says "engineering" only because nobody else could edit that layer, that's a limit of your tooling, not a real ownership decision.

Vipps came to Sanity wanting the whole organization to help write prompts, with product managers owning them rather than only engineers. That only works when the prompt lives somewhere non-engineers can safely edit it.

โœจ

Editors tuned the agent without code changes

Nearform kept its agent's system prompt in a Sanity document: "Storing the system prompt in a Sanity document is genuinely useful. Editors tuned the agent's voice without any code changes." Ownership moved to the people responsible for the voice, while the runtime stayed with engineering.

How should changes to agent instructions be reviewed and released?

Agent instructions should be authored like content and gated like code. The owning team edits them, the change is staged in a release and tested against an eval bench in CI, and it ships only when the bench stays green. You need both halves, and they work best in the same system.

Keeping prompts in code gets the gate right. Strings in a repository have version history and tests, and every change goes through review. It gets authorship wrong: only engineers can edit, and every wording change waits for a code deploy. Letting business teams edit a live prompt in an admin panel fails the other way: authorship is right, but there is no gate.

A working release process for many agents has four steps. First, the owning team drafts the change in its own field of the instruction document. Second, the change goes into a release with any related content, so a new returns policy and the agent's matching escalation wording ship together. Third, CI runs the eval bench (a set of frozen conversations with rubrics) against the staged version. Fourth, a reviewer approves, the release publishes on schedule, and if production scores drop, rolling back takes one action.

Evaluation also shows where a regression came from. Tag each production failure by type, and each tag points to a layer. Hallucination usually means retrieval returned nothing useful and the model filled the gap. Tool-misuse is a tools problem. Scope-violation means the never-say list missed something. Auth-confusion means the agent acted under the wrong identity. Prompt-drift means a prompt change shipped without the bench catching the regression. Fix the problem at that layer, not in the model.

Keep feeding the bench from production. Sample conversations, classify them, and add surprising failures and edge cases to the bench so the next change has to pass them too. Over time, the bench becomes the company's shared definition of correct behavior for every agent it runs.

Why does one governed context layer beat separate context per agent?

One governed context layer beats separate context for each agent because a fact or instruction you update once reaches every agent at the same time, with one history, one permission model, and one audit trail. With separate context, every edit has to be repeated for each agent, and every copy can drift.

Sanity is the AI Content Operating System, the intelligent backend for companies building AI content operations at scale. Instead of the silos that legacy tools create, it gives teams, systems, and agents one shared structured content layer. Sanity Context is the part agents use: structured, governed access to your content through an MCP, a knowledge base, and an ingest path.

In practice, an agent's application prompt becomes a Content Lake document defined by a Schema such as agentPrompt. It has fields for role, voice (owned by Brand), userContext (owned by Product), escalation (owned by Support), and mustNotSay (owned by Compliance). The split into fields works as access control. Because the prompt is content in the Studio, it comes with real-time collaboration, version history, scheduled publishing, and rollback. Content Releases stage agent behavior the same way they stage the website, so one release can ship a homepage change and a prompt change together.

Agents connect through Context MCP, a hosted read-only endpoint that any agent loop can use, with the initial_context, groq_query, and schema_explorer tools. Knowledge Bases (currently in beta) build an index over sources you choose, including sources from more than one project. They flag conflicts such as the 30-day versus 45-day returns claim.

Each agent still has its own harness. A customer-facing harness holds short context, acts under the user's identity with auth-forwarded tools, and injects state on each turn. A coding harness is sandboxed and runs long tasks. Don't bolt a coding harness onto a customer-facing agent. Whatever the harness, the agents draw on the same governed context.

Instructions that retire themselves

In Knowledge Bases (beta), each standing instruction, such as which source to trust when two claims differ, is tied to the sources it depends on. When those sources change, a build can retire the instruction automatically, so stale rules don't quietly outlive the facts they were written for.

How do you audit an existing set of AI agents?

To audit an existing set of AI agents, start with an inventory and finish with a list of gaps for each layer. The checklist below works for five agents or fifty. Most teams can do a first pass in one working session with a person from each owning function: engineering, support, brand, product, and compliance.

1. Inventory every agent in production or pilot, including vendor agents. Record its accountable owner, harness type, and model provider.
2. For each agent, find where its system prompt lives. Count how many copies of shared sections (voice, escalation, and forbidden topics) exist across agents.
3. List every knowledge source each agent retrieves from, and note whether it is a direct query, a vendor crawl, or an uploaded copy. Flag any fact that appears in more than one source.
4. Check identity. Does the agent act under the user's identity with forwarded auth, or under a shared service account? Can you tell from the logs whose behalf an action was taken on?
5. List every MCP server and tool each agent connects to, who approved it, and when its tool descriptions were last reviewed.
6. Confirm an eval bench exists, that it gates prompt and tool changes, and that production failures are tagged by type.
7. Pick a recent prompt change and work out who made it, who reviewed it, when it shipped, and how you would roll it back. If you can't, that is an audit finding.

Score each agent on these seven questions and a pattern usually appears. Most gaps are in instructions and knowledge, because those layers lived in code or in vendor indexes where business owners couldn't see them. Move those two layers into one governed context layer first and assign ownership field by field. Then tighten identity, MCP review, and the eval gate. Run the audit again each quarter, since new agents keep appearing between audits.

Governing agent context across a company: Sanity Context vs common approaches

FeatureSanityPrompts in code (repo + CI)LangSmithDecagon / Sierra AI
Where agent instructions liveStudio documents defined by a Schema such as agentPrompt, structured in the Content Lake and combined into one system prompt at runtime for every agentStrings in the repository, versioned with git and reviewed through pull requests. Only engineers can edit them, and changes ship with codePrompt versioning and tracing for engineering teams, kept in a system separate from the content agents retrieve, so instructions and knowledge are governed in two placesSet up inside each vendor's platform and scoped to agents running there, not shared with agents your teams build elsewhere
Field-level ownership by business teamsBrand owns voice, Product owns userContext, Support owns escalation, and Compliance owns mustNotSay. None of them needs a pull request or a deployEngineering owns prompts by default. Business teams request changes through tickets or pull requests and wait for a deployBuilt mainly for engineering and AI platform teams. It is not designed around business functions owning individual prompt sectionsAdmin consoles let operations teams adjust agent behavior, but this is governed separately from your website content and your other agents
Shared knowledge sourceOne Content Lake queried through Context MCP. Knowledge Bases (beta) index sources across projects and flag conflicting claims for reviewDepends on what engineers build. Each agent often ends up with its own retrieval pipeline and its own indexNot a knowledge store. Retrieval content lives in whichever vector store or backend each agent team connectsEach platform loads or crawls your content into its own index, so every vendor holds a separate copy that can drift
Staging and release of agent behaviorContent Releases stage prompt and content changes together, with scheduling, preview, history, and rollback in the StudioShips with code deploys. Git gives history and revert, but prompt wording is tied to the engineering release scheduleEngineers pull versioned prompts into apps, and they are released separately from the content agents citeChanges are published within each vendor's platform on its own process, separate from your content releases and other agents
Identity and audit trailAuth-forwarded tools log actions against the user, not the model. The Studio keeps drafts, history, permission gating, and audit trails for instructionsGit history shows who changed a prompt. Runtime identity depends on how each team builds its own tool layerDetailed tracing of runs and calls for debugging. User identity and permissions come from your own application layerVendor-managed integrations act within each platform's permission model, which you configure separately for each vendor
Evaluation gate on changesPrompt changes run the eval bench in CI before release, and scores can be stored as conversationScore documents next to the source contentCI tests can gate prompt changes well, but each team has to build and maintain its own eval benchEvaluation and tracing tooling for engineers. It works alongside a governed context layer rather than replacing oneEach platform has its own quality and analytics views, covering that vendor's agents rather than all of your agents