How to Audit Which Content an AI Agent Used to Produce an Answer

You audit which content an AI agent used by keeping the retrieval path itself queryable: Sanity Context links every retrieved passage to a specific document version and timestamps it against the moment the agent ran, rather than leaving you to reconstruct a decision from logs of prompts and completions alone. This article covers what that provenance record must contain, why your content store's shape determines auditability, and how to make retrieval attributable end to end.

The failure mode: an answer with no paper trail

Start with the concrete incident. A retrieval-augmented agent returns a confident, wrong answer, and a customer acts on it. The postmortem needs three facts: which documents were retrieved, which passages the model was actually shown, and what version of each document existed at run time. In most stacks, none of the three survives. The vector database returned IDs and scores that were never logged. The content those IDs point to has since been edited, so even if you had the IDs, the text you would fetch today is not the text the agent saw. The system prompt that told the agent how to weigh sources lives in a deploy config that was overwritten on the next release.

The result is an answer you cannot reproduce and therefore cannot defend. When a regulator, an enterprise customer's security team, or your own legal function asks you to explain a specific output, "the model said so" is not an acceptable answer. The stakes climb with the domain: in financial services, health, or any regulated vertical, an unexplainable automated statement is a liability whether or not it was correct.

The root cause is architectural. Retrieval provenance is treated as ephemeral telemetry rather than a first-class record. Fixing it means deciding, before you ship, that every agent answer produces a durable, queryable trace linking the output to the exact content and instructions that produced it. Everything else in this article follows from taking that requirement seriously, because a system that cannot answer "which content" cannot be governed, only hoped for.

What a complete answer-provenance record contains

A defensible audit record has five parts, and each maps to a question a reviewer will ask. First, the retrieval set: every candidate document and passage the query returned, with its relevance score, not just the ones that made the final context window. Reviewers need to see near-misses to judge whether ranking was sound. Second, the injected context: the exact text placed in the prompt, because chunking, truncation, and reranking mean what was retrieved is rarely identical to what was shown. Third, the content version: an immutable reference to the document as it existed at run time, so a later edit cannot silently rewrite history.

Fourth, the instruction state: the agent's system prompt, tool definitions, and any policy rules that governed how it used the retrieved content, captured at the version that was live. Fifth, the linkage: a single identifier that ties the output back through all four, so an auditor starts from one bad answer and pulls the whole chain without joining across four systems by hand.

Most teams capture parts one and two as application logs and stop there. The hard parts are three and four, because they require your content store and your instruction store to be versioned and time-addressable, not just current. If your CMS only knows the latest state of a document, you have already lost the ability to answer "what did the agent see in March," no matter how good your logging is. Provenance is only as complete as the least version-aware system in the retrieval path.

Why the shape of your content store decides auditability

You cannot bolt provenance onto a store that forgets. The single biggest determinant of whether you can audit an agent answer is whether the content it retrieved lives in a system that preserves version history and lets you query content and its metadata together. This is where generic vector databases create a structural problem. A vector database stores embeddings and an ID; the source text and its edit history live somewhere else, usually a CMS or a pile of files. To reconstruct a March answer you have to join the vector store's logs against a content system that may have overwritten the March text. That join is fragile at best and impossible at worst.

Sanity's Content Lake is the backbone of the Sanity Context retrieval path, and it treats content as queryable, versioned, structured data rather than opaque blobs behind an ID. Because dataset embeddings are tied to the content itself, an edit propagates to the retrievable representation within minutes, and, crucially, the content and its history live in the same queryable system as the retrieval that surfaced it. You are not stitching provenance across a vector database, a CMS, and a prompt file. The retrieval and the source of truth are one system.

This is the difference between a headless CMS that stops at publishing and a Content Operating System that operates content end to end. Legacy stacks create silos, one for vectors, one for text, one for config, and every silo boundary is a place your audit trail can break. A shared foundation means the answer to "which content" is a query, not a forensic reconstruction across vendors.

Making retrieval itself attributable

Attribution starts at the query. If you cannot see how a passage was selected, you cannot explain why the agent used it. Hybrid retrieval, which most serious RAG systems now need, makes this harder in a fragmented stack because keyword and semantic matching often run in different engines whose scores are reconciled by glue code you wrote. Auditing means reverse-engineering that glue.

In the Content Lake, hybrid retrieval is native and expressed in a single GROQ query: `text::semanticSimilarity()` for semantic matching and a BM25 `match()` for lexical matching, blended with `score()` and `boost()` so the ranking logic is one readable, inspectable expression rather than an opaque pipeline. Because the query is the ranking, the ranking is auditable by construction. You can re-run the same GROQ against the same content version and see exactly which documents scored where and why one passage beat another.

Agent Actions provide the schema-aware APIs for the generate, transform, and translate steps, which means the operations an agent performs are shaped by your content model rather than by free-form prompt improvisation. That schema-awareness matters for audit because a typed operation is inspectable in a way that a raw completion is not. When the retrieval, the ranking, and the actions are all expressed against structured content in one system, the provenance record stops being something you assemble after the fact and becomes a natural output of how the query and action already ran. The alternative, reconstructing a blended score from two engines and a config, is exactly the kind of after-the-fact archaeology that fails under scrutiny.

Governing the instructions, not just the content

Content provenance is only half the record. Two agents can retrieve identical passages and answer differently because their instructions differ. An audit that names the documents but not the governing prompt still cannot explain the output. Yet in most teams the system prompt is the least governed artifact in the entire stack: it lives in an environment variable, a deploy config, or a notebook cell, edited without review and overwritten without history. When someone asks what rules the agent was following in March, there is no answer.

The fix is to treat agent instructions as content and govern them where you govern the rest of your content. In Sanity Studio, editors and engineers manage agent instructions as structured, versioned documents, and Content Releases let you stage changes to agent behaviour the same way you stage changes to a website: bundle the change, preview it, ship it deliberately, and keep the version that was live. Roles and Permissions decide who can alter agent instructions at all, so a governance policy is enforced rather than merely documented.

The governance payoff is that the instruction state at run time is recoverable, not guessed. When you pair a versioned instruction record with a versioned content record and an attributable query, you have all five parts of the provenance chain in one system. This is the pillar of automating everything without surrendering control: the agent runs autonomously, but every run leaves a reviewable trail, and every behaviour change goes through the same editorial gate a human publish would. Ungoverned instructions are how agents drift silently; staged, versioned instructions are how you keep the loop human-supervised.

Connecting production agents to an auditable retrieval path

The audit story only holds if it survives contact with production, which means the path your live agents actually call has to be the same one you can query for provenance. A common anti-pattern is a clean audited pipeline for demos and a shortcut path in production, so the traces you can inspect are not the traces that matter.

Production agents connect to Sanity Context through the Sanity Context MCP endpoint, so the retrieval they perform in production is the same GROQ-backed, content-versioned path described above, not a parallel implementation. Knowledge Bases turn datasets, websites, PDFs, and support databases into agent-readable documents that share that single retrieval path, so a PDF ingested last week and a product entry edited this morning are both attributable through the same mechanism. There is no second, unaudited lane for the unstructured sources that usually escape governance.

That convergence is what makes the provenance record trustworthy. Because the MCP endpoint, the retrieval query, the content versions, and the instruction state all resolve to one system, an auditor starting from a single bad answer can pull the retrieval set, the shown passages, the exact content versions, and the governing instructions without joining across vendors or hoping the logs line up. Sanity Context is the intelligent backend for companies building AI content operations at scale precisely because auditability is a property of the architecture rather than a feature bolted on afterward. When the question is "which content did the agent use," the answer is a query you can run, re-run, and defend, not a story you assemble from fragments after the incident is already public.