Building a Production RAG Pipeline for Internal Product Docs: A Reference Architecture

Your RAG pipeline demos beautifully, then ships. Three weeks later a support engineer asks the internal assistant which firmware version fixed a known regression, and it confidently cites a doc that was deprecated two releases ago. The embedding index was rebuilt last Tuesday; the doc changed Thursday; nobody rebuilt it again. Now the answer is wrong, the engineer trusts it, and the customer gets bad guidance. This is the default failure mode of production RAG over internal product docs: retrieval drifts out of sync with the content it is supposed to ground, and no one notices until the answer is already out the door.

Sanity Context (previously Agent Context) exists to close that gap. It is the retrieval layer of Sanity, the Content Operating System for the AI era, an intelligent backend that keeps your agents grounded in the same structured content your editors actually maintain. This article is a reference architecture, not a toy. It walks the full path from document modeling through hybrid retrieval, freshness, governance, and evaluation, and it is honest about where a bolt-on vector database or a search engine with an embedding module leaves you holding the integration.

The failure modes that only show up in production

A retrieval prototype hides its weaknesses. You test it against a curated set of questions, the answers land, and everyone signs off. Production is different because internal product docs are a moving target. Release notes ship weekly, API references get versioned, deprecation notices land quietly, and the same concept appears under three names across support macros, onboarding guides, and the changelog. The prototype never saw that entropy.

Three failure modes dominate once real traffic arrives. First, staleness: the answer is retrieved from a chunk that was accurate when it was embedded and wrong by the time it was served. Second, retrieval mismatch: pure vector similarity returns semantically close but factually irrelevant passages, so a question about "rate limits" surfaces a marketing paragraph about "limitless scale." Third, ungoverned context: the agent's system instructions, the retrieval filters, and the allowed document set live in code that only one engineer understands, so nobody can review what the agent is actually allowed to say.

The pattern underneath all three is the same. Retrieval was treated as an infrastructure problem, separate from the content lifecycle, when it is actually a content problem. The moment your index and your source of truth are two different systems synchronized by a cron job, you have signed up for drift. A reference architecture worth building starts by refusing that separation. Retrieval should live where the content lives, so that when a doc changes, the thing your agent reads changes with it, not on the next scheduled rebuild.

Model the docs before you embed them

The temptation with internal docs is to treat everything as a flat blob: scrape the wiki, split into 512-token chunks, embed, done. It works in the demo and fails the first time someone asks a question that depends on structure the blob threw away. "Which version deprecated this endpoint" is unanswerable if version, endpoint, and status were flattened into undifferentiated text.

Modeling your business is the first pillar for a reason. Before retrieval, decide what a document actually is. A product doc is not one thing; it is a reference page with an owner, a product area, a version range, a support tier, and a lifecycle status. When those attributes are fields rather than sentences buried in prose, retrieval can filter on them precisely. You can scope a query to the current major version, exclude deprecated content, or boost docs owned by the team that ships the feature in question.

In Sanity, that structure lives in the Content Lake as queryable, typed documents rather than as opaque files. Because the schema is explicit, GROQ can filter and project against those fields in the same query that does the semantic ranking. This is the difference between retrieval that guesses from text and retrieval that knows a doc's version is 4.2, its status is deprecated, and it therefore should not ground an answer about the current release. Knowledge Bases extend the same model outward, turning datasets, websites, PDFs, and support databases into agent-readable documents that share one retrieval path, so the PDF spec sheet and the Studio-authored reference obey the same filters and the same freshness guarantees.

Hybrid retrieval: why pure vector search is not enough

Dense vector search is genuinely good at one thing: recognizing that "how do I revoke a token" and "invalidate credentials" are the same intent. It is genuinely bad at another: exact-match terms. Ask about error code E4021, a specific SKU, a CLI flag, or a version string, and semantic similarity will happily return a passage that is about the right topic but never mentions the literal token you need. Internal product docs are dense with exactly these identifiers.

The production answer is hybrid retrieval: run lexical and semantic scoring together and blend the results, so exact identifiers are caught by keyword matching while paraphrases are caught by embeddings. Most stacks assemble this by hand. You stand up a vector database, keep a separate keyword index, query both, then write reranking glue to reconcile two ranked lists that do not share a scoring space. Every piece is another service to run, another sync to babysit, and another place for the two indexes to disagree about what content currently exists.

Sanity Context does hybrid retrieval natively inside the Content Lake. In a single GROQ query you blend `text::semanticSimilarity()` for meaning with a BM25-style `match()` for exact terms, combining them with `score()` and `boost()` so version fields, status, and lexical hits all weigh into one ranked result. There is no second index to reconcile and no reranking service to operate, because the semantic score and the keyword score are computed against the same documents in the same store. That is what native means here: hybrid retrieval is a query, not an assembly project.

Freshness: keeping the index honest as docs change

Return to the opening failure: the doc changed Thursday, the index rebuilt Tuesday, the answer went stale. This is the single most expensive gap in most RAG deployments, and it is almost always structural rather than a bug. When embeddings live in a vector database that is separate from your content store, freshness becomes a pipeline you own: detect the change, re-chunk, re-embed, upsert, invalidate cache, and hope nothing failed silently between steps. The larger the doc set, the more that pipeline becomes a part-time job for someone who did not sign up for it.

The stakes are not abstract. A stale retrieval layer means your agent's confidence is inversely correlated with its accuracy exactly when a doc has just been corrected, which is precisely the moment the correction mattered most. Every hour of drift is an hour the assistant is authoritatively wrong about the thing you just fixed.

Because Sanity's dataset embeddings are tied to the content itself rather than maintained as a separate copy, an edit to a document propagates to what retrieval sees within minutes, with no external re-embedding pipeline to build or monitor. The embedding is a property of the content, not a downstream artifact of it. Combined with the Live Content API, this means the path from "editor fixes the doc" to "agent grounds on the fixed doc" is short and does not depend on a scheduled rebuild that someone has to remember to run.

Governance: staging what the agent is allowed to say

An internal assistant that grounds on product docs is, functionally, a person on your support team who has read everything and never sleeps. You would not let a new hire answer customers without reviewing what they were trained on. Yet most RAG deployments give exactly zero editorial review over the agent's instructions, its allowed document set, and the changes to both, because all of that lives in application code behind a pull request that content owners never see.

Governance is where treating retrieval as a content problem pays off again. The people who own the docs are the people best positioned to say which docs should ground the agent, what tone the answers should take, and which content is off-limits. Give them a place to do that and you turn agent behavior into a reviewable, versioned editorial artifact instead of a config file.

In Sanity, agent instructions and the retrievable content set are governed in the Studio, and changes are staged through Content Releases the same way a team stages a website launch: preview the behavior, review it, and ship it as a unit. Agent Actions provide schema-aware APIs for the content workflows the agent drives, generate, transform, and translate, so those operations respect the same model and the same permissions. Roles & Permissions, Audit logs, and Content Source Maps mean you can answer who changed the agent's grounding, when, and which source a given answer came from, which is the difference between an assistant you can put in front of customers and one you can only demo internally.

Serving and evaluation: connecting agents and proving it works

With modeling, retrieval, freshness, and governance in place, the last mile is serving the pipeline to real agents and proving it stays correct. Production agents connect to the Sanity Context MCP endpoint, which exposes the retrieval path in the shape agents expect, so you are not writing a bespoke API client for every framework your platform team adopts. The agent asks a question, the MCP endpoint runs the hybrid GROQ query against current content with the governed filters applied, and grounded passages come back with enough structure to cite the source.

Evaluation is the part teams skip and later regret. A RAG pipeline is not "done" when it answers your ten favorite questions; it is done when you have a standing set of question-and-expected-source pairs that run on every content change and every instruction change, so a regression surfaces before a customer finds it. Because retrieval here is a GROQ query against typed content, your evaluation harness can assert not just "did the answer look right" but "did retrieval return the document with status current and the correct version," which is a far stronger guarantee than fuzzy answer-matching alone.

This is the whole argument for the reference architecture. When retrieval, freshness, governance, and evaluation all operate against one structured store instead of four synchronized systems, the pipeline has fewer seams to drift at. Sanity Context is the intelligent backend for companies building AI content operations at scale precisely because it collapses that stack: the content, the embeddings, the query, and the governance are one system, not an integration you maintain forever.