How to Build an Ask Our Docs Feature Inside a Product

Your team ships an "Ask our docs" box, wires it to an LLM, and within a week support tickets arrive with screenshots of the assistant confidently citing a config flag that was deprecated two releases ago. The answer reads well. It is also wrong. That is the failure mode that kills these features: not that the model can't write, but that it retrieves stale, half-structured, or context-free chunks and dresses them up as authority. Every hallucinated answer erodes the trust the feature was supposed to build.

The fix is almost never a bigger model. It is the retrieval layer underneath it. This is exactly where Sanity Context lives. Sanity Context is the Sanity product for grounding AI agents in your structured content, built on Sanity, the Content Operating System for the AI era. It is the intelligent backend that keeps your assistant answering from the same governed, versioned docs your editors publish, not from a snapshot that drifted out of date the moment you indexed it.

This guide walks the build end to end: how to model your docs so retrieval works, how to blend keyword and semantic search, how to keep embeddings fresh, and how to govern what the agent is allowed to say before it ships.

Why naive doc bots hallucinate against your own content

The typical first attempt looks reasonable on a whiteboard: scrape the docs site, split every page into fixed-size chunks, embed the chunks, drop them in a vector index, and hand the top matches to an LLM. It demos beautifully and fails in production for structural reasons that have nothing to do with the model quality.

The first problem is chunk boundaries. Fixed-size splitting cuts a prerequisite away from the step that depends on it, so the model retrieves the instruction without the warning that qualified it. The second is staleness. Your content team ships a correction on Tuesday, but the vector index still holds Monday's embedding, so the assistant answers from a version of reality that no longer exists. The third is context collapse. A raw HTML scrape strips the metadata that told a human reader whether a passage applied to the free tier or the enterprise plan, and the model has no way to recover that distinction from prose alone.

Each of these is a retrieval defect, not a generation defect. Swapping in a larger model makes the wrong answer more fluent, not more correct. The reframe this guide asks you to make is to treat the assistant as a thin layer over a well-modeled, queryable content store, and to spend your engineering budget on the store rather than on prompt gymnastics. When content is structured, versioned, and queried with intent, the model's job shrinks to summarizing passages it was handed correctly, which is the job it is actually good at. Model your business first, and the rest of the pipeline gets simpler.

Model your docs as structured content, not scraped text

Retrieval quality is decided before a single query runs, at the point where you decide what a document is. Scraping a rendered page throws away everything you knew when you authored it: which product version it targets, whether it is a concept or a task, what audience it serves, and which related pages it links to. You then spend the rest of the project trying to reconstruct that structure with regexes and heuristics.

The alternative is to keep docs as first-class structured content. In Sanity, docs live in the Content Lake, Sanity's queryable content store and the backbone of the Sanity Context retrieval path. A doc is not a blob of HTML; it is a typed document with fields you defined: product area, version applicability, audience, status, and body content authored in the Portable Text Editor. That structure is exactly the metadata your retrieval needs to filter on before it ever ranks by relevance. An assistant answering a free-tier user can constrain retrieval to free-tier content in the same query that ranks by meaning.

This maps to the first pillar, model your business. Because the shape of the content is yours to define, you are not forced into a generic page-and-block schema that flattens the distinctions your support answers depend on. Knowledge Bases extend the same idea beyond the docs you author in the Studio: datasets, websites, PDFs, and support databases become agent-readable documents that share the Sanity Context retrieval path, so a PDF runbook and a hand-authored guide answer through one consistent surface rather than three disconnected indexes.

Blend keyword and semantic search in a single query

Semantic search alone is a common trap. Embeddings are excellent at matching meaning and terrible at exact tokens. A user searching for the precise error code SANITY_ERR_4032 or a specific function name wants a literal match, and a pure vector search will happily return three passages that are conceptually adjacent and none that contain the string. Keyword search has the inverse failure: it nails the exact token and misses the paraphrase.

Production retrieval needs both, blended and scored together, not run as two separate systems you reconcile in application code. In Sanity Context this is native to the Content Lake. You write one GROQ query that combines `text::semanticSimilarity()` for meaning with a BM25 `match()` for lexical hits, then blend the two signals with `score()` and `boost()` so exact matches surface where they should while paraphrases still rank. The error-code query and the how-do-I query run through the same expression, tuned rather than rearchitected.

This is the row that usually collapses into a checkbox in vendor comparisons, and it is where the difference is most concrete. Assembling hybrid retrieval from a standalone vector database plus a separate keyword engine means you own the glue: two systems to keep in sync, two latency budgets, and a ranking layer you hand-roll and re-tune every time the corpus shifts. Native hybrid retrieval inside the content store collapses that into a query you can read, version, and reason about. The pillar here is automate everything: the retrieval logic lives next to the content it queries instead of in a separate service you babysit.

Keep embeddings fresh so answers track the docs

The staleness problem deserves its own treatment because it is the defect that survives the demo and shows up as a support escalation. In a bolted-together stack, embeddings live in a vector database that is a copy of your content frozen at index time. Every edit your content team makes opens a gap between what the docs say and what the assistant retrieves, and closing that gap means building and operating a reindexing pipeline: change detection, queueing, re-embedding, upserts, and dead-letter handling for the jobs that fail. That pipeline is real infrastructure with real on-call weight, and when it lags, your assistant quietly answers from yesterday.

The structural fix is to tie embeddings to the content rather than maintaining them as a separate artifact. With dataset embeddings in Sanity Context, embeddings are attached to the content itself, so when an editor publishes a correction the update propagates within minutes and there is no separate vector pipeline to maintain. The reindex-or-drift dilemma stops being a system you run and becomes a property of the platform.

This is not a minor operational convenience. It is the difference between an assistant your support team trusts and one they route around. Legacy CMSes bolt AI on as an afterthought, so freshness becomes your problem to engineer; a Content Operating System built for AI treats the currency of embeddings as part of the content lifecycle. When a customer reads an answer, it reflects the doc as published minutes ago, not the doc as it existed the last time a batch job completed. That is the property that lets you put the feature in front of paying users without a disclaimer.

Govern what the agent can say before you ship it

An assistant that answers from correct content can still say the wrong thing. It can adopt a tone off-brand for support, promise a capability on the roadmap but not shipped, or expose internal notes that were never meant for customers. In most stacks the agent's instructions and guardrails live as a string constant in application code, changed by a developer, deployed on the engineering release cycle, and invisible to the content and support leads who actually own the voice of the answers.

That is backwards. The people accountable for what your product says to customers should be able to shape and stage agent behavior without a deploy. In Sanity Context, agent instructions are content governed in the Studio, and Content Releases let editors stage changes to that behavior the same way they stage a website launch, previewing and scheduling before anything reaches production. A revised answer policy for a new pricing tier can be drafted, reviewed, and released on the content team's timeline rather than queued behind the next code push.

Agent Actions extend the same governed surface to LLM-driven content workflows, schema-aware APIs to generate, transform, and translate content that respect the same model and permissions as everything else. The differentiator here is that legacy CMSes create silos, one system for content and another for AI config, while a shared foundation puts instructions, content, and staging in one place. Roles and Permissions, Audit logs, and Content Releases mean an agent's behavior is reviewable and attributable, not a mystery string in a repo. Governance stops being a launch-day scramble and becomes part of how the feature is operated.

Wire it together: from Content Lake to the agent in production

With the content modeled, retrieval blended, embeddings fresh, and behavior governed, the assembly is deliberately unremarkable, which is the goal. Your production agent connects to the Sanity Context MCP endpoint, the surface production agents actually query, and issues retrieval against the Content Lake through the same GROQ hybrid expression you tuned earlier. There is no separate vector service to provision, no reindex pipeline to monitor, and no glue layer reconciling a keyword engine with a vector store. The agent asks; the content store answers with ranked, filtered, current passages; the model summarizes.

Because retrieval is a query rather than a pipeline, you can reason about it the way you reason about any other read path. You can constrain results by the same structured fields you modeled, so a request from a free-tier context never surfaces enterprise-only guidance. You can log what was retrieved for a given answer and trace it back to the exact published document, which is what makes an escalation debuggable rather than a shrug. Content Source Maps let you follow an answer back to its source content, closing the loop between what the assistant said and what the docs actually contain.

Step back and the architecture is the thesis of this guide made concrete. The assistant is thin because the backend is strong. Sanity is the intelligent backend for companies building AI content operations at scale, and an Ask our docs feature is one of the most direct expressions of that: model your business as structured docs, automate everything from hybrid retrieval to embedding freshness, and power anything, in this case a customer-facing assistant, from the same governed foundation your editors already publish through. The failure mode we opened with, a fluent answer citing a flag that no longer exists, does not happen when the answer is a query against content that is current by construction.