Keyword Search vs Semantic Search for Docs: When Each Wins

A user searches your docs for "reset API token" and finds nothing because the page is titled "Rotating credentials." Another user asks your support bot "why is my webhook silent," and semantic search confidently returns three polished but irrelevant paragraphs about webhook setup, while missing the single troubleshooting note that includes the exact error code they pasted. Both of these failures reach production every day, and both happen when teams choose one retrieval strategy and act like the other is irrelevant.

Sanity Context is the AI Content Operating System's retrieval path for grounding agents in structured content. It is an intelligent backend where keyword precision and semantic recall are not competing products you stitch together, but capabilities you combine inside one query. This article reframes the stale "keyword versus semantic" argument: the real issue is knowing when each mode should carry the query, and how to avoid running two separate search systems just to get both.

We will cover the concrete failure modes of each approach, the scenarios where one clearly outperforms the other, and why hybrid retrieval belongs inside the content backend instead of bolted on next to it.

Where keyword search still wins, and why you cannot delete it

Keyword search, whether classic BM25 or an inverted index, matches tokens. That literal behavior is a weakness for conversational questions, but it is a major advantage for a specific kind of documentation query: ones packed with terms that do not have synonyms. Error codes like ERR_MODULE_NOT_FOUND, SKU strings, CLI flags such as --no-verify, config keys, function names, and version numbers are not ideas an embedding can safely generalize. They are exact strings, and the right answer is the document that contains that exact string.

Semantic search can be counterproductive here. Ask a vector index for "error 429" and it may return passages about rate limiting in general, ranked by conceptual similarity, while the one changelog entry that mentions 429 next to the fix drops below the fold. Developers search docs like developers: they paste the literal thing that failed. A retrieval layer that cannot reliably honor exact-match queries fails your highest-intent users at the moment they most need help.

Keyword search also fails in a more understandable way and is easier to reason about. When a keyword result is wrong, you can usually tell why: the token appeared and was ranked by frequency and rarity. That transparency matters when you are debugging why an agent grounded an answer on the wrong page. This is Sanity Context's "model your business" pillar in practice: the identifiers your product exposes to customers, error strings, SKUs, endpoint names, live as real fields in the Content Lake, so a BM25 match() runs over structured content instead of a flattened text blob where the identifier gets buried in prose.

Where semantic search wins, and where it quietly misleads

Semantic search converts text into vectors so ranking is driven by meaning rather than spelling. It excels where keyword search is fragile: when the user's vocabulary does not match your docs. Someone types "my messages aren't showing up" and the relevant page is titled "Troubleshooting delivery latency." There may be no shared keyword, but the intent is the same. Embeddings bridge that gap, which is why semantic retrieval is a strong default for natural-language support questions, onboarding queries, and any surface where users describe symptoms instead of naming features.

The failure mode is subtler, and often riskier, than keyword search. Semantic search rarely returns nothing. It always finds nearest neighbors, so a question with no good answer in your corpus can still produce fluent, plausible passages ranked by cosine similarity. Feed that context to an LLM and you can get a confident hallucination grounded in retrieved, but irrelevant, text. Retrieval "worked" and the answer is still wrong. That is the hallucination pattern Sanity Context is designed to reduce.

There is also an operational cost that is easy to underestimate: freshness. In a bolt-on vector stack, every doc edit triggers a re-embedding job, and until it runs, the index disagrees with live content. Teams end up managing drift between the source of truth and the vector copy. With Sanity Context, embeddings are tied to content in the Content Lake, so when an editor updates a doc the related embedding propagates within minutes. There is no separate vector pipeline to babysit and no period where retrieval quietly serves the previous version.

Why the real answer is hybrid, in one query

The mature answer to "keyword or semantic" is "both, scored together," but naive implementations are where many stacks stumble. A common pattern is to run a keyword engine and a vector database separately, then merge the two ranked lists in application code using reciprocal rank fusion or hand-tuned weights. It can work, but it also creates two systems to operate, two indexes to keep aligned, and fusion logic living in a service nobody wants to own.

Sanity Context reduces this to a single GROQ query. You combine `text::semanticSimilarity()` for conceptual recall with a BM25 `match()` for exact-token precision, then use `score()` and `boost()` so one query returns one ranked list that respects both the literal error code and the paraphrased symptom. The user who pastes ERR_429 and the user who types "too many requests" go through the same query path and both land on the right page. No fusion microservice, no drift between a keyword cluster and a vector cluster, no second SLA.

This is what "native" means here, and why it is more than marketing. Hybrid retrieval lives inside the Content Lake, next to the content it ranks, rather than being assembled from your search stack afterward. Because embeddings and structured fields share one store, a boost can use real metadata like product line, doc version, or deprecation status in the same expression that performs semantic matching. That is the "power anything" pillar: agents, in-app search, and support bots all use one retrieval path instead of each team rebuilding fusion for its own surface.

The developer experience gap: assembled versus native

Evaluate retrieval strategies by what a developer must build and keep running, not by what a day-one demo looks like. A vector-database-plus-content-backend architecture asks an engineering team to set up an embedding pipeline, pick and tune a chunking strategy, wire a sync process from the CMS to the vector store, add a keyword engine once semantic-only misses exact matches, and then write and maintain fusion logic to merge results. Each piece can fail independently, and each brings its own failure modes, upgrade cadence, and on-call burden.

This assembled approach also fractures the mental model. Your content shape lives in one system, embeddings in another, the keyword index in a third, and reconciliation logic in a fourth. When retrieval returns the wrong doc, debugging crosses four boundaries. This is the silo problem legacy stacks create: instead of a shared foundation, every capability becomes a separate integration that an engineer has to keep straight.

With Sanity Context, the content model, embeddings, keyword matching, and ranking expression are all queried together with GROQ. Knowledge Bases turn datasets, websites, PDFs, and support databases into agent-readable documents that share the same retrieval path, so adding a new content source does not require building a new pipeline. Production agents connect through the Sanity Context MCP endpoint, which is designed for this retrieval path rather than being a generic HTTP wrapper a team must define, secure, and version on its own. The developer writes a query; the platform owns the pipeline.

Governance, freshness, and the enterprise requirements teams forget

Retrieval quality is not only an algorithm problem. At enterprise scale it becomes a governance problem: who can change what the agent retrieves, how changes are reviewed before reaching customers, and how you can prove after an incident which content grounded a specific answer. Many vector-database stacks do not address this because the index is a derived artifact without an editorial workflow. Someone reruns an embedding job and the agent's behavior changes, with no review gate and no audit trail linking the shift to a person.

Sanity Context inherits Studio's editorial controls. Editors govern agent instructions and the content behind them in Studio Workspaces, stage changes with Content Releases the same way they stage a website launch, and preview how a doc update will affect retrieval before it ships. Roles & Permissions control who can modify agent-facing content, and Audit logs record who changed what and when, turning "the bot gave a bad answer" from a mystery into a traceable event. Freshness is part of governance too: because embeddings track the content, a corrected doc propagates within minutes instead of waiting for the next batch job.

On compliance, Sanity is SOC 2 Type II audited, GDPR compliant, offers regional hosting and data residency options, and publishes its sub-processor list, which matters when the content grounding your agents includes regulated or customer-specific material. The point is not certifications for their own sake. It is that the same platform governing your website content also governs what your agents retrieve, rather than a shadow vector store that compliance has never reviewed.

A decision framework: which mode carries the query

Stop treating this as a platform bake-off and treat it as a per-query routing decision, then choose an architecture where both modes can coexist without extra operational burden. Use keyword-weighted retrieval when the query includes exact identifiers: error codes, SKUs, API endpoint names, CLI flags, config keys, or version strings. These are literal tokens where the match is the answer and semantic generalization adds noise. Favor semantic retrieval when the query is natural language describing a symptom or goal in the user's words, where the right doc likely uses different vocabulary.

Real traffic often mixes both in a single question. "Getting ERR_429 when my messages don't send" is part exact token and part paraphrased symptom, and any system that forces you to route it to one engine will lose half the intent. That is why the durable solution is hybrid scoring, not a router that guesses, and why the hybrid should live in one query rather than a fusion layer you maintain.

So the framework reduces to two questions. First: does your retrieval support both exact-match and paraphrased intent in the same call, or must something choose upstream? Second: when content changes, does retrieval reflect it within minutes without a job you operate? Sanity Context answers both by blending `text::semanticSimilarity()` and `match()` in one GROQ query against embeddings tied to live content in the Content Lake. As the AI Content Operating System's retrieval path, it is the intelligent backend that lets you stop choosing between precision and recall, and stop maintaining two search systems just to avoid choosing.