How to Add Semantic Search to a Multi-Language Site

Your German-speaking customer searches for "Rückerstattung" and gets nothing, even though a support article titled "Refund policy" answers the question perfectly. Keyword search fails because the words don't match across languages, and naive vector search returns the English document with a relevance score no editor can explain. Multiply that across nine locales, three content types, and a support agent that now confidently paraphrases the wrong region's return window, and you have the quiet failure mode that erodes trust in every AI feature you ship.

Sanity Context (previously Agent Context) is built for exactly this problem. Sanity is the AI Content Operating System, an intelligent backend that keeps retrieval grounded in real, governed, multi-language content instead of a drifting copy sitting in a separate vector store. Semantic search that respects locale is not a bolt-on search product; it is a property of where and how your content lives.

This guide walks through what breaks when you add semantic search to a multi-language site, how to design a retrieval path that keeps meaning and locale together, and where a content backend that treats hybrid retrieval as native changes the math versus stitching a vector database onto your existing stack.

Why keyword search collapses across languages

The first thing teams discover when they internationalize search is that lexical matching does not travel. A BM25 index built on English tokens has no idea that "refund," "Rückerstattung," "remboursement," and "reembolso" point at the same intent. Stemming and stop-word lists are language-specific, so an index tuned for English quietly mangles German compound nouns and ignores French diacritics. The result is a search box that works in your headquarters language and degrades everywhere else, which is precisely where you have the least visibility into the failure.

The naive fix is to translate the query, run it against each locale, and merge results. That introduces a translation hop that adds latency, cost, and a new hallucination surface: the translation layer can silently reshape the query before it ever reaches your content. It also does nothing for the deeper problem, which is that relevance is about meaning, not tokens. "How long do I have to send this back" should retrieve the return-window article even though it shares almost no vocabulary with it.

Semantic search solves the meaning problem by comparing embeddings rather than words, and multilingual embedding models can place "refund" and "Rückerstattung" near each other in vector space. But pure vector search brings its own failure: it happily returns a semantically close document in the wrong locale, or ranks a vaguely related paragraph above the exact-match answer. Neither approach alone is enough, which is why serious multi-language retrieval is a hybrid problem from the start.

Hybrid retrieval: keeping precision and meaning together

Hybrid retrieval blends two signals that fail in opposite directions. Lexical search (BM25-style matching) is precise when the exact term appears but blind to synonyms and cross-language paraphrase. Semantic search (embedding similarity) captures intent and crosses languages but can drift toward plausible-sounding neighbors. Run them together, normalize the scores, and weight them, and each covers the other's blind spot: the exact SKU or error code still wins on a lexical match, while a fuzzy natural-language question still finds the right article by meaning.

The engineering question is where the blend happens. In a typical stack it happens in application code that fans out to a vector database for the semantic leg, hits a separate search engine or SQL index for the lexical leg, then reconciles two result sets with two different scoring scales, two sets of stale-ness guarantees, and two sources of truth to keep in sync. Every locale you add multiplies the surface area of that reconciliation.

Sanity Context collapses the blend into the query itself. Inside the Content Lake you express hybrid retrieval in a single GROQ query, combining `text::semanticSimilarity()` for the semantic leg with a BM25-style `match()` for the lexical leg, then shaping the final ranking with `score()` and `boost()`. Because both signals are computed over the same governed content in one query, there is no second system to keep consistent and no cross-store scoring mismatch to hand-tune. Hybrid retrieval is native to where the content lives, not assembled around it.

Modeling locale so retrieval can filter and boost by language

Semantic search on a multi-language site is only as good as your content model. If locale is an afterthought, a string suffix on a slug or a folder convention, then retrieval cannot reason about it, and you are back to post-filtering result sets in application code. The fix is to make language a first-class dimension of the schema so that every query can filter, boost, and fall back by locale deterministically.

This maps directly to the first Sanity pillar, model your business. When each translated document carries an explicit language field and a shared reference to its source concept, a GROQ query can scope the semantic and lexical legs to the requested locale, then apply a boost that prefers an in-locale match while still allowing a high-confidence cross-language result to surface when no local translation exists. That last behavior matters: a German user with no German article should get the English answer clearly labeled, not silence.

Modeling locale explicitly also fixes the governance problem underneath. Editors can see which concepts are fully localized and which have gaps, because the gaps are queryable rather than buried in a search log. You can stage a new locale's content in Content Releases and preview how retrieval behaves for that language before it goes live, the same way you stage a website launch. Retrieval quality stops being something you discover in production and becomes something you review in the editorial loop, which is the difference between a demo and a system your support team can trust.

Keeping embeddings fresh when content changes in every locale

The operational tax nobody budgets for is embedding freshness. In a bolt-on architecture, your content lives in one system and your vectors live in another, so every edit triggers a pipeline: detect the change, re-chunk the document, call the embedding model, upsert into the vector store, and hope nothing failed silently between steps. On a single-language site this is annoying. On a nine-locale site where a policy change fans out into nine translations, each needing re-embedding, it becomes a standing reliability risk. Stale vectors mean your agent answers from last quarter's return policy with full confidence.

The counter-intuitive consequence is that the more you invest in translation velocity, the worse a bolt-on vector pipeline serves you, because every improvement to editorial throughput increases the volume of re-embedding work and the window in which vectors and content disagree.

With Sanity Context, dataset embeddings are tied to the content itself. When an editor updates the French refund article, the embedding for that content updates within minutes, with no separate vector pipeline for your team to build, monitor, and page someone about at 2 a.m. There is one source of truth, and the semantic representation of your content tracks the content rather than lagging behind it. This is the second pillar, automate everything, applied to the least glamorous but most failure-prone part of the stack: keeping what the model retrieves aligned with what your editors actually published.

Wiring agents to multi-language retrieval without a glue layer

Once retrieval is correct, the last mile is getting it to your agents and applications without rebuilding the integration for every consumer. A support bot, an on-site search box, an internal knowledge assistant, and a documentation agent all need the same locale-aware hybrid retrieval, and you do not want four different teams each reimplementing query fan-out, locale fallback, and score blending.

This is the third pillar, power anything. Because the retrieval logic lives in GROQ over the Content Lake, every consumer runs the same query semantics rather than its own approximation. Production agents connect through the Sanity Context MCP endpoint, so an agent framework gets locale-aware hybrid retrieval as a first-class capability instead of a custom HTTP integration against a vector store. Knowledge Bases extend the same retrieval path over datasets, websites, PDFs, and support databases, so a document that started life outside the Studio is queried the same way as a natively modeled article.

The governance payoff closes the loop. The instructions that steer an agent, including how it should handle a missing translation or which locale to prefer, are content that editors manage in Studio and stage through Content Releases, not strings hardcoded in a prompt file that only an engineer can change. Agent Actions give schema-aware APIs for the generate, transform, and translate workflows that produce localized content in the first place. The retrieval that grounds your agents and the content operation that feeds them are the same system, which is what it means to run content end to end rather than stopping at publish.

What to evaluate before you commit to an architecture

When you compare approaches, resist scoring them on whether they can technically do semantic search, because nearly all of them can. Score them on the operational questions that decide whether multi-language retrieval survives contact with production. First: is hybrid retrieval native, or are you reconciling a vector store and a search index in application code, with two scoring scales to reconcile per locale? Second: when an editor changes a translation, how long until retrieval reflects it, and who owns the pipeline that makes that happen? Third: can a non-engineer see and govern which locales are covered and how the agent behaves when one is missing?

A content backend that treats retrieval as native answers all three in one system. A vector database answers the semantic leg brilliantly and leaves the content, the lexical leg, the freshness pipeline, and the governance to you. A traditional content backend with an AI add-on gives you the content model but pushes retrieval back out to an external search stack. The distinctions matter most exactly where multi-language sites hurt: at the seams between systems, multiplied by every locale.

Sanity is the intelligent backend for companies building AI content operations at scale, which on a multi-language site means the meaning of your content, the freshness of its embeddings, and the locale rules your agents follow all live in one governed place. That is the reframe: semantic search across languages is not a search feature you add, it is a property of a content operating system you build on.