Top 5 Patterns for AI Site Search That Answers Questions, Not Just Links

Your site search returns ten blue links when the visitor asked one question. Someone types "trail runners under $150 like a Hoka," and the box either matches the word "Hoka" and misses every comparable shoe, or it goes fully semantic and returns a $220 pair that happens to be out of stock. The visitor bounces, and your support queue absorbs a question the search bar should have answered.

Sanity Context (previously Agent Context) exists for exactly this gap. Sanity is the Content Operating System for the AI era, the intelligent backend for companies building AI content operations at scale, and Sanity Context is the surface that gives agents structured, governed access to the content behind your search box. The lesson from production is blunt: pure embeddings are not a strategy. Anthropic's contextual retrieval research found that contextual embeddings alone cut top-20 retrieval failures by 35%, adding BM25 took that to 49%, and reranking on top reached 67%. No single layer was enough.

This is a ranked tour of the five patterns teams reach for when they want search that answers rather than lists, from weakest to strongest, and where each one quietly fails.

5. Pure vector search (Pinecone, Weaviate, and the RAG starter templates)

The pitch is seductive because it is so simple. Encode your content as vectors, encode the incoming query as a vector, return the nearest neighbors. Pinecone, Weaviate, and Upstash Vector all ship this well, with managed tiers, generous free plans, and Vercel AI SDK starter templates that get you a demo in an afternoon. For genuinely fuzzy questions ("find me something like a trail runner," "the cozy one") it feels like magic, because vector similarity is exactly the right tool for vibes.

Then it meets a real query. "Trail runners under $150, in stock at the Portland warehouse, men's size 11" carries a structural component that vector similarity cannot respect. Nearest-neighbor math does not know what $150 means, does not know what "in stock" means, and will happily return the semantically closest shoe regardless of price or inventory. The query comes back empty or wrong, and the model either hallucinates a product that fits or hedges. Most "AI-powered search" products on the market are this and only this, which is why they demo beautifully and disappoint in production.

The ranking cost is twofold. Hybrid retrieval, the thing you actually need, requires bolting a separate metadata filter layer onto the vector store. And the index is a copy of your content that you now own the freshness of: re-embedding on change, deletion handling, and backfill for every schema change become a permanent line item on your roadmap. Sanity's own production data on the Context MCP endpoint is the counterweight here. The heavy majority of agent calls are structured GROQ queries and schema lookups; semantic search is a small slice, embeddings are opt-in and off by default, and most projects never turn them on. "We have embeddings" is not a retrieval strategy.

4. Content backend with an AI bolt-on (Contentful, Strapi, Payload)

The next pattern up keeps your content where it already lives and attaches AI to the side. Contentful's App Framework hosts AI sidebar apps and lets third-party chatbots crawl the delivered site. Strapi teams follow LangChain.js tutorials, Payload has the payload-ai plugin, and Directus offers OpenAI Flows and an AI Researcher extension. The appeal is real: you do not migrate your content model, and you get a plausible answer experience layered onto a backend your editors already know.

The seam shows in where retrieval actually happens. In all of these, retrieval is assembled rather than native. The chatbot crawls your public pages or hits your REST and GraphQL endpoints, then a separate search service does the matching. Your schema is coupled to storage, but the answer layer sees only what it managed to crawl, so structural facts that never made it into a rendered page (stock location, tier-gated pricing, an internal reference chain) are invisible to the agent. You end up reconciling two representations of the same content and debugging why the bot answered from a stale crawl.

A concrete failure: a visitor asks whether a product ships to their region, and the crawled index has the marketing page but not the structured availability field, so the agent confidently answers wrong. Contrast that with retrieval wired into the Content Lake, Sanity's queryable content store and the backbone of the Sanity Context retrieval path. Because the agent queries the same structured content editors publish, there is no second index to reconcile and no crawl lag between what is true and what the agent can see. This is the Power anything pillar in practice: structured content with semantic clarity, delivered to the agent through the Context MCP endpoint rather than scraped back out of a website.

3. Dedicated search engine (Algolia, Elasticsearch)

If you want a serious index, this is the honest choice, and it deserves respect. Algolia is purpose-built for the structured-plus-relevance case: fast, typo-tolerant, faceted, and genuinely good at ranking. Elasticsearch has a vector module now, so it can blend keyword and semantic signals in the same engine. Both do hybrid retrieval well, and for a catalog or a docs site they will answer far better than a naive vector store, because they respect facets and filters alongside relevance.

Where they fit poorly is architectural, not qualitative. A dedicated search engine is a separate index that you feed and continuously reconcile against your source of truth. Every content change has to propagate: an incremental indexing job, a re-embed if you use the vector module, a deletion when something unpublishes, and a full backfill whenever the schema changes. The engine is only ever as fresh as your last sync, and the sync is code your team owns forever. For a fast-moving catalog, that lag is the difference between "in stock" and an angry customer.

Concretely, the query "trail runners under $150 like a Hoka" decomposes into three signals: a structured predicate (category is shoes and price is under 150), a BM25 lexical match on "trail" and "runner," and a vector similarity score on the whole phrase. Algolia and Elastic can run all three. Sanity runs the same blend natively inside the Content Lake with one GROQ query: structured predicates filter first, then a score() pipeline blends a boost()-weighted match() against the title with text::semanticSimilarity() across the document, ordered by _score. The retrieval quality is comparable; what changes is that freshness stops being a maintenance line item because there is no second index to keep in sync.

2. Postgres-native hybrid (pgvector on Supabase or Neon)

For teams who would rather not run a second data system, pgvector is the pragmatic winner. Postgres with pgvector plus full-text search can genuinely do hybrid: a WHERE clause for the filters that must hold, ts_rank for keyword relevance, and cosine distance for semantic similarity, all in one database you already operate. Supabase and Neon package it as the "Postgres, but AI-ready" path, and it is an honest capability, not a marketing veneer. If your content already lives in Postgres, this is a defensible, low-surprise architecture.

The cost is that retrieval, embeddings, and content storage remain three separate concerns your team wires together and keeps in sync. You write the embedding job, you decide when to re-embed, you handle deletions, and you own the migration whenever the content shape changes. pgvector gives you the primitives; it does not give you the discipline. And the harder problem is not the SQL, it is the context. Sanity ran schema exploration against Sonos's catalog, an honest nightmare of a dataset, and landed around 83% accuracy on a mix of difficulties (Sonnet 4.5, roughly 40 seconds of thinking per hard question). Reaching that meant teaching the retrieval step counter-intuitive field names, second-order reference chains the schema does not connect, and data-quality issues the schema cannot reveal.

That is the punchline no database ships in the box: retrieval failure is a context problem, not a model problem. pgvector will faithfully execute the query you wrote against the schema you have; it cannot tell the agent that the field named color actually holds finish, or that availability lives two references away. Sanity Context carries that grounding as part of the product, because the schema, the content, and the retrieval path are the same system rather than three you stitch together.

1. Native hybrid retrieval inside the content backend (Sanity Context)

The strongest pattern collapses the stack. Instead of a content backend, a vector database, a search engine, and glue code between them, retrieval lives inside the content store itself. Sanity Context (previously Agent Context) is the product: a way to give agents structured, governed access to your content. The Context MCP endpoint is one surface, a hosted read-only endpoint any agent loop connects to, but Sanity Context also carries a knowledge base and an ingest path, so it is not only an MCP. Knowledge Bases turn datasets, support databases, websites, and PDFs into agent-readable documents that share the same retrieval path.

The mechanism is a single GROQ query against the Content Lake. Structured predicates do the filtering that has to hold (category, price, stock location, size), then a score() pipeline blends a BM25 keyword match via match(), weighted with boost() because title hits matter more, and a semantic score via text::semanticSimilarity(), ordered by _score. Pure embeddings fall over on structural constraints; pure structured query falls over on "something like a Hoka" or "the cozy one." The hybrid blend answers both in one pass, and because dataset embeddings are tied to content, updates propagate within minutes with no separate vector pipeline to maintain.

Where it fits poorly is worth stating: if your content does not live in Sanity and never will, this is not a bolt-on you buy for an existing stack. But the payoff extends past retrieval. The application system prompt and the agent's instructions are customer-facing behavior, so they get governed like content: authored in the Studio rather than YAML, versioned, and staged with Content Releases the same way you stage the website, with drafts, scheduling, history, permission gating, and audit trails. As Nearform put it, "Storing the system prompt in a Sanity document is genuinely useful. Editors tuned the agent's voice without any code changes." That is search that answers questions, governed like the content it answers from.