LangChain.js

Your LangChain.js retriever returns four documents. The agent picks the one that reads closest to the question and confidently answers from a draft that was unpublished three weeks ago. The embedding similarity was high. The answer was wrong. This is the failure mode that vector-only retrieval can't see: the query had a structural component (published state, date range, product variant) that cosine distance has no opinion about.

Sanity Context is Sanity's agent-facing product. Its primary surface today is Context MCP, a hosted, read-only MCP endpoint that exposes schema reads, GROQ queries, reference traversal, and optional semantic search across a Sanity dataset. Knowledge Bases is the second surface, for unstructured sources like PDFs, websites, and support data. It is not another agent framework and it does not replace LangChain.js; it sits underneath your chains as the structured content source your retriever actually calls.

This article stays in LangChain.js first. We'll look at why `similaritySearch` chooses the wrong document, how the `BaseRetriever` interface lets you fix retrieval without rewriting your agent, and where a hybrid GROQ query (structured predicates plus optional semantic ranking) replaces the brittle "embed everything and hope" pattern.

Why similaritySearch returns the wrong document

Start with what actually happens inside a default LangChain.js RAG setup. You build a vector store, embed your documents, and call `similaritySearch`. The retriever ranks every chunk by cosine distance against the query embedding and returns the top k. Nothing in that pipeline knows that one of those chunks is a draft, that another is the 2022 version of a policy that has since been superseded, or that the user asked about the Pro plan and the closest match describes the Free plan.

The reason is structural. An embedding compresses meaning into a vector. Two passages that talk about pricing in similar language sit close together in that space regardless of which plan they describe or whether they're live. The model has no slot for `status == "published"` or `effectiveDate <= now()`. So when the user's real question is 'what's the current refund window for annual Pro subscriptions', the highest-scoring chunk is whichever paragraph is most lexically and semantically about refunds, not the one that is correct.

Developers usually notice this in production, not in the demo. The demo corpus is small and clean. The production corpus has six years of drafts, localized variants, archived campaigns, and three documents that all say 'our refund policy' in slightly different ways. Cranking k higher just hands the LLM more ways to be confidently wrong, and inflates token cost per turn. The problem isn't the embedding model. The problem is that you asked a similarity engine to answer a question that has a filter in it.

import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { OpenAIEmbeddings } from "@langchain/openai";

const store = await MemoryVectorStore.fromDocuments(docs, new OpenAIEmbeddings());

// Ranks purely by cosine distance. No notion of
// publication state, plan, or effective date.
const results = await store.similaritySearch(
  "current refund window for annual Pro subscriptions",
  4
);
// results[0] might be a 2022 draft about the Free plan.

Metadata filters help, until your filter lives in another system

LangChain.js gives you a real lever here: metadata filtering. Most vector stores expose a `filter` argument on `similaritySearch`, and the `SelfQueryRetriever` can even translate a natural-language question into a structured filter plus a semantic query. If you tag every document with `status`, `plan`, and `effectiveDate` at ingestion time, you can constrain the search to published Pro documents and let similarity rank within that set. This is the right instinct, and it fixes the obvious cases.

The trouble starts with where that metadata comes from and how fresh it is. Your filterable fields are a denormalized snapshot taken at embedding time. When an editor unpublishes a document, changes its effective date, or restructures the reference between a plan and its policy, your vector store doesn't know until the next reindex. You end up maintaining a sync job whose entire purpose is to keep the vector store's idea of `status` in agreement with the source of truth. That job fails quietly, and now your filter is filtering on stale truth.

The deeper issue is that metadata filtering treats structure as an afterthought bolted onto a similarity index. Reference traversal in particular falls apart: 'the refund policy referenced by the current Pro plan document' is a join, and a flat metadata filter can't express a join. You can flatten the relationship into the metadata, but every flattening is another thing to keep in sync. At some point you're rebuilding a query engine on top of a vector index, badly.

import { SelfQueryRetriever } from "langchain/retrievers/self_query";
import { AttributeInfo } from "langchain/chains/query_constructor";

const attributes: AttributeInfo[] = [
  { name: "status", type: "string", description: "draft or published" },
  { name: "plan", type: "string", description: "Free, Pro, or Enterprise" },
  { name: "effectiveDate", type: "date", description: "when policy took effect" },
];

const retriever = SelfQueryRetriever.fromLLM({
  llm, vectorStore: store, documentContents: "policy text", attributeInfo: attributes,
});
// Good idea. But status/plan/effectiveDate are a snapshot
// from embedding time, kept in sync by a job that can drift.

Wrap the fix in a custom BaseRetriever

Here's the part that keeps your agent untouched. LangChain.js retrieval is an interface, not a vector store. Anything that extends `BaseRetriever` and implements `_getRelevantDocuments` plugs into the same chains, the same `createRetrievalChain`, the same agent tool. That means you can replace 'embed everything and filter on stale metadata' with a single call to a system that resolves structure and ranking together, and none of your downstream code notices.

The move is to let the source of truth answer the structured part of the query directly. Instead of asking a vector index 'what's semantically close, and by the way filter on a copy of the status field I took last Tuesday', you ask the content source 'give me published Pro policy documents, traverse to the referenced refund policy, and rank those by relevance to this question'. The structural predicates run against live data. The ranking runs over the already-correct candidate set. There's no sync job because there's no second copy of the truth.

This is where Sanity Context comes in as the retriever's data source. The structured query is a GROQ query: filters, references, projections, ordering, all evaluated against the current dataset. Optional semantic ranking layers on top when keyword and structure aren't enough. Your `_getRelevantDocuments` becomes a thin wrapper that runs that query and maps each result to a LangChain `Document`. The agent above it is identical.

import { BaseRetriever } from "@langchain/core/retrievers";
import { Document } from "@langchain/core/documents";
import { createClient } from "next-sanity";

const client = createClient({
  projectId: process.env.SANITY_PROJECT_ID!,
  dataset: "production",
  apiVersion: "2024-03-01",
  useCdn: false,
});

class SanityRetriever extends BaseRetriever {
  lc_namespace = ["app", "retrievers"];
  async _getRelevantDocuments(query: string): Promise<Document[]> {
    const rows = await client.fetch(GROQ_QUERY, { queryText: query });
    return rows.map((r: any) => new Document({
      pageContent: r.body,
      metadata: { id: r._id, plan: r.plan, score: r._score },
    }));
  }
}
// Drop-in: createRetrievalChain(llm, new SanityRetriever()) just works.

Structured retrieval first, semantic ranking only when you need it

It's tempting to assume the answer is 'hybrid search everywhere': embeddings plus keywords plus filters on every query. In practice that's not what production retrieval looks like. The heavy majority of useful retrieval is structured: schema-aware filters and reference traversal that return a small, correct candidate set. Semantic similarity is a smaller slice you reach for when structure and keyword matching genuinely can't disambiguate, for example a long free-text question against a body of prose where the user's wording doesn't match the document's.

That ordering matters for cost and accuracy both. If a GROQ filter narrows ten thousand documents to the six published Pro policies before anything gets ranked, semantic search over six documents is cheap and rarely wrong. Run semantic search over ten thousand documents first and you're paying to rank noise, then filtering the survivors. Embeddings in Sanity Context are opt-in and off by default for exactly this reason: most projects shipping on Context MCP get correct retrieval from structured queries alone and never turn semantic search on.

When you do need ranking, GROQ expresses it inline. `text::semanticSimilarity($queryText)` scores documents by semantic closeness to the query text, `match text::query($queryText)` does BM25 keyword matching, and `score(...)` combines those into a `_score` you can `order()` by. The structural filters stay where they belong, as predicates inside the `*[ ... ]`, evaluated before anything is ranked. One query, one round trip, no second index to keep honest.

*[_type == "policy" && status == "published" && plan == "Pro"]
  | score(
      boost(title match text::query($queryText), 2),
      text::semanticSimilarity($queryText)
    )
  | order(_score desc)[0...6]{
    _id,
    title,
    plan,
    body,
    _score
  }

Skip the sync job with the Context MCP endpoint

Writing a `createClient` call and a GROQ query is the path when you want full control over the shape of the query. There's a faster way in. Sanity ships Context MCP as a hosted, read-only MCP endpoint, and LangChain.js can attach it as an MCP server through the adapter, so your agent gets schema-aware tools without you hand-rolling a single query. The agent can read the schema, run GROQ, and traverse references through tools the endpoint exposes, and it discovers the shape of your content at runtime rather than from hard-coded field names.

The read-only constraint is the point, not a limitation. An agent connected to Context MCP can query and traverse all it wants and cannot mutate your dataset. There is no code path through MCP that publishes, deletes, or edits a document. Writes, when you want them, go through Agent Actions, which is a separate governed surface, not the retrieval endpoint. For a retrieval-augmented agent, read-only is exactly the blast radius you want.

Use the MCP path when you want the agent to assemble its own queries against your schema, and the custom `BaseRetriever` plus GROQ path when you want to pin the query shape yourself. Many teams run both: MCP for exploratory tool use during agent reasoning, a fixed GROQ retriever for the hot path where the query is known and latency matters. Both read the same live dataset, so neither needs a reindex when an editor changes something.

import { MultiServerMCPClient } from "@langchain/mcp-adapters";
import { createReactAgent } from "@langchain/langgraph/prebuilt";
import { ChatOpenAI } from "@langchain/openai";

const mcp = new MultiServerMCPClient({
  mcpServers: {
    sanity: {
      url: "https://mcp.sanity.io/mcp",
      headers: { Authorization: `Bearer ${process.env.SANITY_TOKEN}` },
    },
  },
});

const tools = await mcp.getTools();
const agent = createReactAgent({ llm: new ChatOpenAI({ model: "gpt-4o" }), tools });
// The agent now has read-only schema, GROQ, and reference-traversal tools.

Governance: where editors and agents share one source of truth

There's a class of LangChain.js content that doesn't belong in a vector store at all: your system prompts, the agent's approved answers, brand voice rules, and any canned response a human needs to sign off on before it reaches a user. Teams usually start by hard-coding these as strings in the repo, then graduate to a JSON file, then to a row in whatever database is handy. None of those gives an editor a way to change the agent's behavior, review the change, and roll it back without a deploy.

This is the editorial side of agent state, and it's distinct from ephemeral memory. Per-user chat history is short-lived and high-volume, so it belongs in Upstash, Redis, or your LangChain memory store. The content that should be versioned, reviewed, and previewed before it goes live belongs somewhere a non-engineer can edit it safely. Sanity is the AI Content Operating System for exactly this: an intelligent backend where instructions, knowledge bases, and approved responses are governed content rather than constants in a build.

Concretely, an editor updates a refund-policy answer in the Studio, stages it in a Content Release alongside the related document changes, and an approver previews the whole set before it ships. Your LangChain.js agent reads the published version through the same GROQ query or MCP tool it already uses. The agent's behavior changed, an audit trail exists, and nobody pushed code. For unstructured sources feeding the agent, PDFs, help-center pages, support transcripts, Knowledge Bases turns that mess into ordered documents the same retrieval path can read.