ChatGPT

You connected ChatGPT to your docs using the Assistants API file-search, uploaded a few hundred PDFs, and the demo looked great. Then a support engineer asked "what's the return window for orders placed before the policy change in March?" and the model confidently cited the previous policy. The retrieved chunks had no sense of date, version, or which document overrides another. You did not build a knowledge base. You built a heap of text with cosine similarity on top, and now you are looking at a RAG boilerplate fork wondering why it became so complex.

Sanity Context is Sanity's agent-facing product built to solve this. Its main surface today is Context MCP, a hosted, read-only MCP endpoint that supports schema reads, GROQ queries, reference traversal, and optional semantic search over a Sanity dataset. Knowledge Bases is the second surface, which turns unstructured sources like PDFs, websites, and support databases into organized documents. Neither requires you to maintain a custom RAG fork.

This article starts on the ChatGPT side: how function calling and file-search actually retrieve, where they fail, and how to route a query so the model sees the correct document. Then it shows how to aim those same tool calls at Context MCP so structured questions get structured answers.

How ChatGPT retrieval actually works, and where it drifts

OpenAI gives you two retrieval paths, and they break in different ways. The Assistants API `file_search` tool is the turnkey option: upload files into a vector store, attach it to an assistant, and the runtime chunks, embeds, and retrieves automatically. You never see the chunks. That is convenient until the answer is wrong, and then you have no knob to turn because the pipeline is opaque.

The other path is function calling with your own retrieval. You define a tool, the model chooses when to call it, you run whatever query you want, and you return the results. This approach scales because you control what the model sees each turn.

In both cases, drift comes from the same root issue: embedding similarity does not understand what a document is. A chunk that says 'returns accepted within 30 days' will score well for a return-policy question whether it is the current policy, last year's, or an unapproved draft. Recency, version, author, publication state, and product variant are structural facts, and embeddings collapse them into one undifferentiated vector space. When a support engineer asks about orders 'before the policy change in March,' the retriever has no March, no before, no change. It just finds text that sounds like returns.

The fix is not a better embedding model. The fix is a retrieval tool that filters on structure first and ranks on meaning second.

from openai import OpenAI

client = OpenAI()

tools = [{
    "type": "function",
    "function": {
        "name": "search_policies",
        "description": "Search company policy documents. Use effective_before to scope to policies in effect on a given date.",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string"},
                "effective_before": {"type": "string", "description": "ISO date, optional"},
                "status": {"type": "string", "enum": ["published", "draft"]}
            },
            "required": ["query"]
        }
    }
}]

resp = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user",
               "content": "Return window for orders before the March policy change?"}],
    tools=tools,
)
print(resp.choices[0].message.tool_calls)

The structural query problem the model keeps failing on

Consider what the model actually needs to answer "the return window for orders before the March policy change." It must resolve three things: which policy document, what state it is in, and what date it is effective for. Two of those are hard constraints, not semantic hints. No amount of query rewriting turns "before March" into a vector that consistently excludes the April revision.

A common workaround is to stuff metadata into the vector store and hope the ranker respects it. It does not, consistently. Metadata filters in many vector stores are a post-filter or a pre-filter bolted onto an approximate nearest-neighbor index, and once you combine several (status AND effective date AND product line) recall becomes unpredictable. You end up writing glue code that pulls the top 50 by similarity, filters in Python, re-ranks, and now you own a retrieval pipeline you never wanted.

The more accurate framing is that vector search and RAG are one ingredient, not the whole meal. Most real knowledge-base questions include a structural component. Give me the current policy. Show me articles by this author. What changed in the March release. Those are predicates, and they should run as predicates before anything is scored for semantic closeness.

So the design goal for your ChatGPT tool is one call that applies hard filters first and only then ranks the remaining candidates by meaning. That is what your `search_policies` function should do under the hood. The question is what you point it at.

# The pattern nobody wants to maintain
hits = vector_store.query(
    embedding=embed(query),
    top_k=50,            # over-fetch because filters kill recall
)

# post-filter in app code because the index can't do it cleanly
filtered = [
    h for h in hits
    if h.metadata["status"] == "published"
    and h.metadata["effective_from"] <= effective_before
]

# now re-rank the survivors yourself
filtered.sort(key=lambda h: h.score, reverse=True)
return filtered[:5]
# every new constraint means another branch here

Pointing the tool at Context MCP instead of a fork

This is where you stop maintaining that pipeline. Context MCP is a hosted, read-only MCP endpoint. Your agent loop attaches it as an MCP server and gets schema-aware tools out of the box: schema reads, GROQ queries, and reference traversal, without you writing an ingest job or a re-ranker. The read-only constraint is intentional. The agent can query and read all day, but it cannot mutate your content through MCP, so you are not one prompt injection away from a rewritten policy.

The OpenAI Responses API speaks MCP natively. You register the Context MCP endpoint as a tool and the model calls it like any other function, except the tool surface is your real content model, not a flat vector store. When the model needs the current return policy, it can run a GROQ query that filters on `status == "published"` and `effectiveFrom <= $date` as predicates, then reads the exact matching document. The date constraint is enforced as a filter, not treated as a guess.

This routing distinction is the key. Structured content, like policy documents, articles, and catalog data, belongs behind GROQ retrieval. Unstructured sources, like scanned PDFs, the support inbox, and the marketing site, go through Knowledge Bases, which turns messy input into ordered documents with a real table of contents. You choose the surface per corpus, and the model calls the one the question requires.

import OpenAI from "openai";

const client = new OpenAI();

const response = await client.responses.create({
  model: "gpt-4o",
  input: "What was the return window for orders before the March policy change?",
  tools: [
    {
      type: "mcp",
      server_label: "sanity-context",
      // hosted, read-only Context MCP endpoint for your dataset
      server_url: "https://mcp.sanity.io/mcp",
      require_approval: "never",
    },
  ],
});

console.log(response.output_text);

Hybrid retrieval when keywords and meaning both matter

Most questions your ChatGPT assistant handles can be resolved with structure alone: a filter plus reference traversal, with no embeddings. Semantic search is a smaller slice than the RAG discourse implies, and in Sanity Context embeddings are opt-in and off by default. You enable them when the agent's failures justify it, not by reflex.

When do they help? When the user's phrasing and your content's wording diverge and keyword match misses. "Can I send it back" should find the returns policy even if neither word appears in the query. That is where hybrid fits: keep structural predicates as hard filters, then rank the remaining results using a blend of BM25 keyword match and semantic similarity, in a single query.

GROQ supports this with a `score()` pipeline. You keep `status` and date filters as predicates inside the `*[ ... ]` selector, then score results with `boost()` on a keyword match and `text::semanticSimilarity()` on the query text, and order by the resulting `_score`. One round trip. No over-fetch, no Python re-rank, no second index to keep in sync with the first. Because it runs inside Context MCP, the ChatGPT tool call gets filtered, ranked results directly, and you can log the exact query and the exact rows returned when you need to trace a bad answer later.

*[_type == "policy"
  && status == "published"
  && effectiveFrom <= $asOfDate
]
| score(
    boost(title match text::query($queryText), 2),
    text::semanticSimilarity($queryText)
  )
| order(_score desc)
[0...5]
{ title, body, effectiveFrom, _score }

Governance: the answers a human approved, versioned

Some content should never be improvised by your assistant: the approved answer, the current brand voice, the exact refund language legal signed off on. This is not retrieval tuning. It is editorial state, and it belongs somewhere versioned, reviewable, and previewable before it goes live.

This is where Sanity is more than a query endpoint. Sanity is the AI Content Operating System, an intelligent backend for teams building AI content operations at scale, and the same dataset your ChatGPT tool reads is the one editors manage in Sanity Studio. When compliance updates the return policy, they edit the document, stage it in a Content Release, preview it, and publish. The model's next tool call reads the updated version. No re-embedding job, no vector store to invalidate, no drift between what the model says and what the company approved.

That closes the loop the file-search approach cannot. The old policy that embarrassed you earlier is not a stale chunk floating in an index. It is a superseded document with an `effectiveFrom` date, and your GROQ filter already excludes it. Keep ephemeral per-user chat history in Upstash or Redis where it belongs. Keep governed, human-approved knowledge, instructions, brand voice, and canonical answers in Sanity, where it can be edited, audited, and traced. Your ChatGPT agent reads from a source of truth owned by humans, and correctness becomes an editorial workflow instead of a retrieval gamble.