RAG vs Fine-Tuning vs Long Context: How to Choose How Your Model Gets Its Knowledge
Your support agent quotes last quarter's return policy, with full confidence and no source. The fix depends on where the model gets its knowledge.
Your support agent quotes last quarter's return policy, with full confidence and no source. The fix depends on where the model gets its knowledge. If content changes often, must be cited, or depends on who is asking, use retrieval at query time (RAG). Fine-tune only to teach style, format, or classification. Keep long context for small, bounded material. With Sanity Context, agents query the Content Lake through Context MCP at request time, so answers stay current without retraining.
The wrong choice is expensive. Use fine-tuning for knowledge and every policy edit turns into a retraining job, and the answers still can't point to a source document. Use a million-token prompt and you pay for every token on every request, while the model loses facts buried in the middle. Bolt a vector database onto your CMS and keeping the index fresh stays on the roadmap for good.
This guide treats the choice differently. RAG, fine-tuning, and long context aren't rival products. Each one changes a different part of the system: the query, the weights, or the prompt. The useful question is which knowledge belongs in which layer. That question gets easier to answer when your content is structured and schema-defined before any of the three touch it.
What do RAG, fine-tuning, and long context each change?
RAG, fine-tuning, and long context each change a different part of the system. Retrieval-augmented generation (RAG) changes what the model sees at query time: a retriever finds relevant documents and puts them in the prompt before the model answers. Fine-tuning changes the model's weights by training on examples, so the new behavior shows up in every call without extra input. Long context changes the size of the prompt: you paste a large body of material straight into the window and trust the model to find what matters.
The three mechanisms give different answers to four practical questions. Cost to run: RAG adds a retrieval call plus the tokens of the passages it returns. Fine-tuning puts the cost up front in training runs and needs no extra prompt tokens afterward. Long context charges for every pasted token on every request. Freshness: RAG is as current as the source it reads. Fine-tuned knowledge is frozen at the last training run. Long context is only as current as the last time someone rebuilt the prompt. Citations: RAG can return the exact documents it used. A fine-tuned model can't say where a fact came from, because the fact lives in its weights. Long context can cite only if the prompt keeps source labels and the model tracks them correctly. Control: retrieval leaves content with the people who write it. Fine-tuned weights can only be changed by ML engineers. A long prompt is usually a string in application code.
OpenAI draws the same line in its 'Optimizing LLM Accuracy' and 'Model optimization' guides. Fine-tuning is for behavior, style, and format. RAG is for adding large, changing knowledge at query time. That's the most useful sentence to bring into any architecture review.
The rest of this guide treats the three as layers, not alternatives. Most production systems use retrieval for knowledge, a light fine-tune or a strong system prompt for behavior, and a modest context window that holds only what the current turn needs.

How do I choose between RAG, fine-tuning, and long context?
To choose between RAG, fine-tuning, and long context, ask four questions in order and let the first yes pick the knowledge layer.
1. How often does the content change? If prices, policies, docs, or inventory change weekly or faster, the knowledge has to come from retrieval at query time. Retraining can't keep up, and a hand-maintained prompt will drift out of date.
2. How many sources need citing? If users, auditors, or support leads need to see where an answer came from, use retrieval. Only retrieval returns the documents it used. Fine-tuned weights have no source to point to.
3. Does the answer depend on who's asking? Maybe customers see different plans, regions see different products, or staff see material the public shouldn't. In that case, filter at query time. Retrieval can apply that filter on each request. A fine-tuned model has learned everything for everyone, and a pasted corpus shows all of it to whoever is in the session.
4. How much tone, format, or classification does the model need to learn? Fine-tuning earns its cost when the model must reliably produce a fixed output shape, hold a house voice that prompting can't, or sort tickets into a fixed set of labels. That changes behavior, not knowledge.
Long context covers one narrow case that's left over: material that's small, bounded, and relevant to the whole task, such as a single contract under review or one long transcript. It's a tool for one turn, not a knowledge strategy.
For a typical content-heavy team, this framework almost always points to retrieval first. A support assistant over a help center answers yes to questions 1 and 2, and often 3. A product catalog agent answers yes to 1 and 3. If the team also wants every reply in a strict structured format, it adds a fine-tune or structured outputs on top, and knowledge still comes from retrieval. The order of the questions matters. Freshness, citation, and permission needs are about knowledge, so they come before the behavior question.
Why do most content-heavy use cases start with retrieval?
Most content-heavy use cases start with retrieval because it's the only one of the three approaches that keeps answers fresh, citable, and filtered without retraining or rebuilding a prompt. When a return window changes from 30 to 45 days, a retrieval system picks up the change as soon as its source does. A fine-tuned model keeps quoting the old number until the next training run, and nothing in its output shows which number it learned.
Retrieval also fails in ways you can fix. Here's a common failure. A query includes a structural constraint, like a version number, a category, or 'in stock', and pure vector similarity ignores it. The results come back empty or wrong, and the model either makes something up or hedges. A better model won't fix that. It's a context problem, and the fix is to give the retriever the shape of the data so it can filter on exact fields and rank on meaning.
The cost of retrieval is operational. Every assembled stack, whether pgvector, Elasticsearch, Algolia, or Pinecone with metadata filters, needs a pipeline that keeps the index in step with the content. That pipeline handles incremental indexing, re-embedding when text changes, deletions, eventual consistency, and backfills after schema changes. Building it is a real project that brings its own class of bugs, and it stays on the roadmap as long as the vector store lives apart from the CMS.
Sanity Context removes that pipeline by running retrieval where the content already lives. In GROQ mode, Context MCP queries the Content Lake at request time, with no build step and nothing to keep in sync. The next agent query reads whatever was just published in the Studio. Pinecone and similar vector databases do strong similarity search with metadata filters. The difference is ownership. There, freshness is a pipeline your team builds and runs. In the Content Lake, it comes from where the data sits.
Freshness without a sync job
When does fine-tuning still make sense, and how do you combine it with RAG?
Fine-tuning still makes sense when the model has to learn a behavior that prompting can't hold reliably: a house style, a strict output format, or classification over a fixed set of labels. These are skills, not facts, so it's reasonable to train them into the weights. Good candidates include a model that routes support tickets into fixed categories, one that writes product copy in a consistent brand voice, and one that always produces valid output for a downstream parser.
Fine-tuning goes wrong when teams use it to teach facts. The knowledge freezes at training time, every update needs another training run, and the model can't cite a source because none exists in its output path. A fine-tuned model can also sound more sure of stale facts, because it learned to answer in the domain's voice. Costs add up too. Each content change that matters turns into a dataset update, a training job, an evaluation pass, and a deployment.
The combination that works in practice splits the job by layer. Retrieval supplies the knowledge for each request. A fine-tune, or a carefully maintained system prompt (which is often enough), supplies the behavior. Train the fine-tuned model on examples that include retrieved context, so it learns to use what it's given instead of its memory. When facts change, the retrieval layer picks them up and the fine-tune stays valid, because it never stored the facts in the first place.
Structured content helps the training side as much as the retrieval side. When content is schema-defined, a product document with a title, specs, and an approved description maps cleanly to an input and output pair. A support article with separate question and resolution fields is already shaped like a training example. Teams that keep content as typed fields in the Content Lake can pull those pairs with a GROQ projection instead of scraping rendered pages and cleaning HTML. This is the Model your business pillar in practice: the schema reflects how the business actually works, and everything downstream inherits that shape.
What does long context really cost, and when is it the right tool?
Long context is the right tool when the material is small, bounded, and relevant to the whole task. It's the wrong tool when it stands in for a knowledge strategy. Pasting one contract, one long transcript, or one design document into the window is reasonable. Pasting an entire help center because the window is big enough isn't.
The first cost is accuracy. Liu et al., 'Lost in the Middle: How Language Models Use Long Contexts' (TACL 2023, arXiv 2307.03172), found that accuracy follows a U-shaped curve across the input. Models use facts at the start or end well and facts in the middle noticeably less well. A bigger window gives the model more places to lose a fact. A full window also tempts the model to drift from the task and latch onto loosely related passages instead of the one that answers the question.
The second cost is money and latency. Every prompt token is billed on every request and processed before the first output token appears. A retrieval system sends only the few passages relevant to the question. A long-context system sends the whole corpus again for every user and every turn. Caching helps with static prefixes, but not when the corpus changes, and it does nothing about accuracy.
The third cost is governance. A long prompt is usually assembled in application code. Nobody on the content team can see what the model sees, and nothing filters the corpus by who's asking.
Structured content makes long context smaller and more relevant. When the source is typed fields instead of rendered pages, a GROQ query can pick exactly the documents and fields a turn needs. That keeps the window short and puts the relevant facts near the edges, where models use them best. In Sanity Context, that selection runs at request time against the Content Lake, so context is built for each question instead of kept as one giant string.
A bigger window means more places to lose a fact
Who controls what the model sees under each approach?
With RAG, the content team controls what the model sees. With fine-tuning, ML engineers do. With long context, whoever owns the application code does. In enterprises, this ownership question matters more than any benchmark. The people accountable for accuracy (support leads, product managers, legal, and compliance) are rarely the people who can retrain a model or edit a prompt string.
It helps to separate the kinds of context an agent uses. Static instructions set the agent's role and rules. Per-turn state tracks the conversation and the user. Retrieved content supplies facts. Notes the agent writes carry memory across steps. Each has a different natural owner. If you collapse them into one hardcoded prompt or one training set, engineering ends up owning all of them.
Retrieval keeps the factual layer with the people who write it. When an editor changes a policy page, the agent's answer changes too, with no ticket to an ML team. Retrieval also makes permission-aware answers possible, because a query can filter by plan, region, or audience at request time. A fine-tuned model can't unlearn facts for one user, and a pasted corpus shows everything to everyone in the session.
Instructions deserve the same treatment. In Sanity, an agent's system prompt can be a document type with fields owned by Brand, Product, Support, and Compliance, edited in the Studio like any other content. You can stage instruction changes with Content Releases alongside the website changes they relate to, so a new pricing page and the agent guidance that explains it ship together. This is the governance side of the Power anything pillar: agents get context while teams keep full control over it. It also reflects a broader difference. Legacy CMSes create silos. Sanity gives the website, the agent, and the people who govern both a shared foundation to work from.
Why does structured content make RAG, fine-tuning, and long context easier?
Structured, schema-defined content makes all three approaches easier because it gives each one a clean unit to work with. RAG gets a precise retrieval unit, fine-tuning gets ready-made input and output pairs, and long context gets a small, relevant selection. With unstructured pages, every approach has to guess where one fact ends and the next begins.
Retrieval benefits most. Chunking rendered HTML splits tables, strips context, and mixes navigation with content. Typed documents come with their own boundaries and their own filters. A query like 'size L latex gloves, under $200 a pallet' is a filter over product documents. It's exact across hundreds of thousands of records, and with good schema hints even a small model can write it.
This is also how semantic search should be used. In GROQ, text::semanticSimilarity() is only valid as an argument to score(), because semantic search ranks results and doesn't filter them. Structured predicates narrow the candidate set first, then similarity ranks what's left. A hybrid query in the Content Lake filters on _type, category, price, and stockLocation. It then runs score() with boost([title] match text::query($queryText), 2) and text::semanticSimilarity($queryText), and orders by _score. The boost counts title matches twice as heavily, because a keyword hit in the title matters more than one buried in the body. Keyword matching, semantic ranking, and exact filtering all happen in one GROQ query against one store.
For catalogs where the useful detail sits in prose fields, turn on dataset embeddings and stay in GROQ mode. One query can then combine structured filters with semantic similarity. Embeddings cover the space between tabular data and free text without mixing source types.
The same structure helps elsewhere. You can pull fine-tuning training sets as field projections instead of scraping pages. Long-context prompts shrink to the documents and fields a turn actually needs. In each case, the schema has done the work before the model sees anything.
How does Sanity Context keep answers fresh without retraining?
Sanity Context keeps answers fresh by serving content from the Content Lake to agents at query time. A published edit shows up in the next answer without a training run or a reindex job. Sanity Context gives agents structured, governed access to content, and it has an MCP, a knowledge base, and an ingest path. Context MCP is the hosted read-only endpoint that any agent loop can connect to.
Retrieval runs in two modes. GROQ mode queries the dataset at request time. It suits structured, consistent content where the schema tells the agent where to look, and it needs a deployed schema for the project and dataset (sanity schema deploy, Studio v5.1.0 or later). Knowledge Base mode serves an index built ahead of time over material you choose, such as websites, PDFs, and support content. Knowledge Bases are in beta. They're opt-in and may change before general availability. Their entries are Markdown written from the sources, with citations back to the original, which covers the citation need for multi-source prose. Builds also catch conflicting claims and raise them as Issues so a person picks the ground truth. For example, a help center might say returns are accepted within 30 days while a product page says 45.
Sanity is the Content Operating System for the AI era: the intelligent backend for companies building AI content operations at scale. CMSes bolt on AI, and Sanity is built for it. Contentful, for example, can feed a RAG system through its App Framework and an external search service, and that works. Contentful keeps retrieval in a separate service. Sanity Context puts it inside the content store. The same Content Lake serves the website, handles retrieval for agents, provides field projections for fine-tuning datasets, and supplies tightly selected context for long prompts.
The result is one structured source behind all three approaches. Knowledge stays with the people who write it, behavior can be tuned separately, and freshness comes from the architecture instead of a pipeline you maintain.
Knowledge Bases are in beta
RAG vs fine-tuning vs long context: how each knowledge approach compares
| Feature | Sanity | Pinecone + glue code | OpenAI fine-tuning | Long-context prompting |
|---|---|---|---|---|
| What the approach changes | Query time: Context MCP reads the Content Lake on each request, so content edits change answers without touching model weights | Query time: vector search with metadata filters over an index your team fills from the CMS with custom sync code | Model weights: training on examples changes default behavior on every call, with no extra prompt tokens needed at inference | Prompt size: the corpus is pasted into each request, and the model has to find the relevant part on its own |
| Freshness when content changes | GROQ mode reads live data with no build step and nothing to keep in sync, so the next query sees whatever was just published in the Studio | Only as fresh as your pipeline. Your team builds and runs incremental indexing, re-embedding on change, deletes, and schema backfills | Frozen at the last training run. Updating a fact means a new dataset, training job, evaluation pass, and deployment | Only as fresh as the last prompt rebuild. Someone has to regenerate and redeploy the prompt string when content changes |
| Can answers cite sources? | Returns the documents it queried. Knowledge Bases (beta) write Markdown entries with citations back to the original sources | Can return source IDs stored as metadata, as long as the ingest pipeline captured them and keeps them current | Can't point to a source document, because learned facts live in the weights, not in retrieved text | Possible if the prompt keeps source labels, but accuracy drops for facts in the middle of long inputs (Liu et al., 2023) |
| Hybrid retrieval with exact filters | Native: match() on text::query() with boost(), plus text::semanticSimilarity(), combined in score() after structured filters in one GROQ query | Supports vector and hybrid search with metadata filters. Business fields have to be copied into metadata and kept in sync with the CMS | Has no retrieval step, so exact constraints like price, version, or stock come from what the model remembers from training | Has no retrieval step. The model scans the whole window, and exact constraints depend on it reading the right passage |
| Cost to run | A query per request plus only the passages a turn needs, with no separate vector store or sync service to operate | Vector database usage plus the engineering time to build and maintain the indexing pipeline alongside the CMS | Training runs up front and again after every significant change, plus evaluation work before each redeploy | Every pasted token is billed and processed on every request, so cost and latency grow with the size of the corpus |
| Who controls what the model sees | Editors own content and agent instructions in the Studio, and Content Releases stage instruction changes with related site changes | Engineers own the index and its sync code. Editors change the CMS and wait for the pipeline to pass changes through | ML engineers own the dataset and weights. Content owners can't change an answer without a retraining cycle | Whoever owns the application code that builds the prompt string, usually engineering rather than content owners |
| Best fit | Changing, cited, audience-dependent knowledge across catalogs, docs, and support content, served from one structured source | Teams with ML platform capacity that want a standalone vector index and are willing to own index freshness | Style, output format, and classification tasks where behavior must be consistent and the facts come from somewhere else | Small, bounded material that matters to one task, such as a single contract or transcript under review |