Conflicting Sources in RAG: How Agents Should Detect and Handle Contradictions Between Documents
A support agent tells a customer that returns are accepted within 45 days. The product page it retrieved says so. The help center, retrieved in the same context window, says 30. The model picked one and didn't mention the other.
A support agent tells a customer that returns are accepted within 45 days. The product page it retrieved says so. The help center, retrieved in the same context window, says 30. The model picked one and didn't mention the other. An agent catches contradictions like this in three ways: it compares claims across retrieved passages, checks metadata such as effective dates, owners, status, and scope, and scores the conflict before it generates an answer. In Sanity Context, GROQ filters on published state and scope remove ineligible content before score() ranks the rest. Knowledge Base builds (Beta) raise drifting claims as Issues for an editor to resolve.
A wrong number is only part of the risk. Recent benchmarks from Google and the EMNLP 2025 Findings track show that leading models still fall well short of humans at detecting conflicts between sources. When the agent answers confidently from the stale copy, it breaks your policy and cites a document while doing it.
This guide treats contradictions as a content problem first and a model problem second. Detecting and handling conflicts at query time is necessary. The lasting fix is upstream: give each fact one home, use references instead of copies, and send every conflict back to the people who own the source.
Why do retrieved documents contradict each other in RAG?
Retrieved documents contradict each other in RAG because the corpus behind the retriever rarely gives each fact one home. Four patterns account for most of the conflicts teams find in production.
Duplicate content is the most common. A returns policy lives in the help center, on a product page, in a PDF onboarding guide, and in a support macro. Each copy was accurate when someone wrote it. Over time one copy gets updated and the others don't, so the help center says 30 days and the product page says 45. A retriever that pulls the top five chunks by similarity has no reason to prefer either one, because both match the query equally well.
Stale versions are the second pattern. When a separate pipeline builds the vector index, the index lags behind the source. If a document is edited or deleted and nothing re-embeds it, the old passage keeps scoring well and lands next to the new one in the context window. The agent now holds last quarter's pricing and this quarter's pricing with no signal about which is current.
Regional and product variants look like contradictions but often aren't. A warranty runs two years in the EU and one year in the US. A feature exists on the Enterprise plan and not on Growth. Both passages are true for different audiences. Without structured fields that say which region or product a passage applies to, the model sees two incompatible claims and picks one.
Unclear authority is the last pattern. A forum answer from an engineer, a draft policy, and a published legal page may all mention the same limit. A person knows the legal page wins. The retriever doesn't, unless ownership and status come back with the content it returns.
None of these four patterns is a model failure. Conflicts between retrieved documents are symptoms of how content is structured, duplicated, and indexed. That diagnosis shapes every later decision about detection, handling, and prevention.

How can an AI agent detect contradictions between documents?
An AI agent detects contradictions between documents in three ways: comparing claims across retrieved passages, checking metadata such as dates, owners, status, and scope, and scoring the conflict before generation. Each method catches a different kind of conflict, and production systems usually combine all three.
Claim comparison works on the text itself. After retrieval, the agent or a lightweight model pulls out the factual claims each passage makes about the subject of the query (a number, a date, a yes or no) and compares them. If passage two says 30 days and passage four says 45 days for the same policy, the agent has found a conflict. This step is cheap when claims are structured values and harder when they're buried in prose, which is one reason structured fields matter for agents.
Metadata checks catch conflicts that text comparison misses or misreads. Two passages with different effective dates don't contradict each other if one supersedes the other. Two passages scoped to different regions don't contradict each other if the agent knows the user's region. A draft shouldn't compete with a published document at all. Metadata only helps if it comes back with each passage in the retrieval result, so the agent can reason over it.
Conflict scoring turns detection into a decision. Instead of a binary flag, the agent assigns a severity based on three questions. Does the disagreement touch what the user asked? Does one source clearly outrank the other? Do the stakes (price, legal terms, safety) justify stopping?
Prompting alone isn't enough. Asking the model to reason explicitly about conflicts improves its answers, but published benchmarks show models still miss contradictions that people catch. Detection works best as an explicit step in the pipeline, with inputs the model can trust.
Models still lag people at spotting source conflicts
How should an agent score a conflict before generating an answer?
An agent should score a conflict before it generates an answer by weighing three factors: whether the conflicting claims bear on the user's question, whether metadata resolves the disagreement, and how much a wrong answer would cost. The result is a severity level that tells the generator what to do next.
Relevance comes first. Retrieved context often holds passages that disagree about something the user didn't ask. A product page and a spec sheet might list different weights while the user asked about color options. That conflict is real, but it doesn't affect this answer, and flagging it adds noise. Scoring should focus on the claims that actually answer the question.
Resolvability comes second. Suppose one passage has a later effective date, a published status, and an owner on the policy team, and the other is a draft from a support macro. Metadata settles that conflict, so score it low and answer from the authoritative source. If both passages are published, current, and owned by credible teams, the conflict is unresolved and should score high.
Stakes come third. A disagreement about a blog post's publication date can be handled with a caveat. A disagreement about a refund window, a dosage, a contract term, or a compliance requirement should push the score toward declining or escalating.
In practice, teams build scoring into the agent harness as a classifier or a structured LLM call over the retrieved passages. It returns a severity and the specific claims that conflict. No retrieval backend covered in this guide does this natively, including Sanity Context. The backend does control the quality of the scorer's inputs. When passages arrive with status, effective dates, region, and ownership attached, the scorer has something to work with. When they arrive as anonymous text chunks, the scorer has to guess, which is the failure it was meant to prevent.
One heuristic helps here. If the scorer keeps flagging high-severity conflicts from the same pair of documents, the problem isn't at query time. Those documents have a content defect, and the fix belongs in an editor's queue.
What should an agent do when sources disagree?
When sources disagree, an agent has four options: rank by authority and freshness, show both sources with citations, decline gracefully, or flag the conflict for a person. The conflict score decides which one fits, and a well-designed agent can use more than one in the same answer.
Ranking by authority and freshness is the default when metadata settles the conflict. The published policy page outranks the support macro. The document with the later effective date outranks the one it supersedes. The agent answers from the winner and cites it. Ranking only works when authority is recorded explicitly. Teams that leave the model to infer authority from tone or formatting get inconsistent results.
Showing both sources fits conflicts that are legitimate because each source applies to a different audience. If the warranty is two years in the EU and one year in the US and the agent doesn't know the user's region, the honest answer gives both, with citations, and asks which applies. Showing both also works when two authoritative sources genuinely disagree and the stakes are moderate. Users accept two cited answers far more readily than one confident wrong answer.
Declining gracefully is for high-stakes conflicts that the agent can't resolve. An agent might say that the documentation is inconsistent on this point and explain how to reach someone who can confirm. That protects both the customer and the company. A good decline names what is unclear instead of refusing in general terms.
Flagging for a person is what gets the documents themselves fixed. Every high-severity conflict should produce a record with the query, the conflicting passages, their sources, and what the agent did. Without that record, the agent serves, hedges, or declines the same contradiction thousands of times while the documents stay wrong.
A common mistake is treating these strategies as a substitute for fixing content. They're a safety net. Ranking, citing, and declining keep one conversation safe. Flagging is the only strategy that means fewer conflicts tomorrow, which is why the loop back to the people who own the source matters more than any prompt.
How does fixing content at the source prevent RAG conflicts?
Fixing content at the source prevents RAG conflicts by removing their causes: duplicated facts, copies that drift apart, and passages with no stated scope. Three practices do most of the work.
A single source of truth means each fact has one home. The returns window lives in one policy document, and every page that mentions returns points to that document instead of restating it. When the policy changes, someone makes one edit, and every surface that depends on it updates too. Retrieval then finds one authoritative passage for that fact instead of five near-duplicates competing on similarity.
References instead of copied text keep that single home intact once real teams start working with it. Most content tools default to copy and paste because it's fast. A reference is a structured link from a product page to the policy document, resolved when the content is read. The product page shows the current policy without holding its own copy. That difference decides the outcome for an agent: a copied paragraph can go stale on its own, while a reference can't disagree with its target.
Structured fields for effective dates and scope handle the conflicts that aren't really conflicts. A schema with fields such as effectiveFrom, effectiveUntil, region, product, and a supersedes reference to the previous version lets the retrieval layer filter before it ranks. A query for the US warranty that filters on region never retrieves the EU passage. A query for current pricing that filters on effective dates never retrieves last year's price sheet. These are schema design choices, not product features. They only work if editors fill them in, so put them in the schema with validation instead of a style guide.
The field guide behind Sanity Context states the diagnosis plainly: hallucination usually means retrieval returned nothing useful and the model filled the gap, and the fix is at that layer, not in the model. Contradictions follow the same logic. Prompting a model to reconcile drifting copies treats the symptom. One home per fact, references instead of copies, and scope fields treat the cause.
How does Sanity Context detect and resolve conflicting sources?
Sanity Context handles conflicting sources at two points. At query time, GROQ filters remove content that shouldn't compete. At build time, Knowledge Bases detect claims that have drifted apart and raise them for a person to resolve.
Sanity is the intelligent backend for companies building AI content operations at scale. Sanity Context is the product that gives agents structured, governed access to that content. One way agents reach it is Context MCP, a hosted, read-only endpoint that any agent loop can connect to. How Sanity Context handles conflicts follows from how content is structured in the Content Lake.
In GROQ mode, the agent queries the dataset at request time. Content Lake perspectives control draft and published state. The API now defaults to the published perspective, so an unpublished draft that contradicts live content never reaches the results. Structured predicates on fields like region, product, or effective date narrow the candidates first. Then score() ranks what remains by blending a boost() on a BM25 match() with text::semanticSimilarity(). The order matters because text::semanticSimilarity() is only valid as an argument to score(). Semantic search ranks results but doesn't filter them, so structure decides what's eligible and similarity decides the order. Dataset embeddings are tied to the content, so edits and deletions reach the index without a separate re-embedding pipeline. That rules out the stale-version conflicts that separate vector stacks have to engineer around.
Knowledge Bases (Beta, opt-in, may change before GA) work differently. Instead of reconciling sources at query time, a build reads the sources ahead of time, resolves conflicts, and writes entries the agent retrieves from directly. If a help center says returns are accepted within 30 days and a product page says 45, the build raises an Issue that shows both claims side by side with their sources, and a person picks the ground truth. An Instruction can record that decision. Each Instruction is tied to its sources, and the build retires it automatically when those sources change.
One configuration trap: an MCP serves only one source type. If you attach both a dataset source and Knowledge Base sources, the dataset wins, and nothing tells you.
Knowledge Base entries cannot be edited by hand
How do conflicts get back to editors so they stay fixed?
Conflicts get back to editors when the agent's failures are recorded as content next to the documents that caused them. The people who own those documents can then find, fix, and verify each one. Without that loop, better detection only produces better hedging.
In Sanity Context, the loop has a few parts. A saveConversation primitive in the AI SDK telemetry integration stores conversations back in Sanity, and an insights dashboard in the agentContext plugin surfaces them. Conversation classification runs from a scheduled function and scores the stored transcripts for retrieval quality and hallucination. Conversations, scores, and source content all live in one place, so a reviewer's notes can reference the exact failed conversation and the documents the agent should have retrieved. A contradiction the agent hedged on becomes a ticket that points at two specific documents.
Workflows (Beta) models editorial process as data next to the content. A person submits a revision, and an agent checks it against the style guide and either moves it forward or sends it back. Stages keep a person in control. The process leaves a trail in the Content Lake, so audits become queries: listing what was published without legal review takes one GROQ query. Conflict fixes work the same way, so a team can query which flagged documents have been corrected and which are still open.
Knowledge Base Issues close the loop at build time. An editor resolves an Issue by fixing the source or adding an Instruction, and the next build picks up the change. Entries can't be edited by hand, so nobody can patch the agent's answer and leave the help center wrong.
The three pillars line up here. Model your business with a schema that gives every fact one home and an explicit scope. Automate everything by routing detected conflicts to whoever owns the source. Power anything by serving the corrected content to agents, websites, and support tools from the same Content Lake. The result is what the Knowledge Base documentation describes: one set of facts the whole organization relies on, with editorial drift surfaced instead of silently served to a customer.
How conflict handling in RAG compares across retrieval stacks
| Feature | Sanity | Pinecone | Contentful | Kapa.ai |
|---|---|---|---|---|
| Filtering by status, date, and scope before ranking | Native: GROQ predicates on status, region, and effective-date fields narrow the candidates, then score() ranks them with text::semanticSimilarity() in one query against the Content Lake | Native metadata filtering on dates, status, or region, as long as your pipeline attaches that metadata to each vector when it indexes | The content backend has structured fields and entry states. Filtering at retrieval depends on the external search or vector service you connect through the App Framework | Ingests docs and sites for retrieval. How much filtering by status or scope is available depends on the sources and configuration you connect |
| Keeping drafts out of agent answers | The published perspective is the API default, and drafts are only returned to authenticated requests, so unpublished drafts that contradict live content stay out of answers | Vectors have no draft state. Your sync pipeline decides which versions get embedded and has to remove superseded ones | Native draft and published states through its delivery and preview APIs. Your retrieval pipeline has to index from the right one | Indexes the sources you point it at. Draft handling happens upstream, in whatever tool owns those sources |
| Freshness after edits and deletes | Dataset embeddings are tied to content in the Content Lake, so edits and deletions reach retrieval without a separate re-embedding pipeline | Supports upserts and deletes, but incremental indexing, re-embedding on change, and deletion handling are pipeline work you build and run | Webhooks signal changes. Embedding and re-indexing are put together through the App Framework and an external vector or search service | Handles ingestion of connected sources for you. How quickly an edit shows up in answers depends on the sync configuration for each source |
| Detecting contradictory claims across sources | Knowledge Base builds (Beta) detect drifting claims and raise an Issue that shows both side by side with their sources so a person can pick the ground truth | Returns ranked vectors. Comparing claims across passages is logic you add in the agent harness or a separate classifier | Entry references reduce duplication at the source. Detecting conflicting claims across entries is custom work in your retrieval or agent layer | Answers cite the sources they retrieved, which shows where each claim comes from. Resolving the underlying disagreement happens in the source tools |
| Recording which source to trust | Knowledge Base Instructions (Beta) record which source wins. Each is tied to its sources and retired automatically when those sources change | Source priority is written into your own ranking code or metadata weights and maintained with the pipeline | Authority can be modeled as entry fields. Applying it at retrieval time is logic in your connected search or agent stack | Source selection is configured in the platform. Finer-grained trust rules depend on what its configuration options expose |
| Routing conflicts back to editors | Conversations and classification scores are saved next to source content, so a reviewer can link a failed answer to the exact documents to fix. Workflows (Beta) moves the fix through review | Outside its scope as a vector database. Feedback loops to content owners are built in your observability and ticketing tools | Editorial workflows exist in the content backend. Linking agent failures to specific entries means connecting your own agent telemetry | Conversation data is available in the platform. Routing a conflict to the owner of the source document runs through your own processes and tools |