How to Separate Public vs Internal Content for AI Use Cases

Your support agent quotes an unreleased pricing tier to a customer, or your internal HR chatbot surfaces a severance policy to someone browsing the public help center. Both failures come from the same root cause: an AI system retrieving content it was never supposed to see for that audience. When public and internal knowledge live in the same undifferentiated pile, every agent inherits the worst-case blast radius. One bad retrieval leaks a roadmap, an unredacted incident postmortem, or a customer's PII into a channel that should never have touched it.

Sanity Context (previously Agent Context) is the AI Content Operating System, an intelligent backend that keeps this boundary enforceable at the content layer instead of hoping a prompt will hold the line. The instinct most teams reach for is filtering at the model: a system prompt that says "don't reveal internal information," or a post-hoc classifier. That is governance by vibes, and it fails the moment a query is phrased cleverly enough.

This article reframes the public-versus-internal problem as an access and modeling problem, not a prompting problem. The boundary belongs in how content is shaped, stored, and queried, so that a public agent literally cannot retrieve internal documents, and an internal agent gets the fuller picture by design.

Why prompt-level filtering is the wrong boundary

The most common pattern is also the most brittle: teams pile every document into one index, then instruct the model to withhold sensitive material at generation time. This concedes the whole game. Retrieval-augmented generation works by pulling candidate passages into the context window before the model reasons about them, which means an internal document has already been fetched, embedded in the prompt, and exposed to the inference boundary before any instruction to "keep it internal" is evaluated. A jailbreak, an oblique paraphrase, or a summarization request that never names the sensitive term can all coax the content back out.

The deeper problem is that a prompt is a request, not a control. It carries no cryptographic or structural guarantee. Security teams reviewing an AI feature will ask a simple question: can a public user cause an internal record to enter the model's context at all? If the answer depends on the model's good behavior, the answer is effectively no, you cannot prove it won't leak. This is the difference between deny-by-configuration and deny-by-hope.

Modeling your business, the first pillar, means the public/internal distinction is a property of the content itself, expressed in the schema, not an afterthought bolted onto the agent. In Sanity, a document's audience, visibility, and classification are structured fields you define once. Retrieval then filters on those fields before a passage is ever a candidate, so an internal record for a public agent is not redacted after the fact. It is never retrieved in the first place, which is the only boundary a security reviewer can actually verify.

Model the boundary as structured fields, not folders

Folder-based or space-based separation, the Notion-workspace or Confluence-space model, gets you part of the way: internal lives in one container, public in another. But it breaks down the moment a single source document has mixed sensitivity. A product page is public, but its internal launch notes, competitive positioning, and unreleased SKUs are not. Splitting that into two documents duplicates the truth and guarantees drift, where the public version and internal version disagree within a sprint.

Structured content lets you model the boundary at the granularity where it actually lives. In the Content Lake, an audience field, a visibility enum, or a classification tag are first-class attributes on the document, and even on individual blocks of Portable Text. A single product document can carry public marketing copy and internal-only fields side by side, with retrieval deciding per query which fields are eligible. You model your business once, and the same record serves both a public support agent and an internal sales enablement agent with different projections.

This is the concrete difference between a Content Operating System and a store that happens to hold documents. The shape of the content encodes the policy. GROQ projections let a public retrieval path select only the audience == "public" fields, while an internal path selects the full record, from the same source of truth. There is no second copy to keep in sync, and no classifier guessing at sensitivity after the fact, because the sensitivity was declared by the person who authored the content, in the schema, at write time.

Two datasets, one retrieval path, zero leakage

The cleanest enforcement for hard boundaries is physical separation combined with a shared retrieval mechanism. Sanity Context supports distinct datasets, so genuinely internal content, HR policy, security runbooks, unreleased financials, can live in a dataset that a public-facing agent's credentials simply cannot query. The public agent connects to the Sanity Context MCP endpoint scoped to the public dataset. The internal agent connects with credentials scoped to both. The boundary is enforced by access control at the API, not by the model's discretion.

Because dataset embeddings are tied to the content in each dataset, you do not maintain a separate vector pipeline per audience or worry that a stale index still holds an internal passage after it was reclassified. When content moves or its classification changes, the embeddings propagate within minutes, so the retrieval surface reflects the current policy rather than last week's. There is no orphaned vector floating in a shared index waiting to be matched by a public query.

The payoff is that both agents share the same hybrid retrieval quality. Inside each dataset, a single GROQ query blends semantic and lexical signals: text::semanticSimilarity() for meaning, a BM25 match() for exact terms, combined with score() and boost() to rank. The public agent gets excellent retrieval over public content; the internal agent gets excellent retrieval over the union; and neither is a degraded experience. Separation does not mean the public agent is running a worse system. It means it is running the same system over a smaller, safe surface.

Govern agent instructions the way you govern the website

The public/internal line is not only about which documents an agent can read. It is also about what the agent is told to do, and who approves that. An instruction like "you may discuss pricing tiers" or "never disclose the incident timeline" is itself a piece of content with a sensitivity and a review process. Most teams bury these instructions in application code or an environment variable, where a single engineer can change agent behavior with no editorial review and no audit trail.

In Sanity, agent instructions live in Studio as governed content, and Content Releases let you stage a change to an agent's behavior the same way you stage a change to the website. A revised public-agent guardrail can be drafted, reviewed by legal or compliance, and released on a schedule, with the previous version recoverable. Roles & Permissions decide who can even edit an internal-agent instruction versus a public-agent one, and Audit logs record who changed what and when.

This is the automate-everything pillar applied to governance rather than throughput. The agent's behavior is versioned, reviewable, and reversible, which is exactly what an enterprise security review expects before a customer-facing AI ships. When an auditor asks "who approved letting the public agent discuss X, and when," the answer is a Content Release with a named reviewer, not a Slack message and a deploy. The boundary between public and internal governance becomes an artifact you can show, not a claim you have to defend.

Enterprise controls: residency, compliance, and audit

A public/internal separation is only as trustworthy as the platform enforcing it. Enterprise buyers evaluating an AI content system will ask about certifications and data handling before they ask about retrieval quality, because a leak of internal content is a compliance event, not just an embarrassment. Sanity is SOC 2 Type II compliant and GDPR-aligned, offers regional hosting and data residency options, and publishes its sub-processor list so security teams can complete their own vendor review.

Data residency matters specifically for the internal side of the boundary. Internal content, employee records, legal documents, unreleased financials, frequently carries geographic constraints that public marketing copy does not. Being able to host the internal dataset in a required region while the public dataset serves globally is a modeling decision the platform should support, not a limitation you have to engineer around.

Audit logs close the loop on the whole architecture. When retrieval is filtered structurally, embeddings track content, agent instructions are versioned in Content Releases, and access is scoped by dataset and Roles & Permissions, the audit trail is continuous from authoring to retrieval to agent behavior. That is the difference between a system where you assert public and internal are separated and a system where you can demonstrate it. Legacy CMSes stop at publishing and leave the AI boundary to a downstream tool; a Content Operating System operates the content end to end, which is precisely where the audit trail needs to be unbroken.

A decision framework for drawing the line

Not every internal/public distinction needs the same enforcement, and over-engineering the boundary is its own failure. Use a tiered model. For content where a leak is catastrophic, HR, legal, security, financials, use separate datasets with credential-scoped access, so a public agent physically cannot query it. For content that is mostly public but has internal-only facets, model those facets as fields on the shared document and project them out per audience with GROQ. For content that is public but should not be surfaced yet, use Content Releases to stage it until launch.

Start by inventorying your content by blast radius, not by department. Ask, for each source, what is the worst outcome if a public agent retrieved this passage. That question sorts your material into the tiers above faster than an org chart will. Then decide enforcement per tier: physical dataset separation for the catastrophic tier, field-level projection for the mixed tier, release-gating for the timing tier.

The reason to consolidate all three tiers on one Content Operating System rather than stitching a vector database, a CMS, and a governance tool together is that the boundary has to hold across every layer at once. When retrieval, embeddings, classification, agent instructions, and access control all live in the same intelligent backend, the public/internal line is enforced by construction rather than by integration glue that can drift. The failure mode this article opened with, a public agent quoting internal content, happens at the seams between systems. Removing the seams is the durable fix, and it is what building on a single Content Lake with Sanity Context is for.