Slack

Your Slack knowledge bot demos beautifully. A user @-mentions it, it retrieves an internal doc, it answers in-thread. Then someone in Legal @-mentions the same bot in a public channel, asks about an unreleased acquisition, and the bot cheerfully posts the answer where forty people can read it. The problem is not the model. The bot ran its retrieval as its own bot token, not the asking human's identity, so it fetched a document the asker's colleagues were never entitled to see, then broadcast it to the whole thread.

Sanity Context is Sanity's agent-facing product. Its primary surface today is Context MCP, a hosted, read-only MCP endpoint that exposes schema reads, GROQ queries, reference traversal, and optional semantic search across a Sanity dataset. A second surface, Knowledge Bases, turns unstructured sources like PDFs, websites, and support databases into well-ordered documents. Neither one is where the leak gets fixed on its own. The leak gets fixed by who the retrieval runs as.

This article stays on the Slack side first: which entry points hand you the asking user's identity, why chat.postEphemeral exists, and how Slack's own AI security model draws the boundary. Then it shows where Sanity Context slots in as the governed, per-user content source once the identity is flowing.

The leak lives in the gap between the bot token and the asking user

Start with the actual mechanism, because it is not obvious until it bites you. A Slack channel is multi-user. Your bot authenticates with its own bot token, a single identity that is a member of every channel it was invited to. When app.event('app_mention') fires, the payload hands you event.user (the human who @-mentioned you), event.channel, and event.ts. But nothing forces you to use event.user. The lazy path is to take event.text, run retrieval as the service account, and post the answer back. That path leaks.

Here is why. The retrieval ran under the bot's permissions, which are the union of everything the bot can reach, not the intersection with what the asking user can reach. So the bot can pull a document event.user was never entitled to open. Then chat.postMessage puts that answer in the thread, visible to every member of the channel. Two failures stack: the wrong identity did the fetch, and the result went to the wrong audience.

Slack states its own AI security model as first principles, and rule three is the one you just broke: Slack AI only operates on the data that the user can already see. Their enterprise search and Agentforce integrations enforce this as participant-aware access control, meaning an agent can only touch messages and files visible to the requesting user. That is the bar. A bot that retrieves as itself and answers in-channel fails it on both axes. The fix is not a smarter prompt or a bigger model. It is plumbing: carry event.user through the whole call, and choose an output surface that respects who asked.

import { App } from '@slack/bolt';

const app = new App({
  token: process.env.SLACK_BOT_TOKEN,
  signingSecret: process.env.SLACK_SIGNING_SECRET,
});

app.event('app_mention', async ({ event, client }) => {
  // event.user is the asking human. This handler throws it away.
  const answer = await retrieveAsServiceAccount(event.text);

  // Posted to the whole thread. Everyone in the channel sees it.
  await client.chat.postMessage({
    channel: event.channel,
    thread_ts: event.thread_ts ?? event.ts,
    text: answer,
  });
});

chat.postEphemeral and scoped DMs decide who sees the answer

Fix the output surface first, because it is the cheaper of the two fixes and it caps the blast radius while you work on retrieval. Slack ships chat.postEphemeral for exactly this. It posts a message visible only to a single user in a channel, and Slack's docs call it the recommended default for command acknowledgments, inline errors, confirmations, and 'here is what I am about to do' prompts. Nobody else in the thread sees an ephemeral message. If your bot must answer in a public channel, an ephemeral reply to event.user is the difference between one person reading a confidential answer and forty people reading it.

There is a second, cleaner surface: move the whole conversation into a DM. app.message with a channel_type === 'im' filter handles direct messages, where there is exactly one recipient and no audience problem to reason about. For a knowledge bot that regularly returns sensitive material, a slash command that acknowledges ephemerally and then continues in a DM is a defensible default. app.command handlers expose command.user_id and command.channel_id, so you have the asking identity at the entry point there too.

Note what this fix does and does not solve. Ephemeral output stops the broadcast. It does not stop the wrong identity from doing the fetch. If your retrieval still runs as the bot token, an ephemeral message can still show event.user a document they personally were never entitled to see. You have narrowed the audience to one, but that one might still be the wrong one. Output scoping and identity scoping are two separate controls, and you need both.

app.event('app_mention', async ({ event, client }) => {
  const answer = await retrieveForUser(event.user, event.text);

  // Visible only to event.user, even in a busy public channel.
  await client.chat.postEphemeral({
    channel: event.channel,
    user: event.user,
    thread_ts: event.thread_ts ?? event.ts,
    text: answer,
  });
});

// Or move sensitive Q&A into a DM entirely.
app.message(async ({ message, client }) => {
  if (message.channel_type !== 'im') return;
  const answer = await retrieveForUser(message.user, message.text);
  await client.chat.postMessage({ channel: message.channel, text: answer });
});

Forward the asking user's identity into the retrieval call

This is the load-bearing fix, and it is a plumbing job, not a security research project. The rule to internalize: the agent's reach should be the user's reach. In a demo, your bot runs as a service account because a demo has no users. In production it answers on behalf of a specific logged-in human, and that difference is the entire security posture. Every Slack entry point already hands you that identity: app_mention gives you event.user, slash commands give you command.user_id, actions and shortcuts give you body.user.id. The mistake is discarding it before the retrieval call.

So don't discard it. Map the Slack user id to your identity system, mint or look up that user's token, and pass it down through the agent runtime into the tool layer and onward to your content API. When the retrieval call runs under the user's permissions rather than the bot's, three things fall out for free. Personalized retrieval, so the agent sees exactly what the user can see and nothing more. Personalized action, so it can only do what the user could do. Traceable audit, so the fetch is logged against the human, not the model. As the Sanity auth-forwarding guidance puts it: you don't build 'AI security' as a separate discipline, you make sure the token flows.

The side benefit is that the agent inherits your existing security model whole. Same row-level permissions. Same rate limits. Same regulatory boundaries. You are not reimplementing access control for the AI path; you are reusing the one you already trust for human requests. That is what turns a leaking channel bot into one that satisfies Slack's own participant-aware bar.

app.event('app_mention', async ({ event, client }) => {
  // 1. Resolve the asking human to your identity system.
  const userToken = await resolveUserToken(event.user);

  // 2. Run retrieval UNDER that user, not the bot service account.
  const answer = await runAgent({
    question: event.text,
    // The token flows: Slack -> app -> agent runtime -> tool -> API.
    // The content API call executes with the user's permissions.
    onBehalfOf: userToken,
  });

  // 3. Answer only the asker.
  await client.chat.postEphemeral({
    channel: event.channel,
    user: event.user,
    thread_ts: event.thread_ts ?? event.ts,
    text: answer,
  });
});

Structured retrieval is where the answer usually comes from

Once identity is flowing, the next question is what the agent actually queries, and here Slack teams tend to over-index on vectors. The reflex is to embed every internal doc and run a similarity search. That is one ingredient, not a retrieval strategy. Semantic search is a small slice. Embeddings are opt-in, off by default, and most projects never turn them on.

That is not because embeddings are bad. It is because most retrieval failures are structural, and pure vector search cannot resolve them. The failure has a recognizable shape: the question carries a real structural component, a department, a document status, a date range, an owning team, and vector similarity does not respect any of those constraints. The query comes back empty or wrong, and the model either hallucinates or hedges. The fix is not a better model. The model needed to know the shape of the data, not just its meaning.

Sanity Context exposes this over Context MCP, a hosted read-only endpoint your agent loop connects to for schema reads, GROQ queries, and reference traversal across the dataset. Read-only matters here: an agent querying internal knowledge via MCP can read but cannot mutate, so the surface area for accidental writes is zero. Writes, when you need them, go through Agent Actions, a separate path. For a Slack knowledge bot, that means the MCP tool the agent calls is inherently incapable of changing your content while it answers a question. Structured predicates plus fresh content plus a read-only boundary covers most of what a knowledge bot needs before you reach for anything fancier.

When structured retrieval is not enough: hybrid and Knowledge Bases

Reach for semantic ranking when structured retrieval demonstrably runs out, not before. The pattern is: predicates enforce the constraints that must hold, and a score pipeline blends a keyword match with semantic similarity for the ranking within that filtered set. In GROQ, using the documented text search operators, hybrid retrieval looks like the query below. Read it carefully, because the operators are exact. text::query($queryText) is the BM25 keyword match, wrapped in boost() to weight title hits. text::semanticSimilarity($queryText) takes the query text itself, not an embedding field. score() combines them into _score, and order(_score desc) ranks the result.

What the predicates inside the brackets do is the important part for a Slack knowledge bot: they filter down to the documents that structurally qualify before anything gets ranked. That is where your category, price, or warehouse constraints live in the example, and by extension where the structural component of an internal query would live. The score pipeline never sees a document the predicates excluded.

Structured content is only half of internal knowledge, though. The other half is messy: onboarding PDFs, an internal wiki, exported support threads. That is the shape Knowledge Bases is built for, the second surface of Sanity Context. It turns those unstructured sources, Sanity datasets, support databases, websites, and PDFs, into well-ordered documents with a clear table of contents, so an agent can answer against them faster and more accurately than raw-chunking the same files. The routing rule is simple: GROQ retrieval for structured content, Knowledge Bases for unstructured. The agent picks the strategy based on the question.

import { defineQuery } from "groq"

export const PRODUCT_SEARCH_QUERY = defineQuery(`*[
  _type == "product"
  && category == $category
  && price < $maxPrice
  && stockLocation == $warehouse
]
| score(
    boost([title] match text::query($queryText), 2),
    text::semanticSimilarity($queryText)
  )
| order(_score desc)
[0...10] {
  _id,
  title,
  price,
  "stock": stockLocation->{ name, available }
}`)

Govern the bot's instructions and never-say list as versioned content

The last leak vector is not retrieval at all, it is the bot's own behavior. Right now your Slack bot's system prompt is almost certainly a string in the codebase. Marketing cannot read it. Compliance cannot review the forbidden-topics list. The support lead cannot update the escalation language without a pull request and a deploy. When Legal asks you to add 'never discuss the acquisition' to the bot, that becomes an engineering ticket, and the window between the ask and the deploy is a window where the bot can still leak.

The better shape is to store the agent's instructions, approved answers, and never-say list as structured documents with fields owned by different teams. Brand owns voice. Product owns how the agent uses user context. Support owns escalation. Compliance owns the forbidden-topics list. Splitting the prompt into fields is not cosmetic, it is access control: none of those teams files a pull request, and none waits for a deploy. In Sanity that content lives in the Studio, so you get real-time collaboration, version history, attribution, and rollback for free.

Content Releases stages agent behavior the same way you stage a website: draft the change, preview it, gate it behind an eval, schedule it, and keep the audit trail. That is the same governance you already trust for your public site, applied to what your Slack bot is allowed to say. This is where Sanity fits the larger picture as the Content Operating System for the AI era, the intelligent backend that keeps AI workflows governed, reviewable, and safe inside the editorial loop. For an internal knowledge bot, that means a confidential-topics change is a content edit with an audit log and a rollback, not a hotfix under pressure. Sanity is SOC 2 Type II compliant, GDPR-aligned, and offers regional hosting with a published sub-processor list, which is the paperwork your security review will ask for.