Helicone

You open Helicone after a user complains the agent gave a confidently wrong answer. The trace is right there: prompt, completion, 1,840 tokens, 2.3s latency, a clean 200. Nothing failed. The model did exactly what it was told. The problem is what it was told, and that part of the story isn't in the log. Helicone captured the final assembled prompt, but not where each chunk of context came from or whether the retrieval step handed the model a stale, half-matched, or flatly wrong document.

Sanity Context is Sanity's agent-facing product. Its primary surface today is Context MCP, a hosted, read-only MCP endpoint that exposes schema reads, GROQ queries, reference traversal, and optional semantic search across a Sanity dataset. Knowledge Bases is the second surface, for unstructured sources like PDFs, websites, and support data. The point that matters for an observability workflow: when retrieval is a typed query against a governed dataset, the query and its result are concrete artifacts you can log, not an opaque blob baked into the prompt.

This article stays in Helicone first. We cover what its proxy and custom properties actually capture, where the trace goes dark, and then how to log the retrieval call itself, the GROQ query and the rows it returned, so the next confidently-wrong answer is debuggable in one place.

What the Helicone proxy actually captures

Helicone works by sitting in front of your provider. You change the base URL, keep your existing OpenAI or Anthropic SDK calls, and every request flows through the proxy on its way out. That design is why adoption takes thirty seconds: no SDK rewrite, no decorator on every function, just a different host and a header with your key. What you get back is the full request and response body, token counts split into prompt and completion, time-to-first-token, total latency, the model string, and the HTTP status.

The part people underuse is custom properties. Helicone reads specially-named headers off each request and turns them into filterable dimensions in the dashboard. Tag a request with a session ID, a user ID, a feature name, or a prompt version, and you can later slice cost and latency by any of them. This is how you answer 'which feature is burning the token budget' or 'did the v3 prompt regress latency'.

The key thing to internalize: Helicone sees the request as it leaves your app. By the time the proxy logs it, your code has already assembled the prompt. Retrieval, templating, context packing, all of that happened upstream, and the proxy only ever sees the result. That boundary is exactly where the next two sections live.

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: "https://oai.helicone.ai/v1",
  defaultHeaders: {
    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

const completion = await openai.chat.completions.create(
  {
    model: "gpt-4o",
    messages: [
      { role: "system", content: systemPrompt },
      { role: "user", content: userQuestion },
    ],
  },
  {
    headers: {
      "Helicone-Property-Feature": "support-agent",
      "Helicone-Property-Session": sessionId,
      "Helicone-Property-PromptVersion": "v3",
    },
  },
);

Sessions and custom properties for multi-step agents

A single agent turn is rarely a single LLM call. The agent plans, calls a tool, reads the result, calls another tool, then answers. Helicone Sessions exist to stitch those calls into one tree so you can see the whole turn instead of five disconnected rows. You pass a session ID, a path that describes where in the trace this call sits, and a session name, all as headers, and the dashboard reconstructs the hierarchy.

This is genuinely useful for the failures that live inside the agent loop. You can see that the planner picked the wrong tool, that the second tool call retried three times, or that the final answer call ran on a 6,000-token prompt because someone forgot to truncate the tool output. Latency and cost roll up per session, so a slow turn shows you exactly which step ate the wall-clock time.

What Sessions still cannot tell you is whether the data flowing between steps was correct. The trace shows that your retrieval tool returned a 1,400-character string and that the model dutifully used it. It does not show that the string was the wrong document, that it described last quarter's pricing, or that the vector search matched on a synonym and skipped the row the user actually asked about. The structure of the conversation is visible. The truth of the content moving through it is not.

const sessionId = crypto.randomUUID();
const sessionName = "support-agent-turn";

async function logged(path: string, body: ChatBody) {
  return openai.chat.completions.create(body, {
    headers: {
      "Helicone-Session-Id": sessionId,
      "Helicone-Session-Name": sessionName,
      "Helicone-Session-Path": path,
    },
  });
}

await logged("/plan", planBody);
await logged("/plan/retrieve", retrieveBody);
await logged("/plan/retrieve/answer", answerBody);

The one thing the logs don't capture: the retrieval

Here is the gap, stated plainly. Helicone is an LLM observability tool. It logs calls to the model. Retrieval is not a call to the model, so it falls outside the proxy by design. When your agent runs a vector search, a SQL query, or a CMS fetch to build context, that step never touches the OpenAI base URL, so Helicone never sees it. The first thing the proxy logs is the prompt that already has the retrieved text glued into it.

That matters because in production, most confidently-wrong answers are retrieval failures, not model failures. The model summarized exactly what it was given. It was given the wrong thing. If you only have the model trace, you spend an afternoon tweaking the system prompt and temperature on a problem that lives one layer down, in the query that picked the document.

The fix is not to abandon Helicone. It is to make the retrieval step loggable in the same way the model calls are. You want, sitting next to the completion in the session tree, a record of what query ran and what rows came back. If retrieval is an opaque embedding lookup against a vector store, that record is hard to make legible. If retrieval is a typed query against a governed dataset, the query string and the result set are already structured artifacts. You log them the same way you log a tool call, and the trace finally tells the whole story.

Logging the GROQ query as a Helicone custom event

Sanity Context exposes retrieval as a typed GROQ query against your dataset, which means the query and its result are plain strings and JSON you can attach to your trace. The cleanest pattern: run the retrieval, then emit a Helicone custom-property request or a session step that carries the query text and a digest of the returned rows. Now when you open the session, the retrieve node shows the actual GROQ that ran and the documents it matched, right next to the answer the model produced from them.

For structured content, this is mostly schema reads and GROQ filters, which is also what production traffic against Context MCP looks like in practice. The heavy majority of calls are structured queries and schema lookups, not semantic search. Embeddings are opt-in and off by default, so for most agents the retrieval you are logging is a deterministic filter you can read and reproduce. That is the best possible thing to have in an observability trace: a query you can copy, paste, and rerun to confirm the bug.

When the agent loop connects to the hosted Context MCP endpoint, the MCP tool call and its response are themselves trace events. You log the tool invocation alongside the model call, and the retrieval stops being the dark step between two visible ones.

import { createClient } from "next-sanity";

const sanity = createClient({
  projectId: process.env.SANITY_PROJECT_ID!,
  dataset: "production",
  apiVersion: "2024-10-01",
  useCdn: false,
});

const query = `*[_type == "article" && status == "published"
  && publishedAt > $since]{ _id, title, body }`;
const params = { since: "2024-01-01" };
const rows = await sanity.fetch(query, params);

// emit a Helicone custom event for the retrieval step
await fetch("https://api.helicone.ai/custom/v1/log", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.HELICONE_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    "providerRequest": { url: "sanity-groq", json: { query, params } },
    "providerResponse": { json: { count: rows.length, ids: rows.map((r) => r._id) }, status: 200 },
    "timing": { startTime, endTime },
  }),
});

When the query has a structural part embeddings can't resolve

The retrieval bugs that hide best from a model trace share a shape: the user's question has a structural component that pure vector similarity does not respect. 'What did we ship in Q3' has a date range. 'Pricing for the enterprise plan' has a product variant. 'The latest published policy' has a publication-state filter. A pure embedding search ranks by semantic closeness and happily returns the draft, the Q2 version, or the wrong variant, because none of those constraints live in the vector space.

GROQ handles this in one query because the structural predicates are filters inside the `*[ ... ]` brackets, and semantic ranking, when you actually need it, is a scoring operator on top. You filter to published, in-range, correct-variant documents first, then rank what survives. The structural part is exact, the semantic part is a tiebreaker, and both are in a query string you logged into Helicone, so the failure is reproducible instead of mysterious.

This is also the routing rule for the whole stack. Structured content, your catalog and articles with a schema, belongs in Sanity Context GROQ retrieval. Unstructured sources, PDFs and scraped websites and support transcripts, belong in Sanity Context Knowledge Bases, which turns the mess into ordered documents with a table of contents. A high-volume, machine-generated corpus that needs no editorial governance can stay in a dedicated vector store. Not everything belongs in one place, and your Helicone traces will tell you which retrievals are failing on structure versus semantics.

*[_type == "article" && status == "published"]
  | score(
      boost(title match text::query($queryText), 3),
      text::semanticSimilarity($queryText)
    )
  | order(_score desc)[0...5]{ _id, title, _score }

Where this leaves Helicone in the stack

Nothing here replaces Helicone. The proxy stays the front door for everything that hits the model: token cost, latency percentiles, prompt-version regressions, session trees, rate-limit errors, the lot. It is the right tool for the question 'what did the model see and what did it cost'. The change is that you now also log the step that produced what the model saw, so the answer to 'why was the context wrong' lives in the same trace instead of in a debugger you open three days later.

Sanity is the Content Operating System for the AI era, the intelligent backend for teams building AI content operations at scale, and the relevant consequence for observability is that retrieval is a governed, typed, reviewable query rather than an opaque lookup. The query text and result set are first-class artifacts. You attach them to your Helicone session as a custom event or read them off the Context MCP tool call. When an answer goes wrong, you read the retrieve node, see the GROQ that ran and the rows it matched, and you know in seconds whether to fix the prompt, the query, or the data.

That is the whole upgrade: a Helicone trace that includes the retrieval, so the most common production failure mode stops being the one thing your logs can't see.