Every message creates a run, and every run is fully recorded by the runtime itself. There is no tracing SDK to wire in and no sampling: the same trace you use in the builder's test chat is the one you open when a production user reports a wrong answer.
LLM observability is the ability to see exactly what an AI application did on every request: the prompt that was sent, the tokens and cost, the tools that were called with their inputs and outputs, the documents retrieved, and where time or money went. Without it, an agent that answers wrongly is a black box. With it, every failure has a step, a prompt and a cause.
Runs: status, timing, input and output, token usage (actual when the provider reports it, estimated otherwise), cost, the workflow version and the checkpoint reference for resuming. Steps: one per node, with a context report of what the node kept, what it trimmed and why, and how long it took. Edges: what moved from one node to the next.
Generations: each model call with provider and model, the exact messages sent (system prompt, history, memory and knowledge context, tool results), the raw request settings, the response, finish reason, latency, time to first token, tokens and cost. Tool calls: input, output, status, duration, retries, errors, and what the model was shown of the result. Retrievals: query, candidates, scores, selected chunks and trims. Memory reads and writes: items considered, selected and extracted.
Open the run from Logs or Runs, find the step where the answer went wrong, and open its generation to read the exact prompt the model received. If a tool was not used, the toolbox step shows which tools were bound for that call and why others were excluded. If the knowledge was wrong, the retrieval shows the chunks and scores. If a tool failed, the error is on the call and the model's next turn shows what it did with it.
Because nested app runs, drafts in the test chat and production runs all produce the same trace, the fix is usually a settings change you can verify in the test chat before publishing.
Tokens and cost appear per generation and in the run total. Analytics summarises agents, tools, conversations and costs, and Usage reports daily and monthly consumption per app. The run detail also shows how much the conversation occupies as stored, split into logs, checkpoints and shared retrieved text, so storage growth is never a surprise.
No. The runtime records runs, steps, generations, tool calls, retrievals and memory activity itself. Observability is a property of the platform, not an add-on.
Yes. When an app routes to an agent, the agent's run is recorded with the same detail as the parent and linked to it, so orchestration never hides a step.
Yes. Each run carries its tokens and cost, runs can be filtered by app, agent, status and time, and the conversation's storage footprint is shown on the run detail.
Design AI agents as executable workflow graphs: typed nodes, conditional edges, validation shared with the runtime, drafts and one-click publishing.
Connect AI agents to HTTP APIs, webhooks, web pages and other agents with typed tools: JSON schemas, auth types, safety levels, limits and call logs.
See how this feature shows up in a real agent app.
Build GitHub AI agents: fetch issues and pull requests through the REST API, browse docs with the browser tool, and trace every debugging step.
Join the waitlist for early access, or read how the rest of the platform fits together.
