Tool Studio¶
Tool Studio - counts in-process Spring AI tool callbacks, calls through the built-in MCP server's self-loopback (the default chat path), external tools the built-in server proxies, and calls external MCP clients make to /mcp. Direct calls to an external MCP server appear only in MCP Servers. The Sandbox guard blocks KPI ties this dashboard to the prevention layer in Safety Architecture regardless of where the tool was routed.
Purpose - observe in-process tool execution: the JS-sandbox tools authored in Tool Studio, every Spring AI @Tool method, anything the agent calls without leaving the JVM. Cross-references the sandbox prevention layer through the sandbox.guard.blocked counter.
When to look here¶
- "Did the agent's tool selection finally trigger the expected
searchWikipediacall?" - Top tools horizontal bar. - "Why is my own in-process tool slow?" - Tool latency p50 / p95 / p99.
- "How often is the sandbox actually blocking unsafe actions?" - Sandbox guard blocks counter + chart (zero is healthy; sustained non-zero means agent is repeatedly trying something the sandbox denies).
- "What Risk Level posture is the running tool catalog?" - Sandbox level histogram (L0 / L3 / L4 / L5 distribution).
- "Are tools failing more than they used to?" - Error rate KPI + the Tool calls / minute trend line.
Span filter¶
spring.ai.tool spans that are in-process (mcp.method.name absent), that go through the built-in MCP server's own loopback, or that call an external tool the built-in server proxies (saip.tool.origin=wrapped_external).
A proxied call is one span. It is listed here under its exposed name and on MCP Servers under its upstream server and tool; Overview counts it once. The Proxied calls card says how many of the tool calls are of this kind. Direct calls to an external MCP server appear only on MCP Servers.
External clients. A call an external MCP client makes to the built-in /mcp endpoint, to a Tool Studio tool or to a proxied tool, gets its own spring.ai.tool span tagged saip.via=mcp-server and saip.mcp.client=<client name>, so use of the built-in server from outside the app is counted here as well. The Callers chart splits the tool calls between Agentic Chat and each external client. Such a trace holds only the tool span, so it is not counted as an LLM call in Overview, AI Models or Tokens & Cost.
Denied calls. A tool call that was requested but not run because human-in-the-loop approval was declined, timed out or could not be asked is still recorded as a spring.ai.tool span, tagged saip.hitl.outcome (declined, timeout, ask-failed on the chat side; denied, declined, timeout, elicit-failed for an external client). It counts in Total tool calls, Top tools and Callers, and is shown separately as Denied calls; it is left out of latency, error rate and the sandbox level chart. A denied proxied call never reaches the upstream server, so MCP Servers shows it only on its own Denied calls card, outside the server's call counts. The Safety tab keeps showing the approval decisions by side.
Controls¶
All dashboards share the Observability global settings - time window, refresh interval, custom range. Tool Studio has no tab-specific controls beyond those.
KPI cards (eight)¶
| Card | Shows | Source |
|---|---|---|
| Traces with tools | Number of traces with at least one tool call on this tab (in-process, self-loopback or proxied) | Distinct trace IDs with a qualifying spring.ai.tool span |
| Total tool calls | Count of all qualifying tool spans, denied calls included | Sum of qualifying spans |
| Distinct tools | Number of unique tool names called | set(spring.ai.tool.definition.name) size |
| Proxied calls | Part of Total tool calls that went to an external tool exposed on the built-in MCP server | Spans with saip.tool.origin=wrapped_external |
| Denied calls | Part of Total tool calls that were not run because approval was not granted | Spans with saip.hitl.outcome |
| p95 latency | 95th-percentile duration of executed tool calls | Span duration distribution |
| Error rate | Percentage of executed tool spans with status ERROR (denied calls excluded) |
Span status |
| Sandbox guard blocks | Cumulative sandbox.guard.blocked counter |
MeterRegistry counter incremented by the sandbox at each block |
The Sandbox guard blocks KPI is the concrete bridge to the Safety layer. See Safety Architecture → AI Agent Tool Safety for what triggers a block and Observability Architecture → Foundation for behavioural controls for how this counter ties Sandbox prevention to Observability visibility.
Charts (six)¶
| Chart | Type | Reading |
|---|---|---|
| Tool calls / minute | Stacked bar (top 4 tools), in-process spans/min | Spike → agent in a tool-heavy phase; sustained zero with chat traffic → tools not being selected (check Top tools below) |
| Tool latency p50 / p95 / p99 | Multi-line, ms | Diverging p99 from p50 → tail-latency regression in one tool |
| Top tools | Horizontal bar (top 8 by call count) | The most-called tool head; helps spot accidental tool selection |
| Sandbox level | Histogram across L0 / L3 / L4 / L5 by call | L0 = safest baseline; high L5 traffic warrants per-tool review |
| Sandbox guard blocks | Horizontal bar (lifetime), by category / reason |
Cluster of blocks → agent attempting forbidden action repeatedly (e.g. SSRF probe) |
| Callers | Horizontal bar, Agentic Chat and each external MCP client by call count | A client name here means the built-in MCP server is being used from outside the app |
Cross-references¶
- Safety Architecture - sandbox layers, Risk Level, what triggers a guard block
- Tool Studio (feature) - how in-process tools are authored
- MCP Servers - sibling tab for the same shape of data, externally routed
- Observability Architecture → Tool and MCP observability -
McpToolObservationFilterinjection mechanism
