Anthropic auto-instrumentation plugin drops tool_use output in streaming aggregation

[braintrust-bot]

Summary

The Anthropic auto-instrumentation plugin's aggregateAnthropicStreamChunks function only handles text_delta events when aggregating streaming chunks. It does not handle input_json_delta events (used for streaming tool_use blocks), content_block_start, or content_block_stop. This means when a streaming Anthropic call includes tool use, the aggregated span output contains only the text portion — tool call names, IDs, and input JSON are silently dropped.

The wrapper (wrapAnthropic) correctly handles input_json_delta and tracks tool_use blocks through content_block_start/content_block_stop, so this is a parity gap between the two instrumentation paths.

What is missing

Plugin aggregation (js/src/instrumentation/plugins/anthropic-plugin.ts, lines 138–198)

aggregateAnthropicStreamChunks processes these event types:

• message_start — extracts initial metrics ✅
• content_block_delta — only handles text_delta (line 161), ignores input_json_delta ❌
• message_delta — extracts final metrics/metadata ✅

Missing entirely:

• content_block_start — needed to know block type (text vs tool_use) and capture tool name/ID
• content_block_stop — needed to finalize tool_use blocks
• input_json_delta handling — needed to accumulate tool input JSON

Wrapper comparison (js/src/wrappers/anthropic.ts, streamNextProxy)

The wrapper correctly handles the full streaming lifecycle:

• content_block_start: saves block type and initializes delta accumulator (line 330–335)
• content_block_delta: handles both text_delta (line 343) and input_json_delta (line 348)
• content_block_stop: assembles tool_use output with parsed JSON input (lines 361–374)

Impact

Users relying on auto-instrumentation (via --import braintrust/hook.mjs) with Anthropic streaming tool_use will see spans with incomplete output — only text content, no tool calls. Users using the wrapper (wrapAnthropic) see full tool call details.

Upstream reference

• Anthropic streaming format: https://docs.anthropic.com/en/api/messages-streaming
• Tool use with streaming: content_block_start → content_block_delta (with input_json_delta) → content_block_stop
• @anthropic-ai/sdk: https://github.com/anthropics/anthropic-sdk-typescript

Braintrust docs status

Anthropic tool use is documented as supported (supported), but the docs don't distinguish between wrapper and auto-instrumentation fidelity.

• https://www.braintrust.dev/docs/reference/wrappers/anthropic

Local files inspected

• js/src/instrumentation/plugins/anthropic-plugin.ts (lines 138–198: aggregateAnthropicStreamChunks)
• js/src/wrappers/anthropic.ts (lines 286–398: streamNextProxy)
• js/src/vendor-sdk-types/anthropic.ts (lines 102–122: AnthropicStreamEvent — correctly models input_json_delta)