fix(streaming): get_final_message() drops container, breaking code execution continuation - #1776
Open
epatey wants to merge 1 commit into
Open
fix(streaming): get_final_message() drops container, breaking code execution continuation#1776epatey wants to merge 1 commit into
epatey wants to merge 1 commit into
Conversation
…ecution continuation When streaming with client.messages.stream(), the final message's container field is always None, even though the API sends it in the message_delta event. The streaming code copies stop_reason, stop_sequence, stop_details, and usage from that event, but not container. The equivalent code for client.beta.messages.stream() does copy it. Without the container id, a caller can't continue a conversation that has unfinished code execution -- the next request fails with: 'container_id is required when there are pending tool uses generated by code execution with tools.' This adds the missing copy, plus sync and async tests that fail without it. Fixes anthropics#1424
epatey
marked this pull request as ready for review
July 23, 2026 16:47
5 tasks
epatey
added a commit
to UKGovernmentBEIS/inspect_ai
that referenced
this pull request
Jul 23, 2026
Three fixes for the 400 'container_id is required when there are pending tool uses generated by code execution with tools': - Replay pending server tool spans: a span whose results never arrived (the turn ended first, e.g. a client tool call cut in) had no content item to anchor it and was silently dropped from the assistant replay. Spans that never produced content now replay verbatim; spans whose content was deliberately deleted by a scaffold still drop. - Replay the container id across turns: record each response's container id (keyed by assistant message id, incl. dump/restore and head/tail continuation merge) and pass it as the 'container' request param when the last assistant turn left container work pending. - Capture the container under streaming: the SDK's non-beta stream accumulator drops message_delta.container (fix proposed in anthropics/anthropic-sdk-python#1776), so capture it from the raw event in _capture_compaction_from_stream. This also un-breaks the existing same-turn pause_turn continuation, which reads head_message.container.
dragonstyle
pushed a commit
to UKGovernmentBEIS/inspect_ai
that referenced
this pull request
Jul 27, 2026
…alls (container id not replayed) (#4603) * Add failing test: pending code execution lost after client tool call When a turn mixes code-execution server tool use with a client tool call, the response ends with stop_reason 'tool_use' while the code execution is still pending, and carries a container id. The follow-up request must replay the pending server_tool_use block and include the container id as the 'container' param. inspect does neither: the pending block is dropped from the assistant replay, and the container id is only threaded through same-turn pause_turn continuations, never across turns. The API rejects such follow-ups with 'container_id is required when there are pending tool uses generated by code execution with tools.' (The id is also lost under streaming by an SDK bug -- the non-beta stream accumulator drops message_delta.container; fix proposed in anthropics/anthropic-sdk-python#1776. This test uses the non-streaming path, where the response includes the id, to isolate inspect's gaps.) * Fix Anthropic container continuation after client tool calls Three fixes for the 400 'container_id is required when there are pending tool uses generated by code execution with tools': - Replay pending server tool spans: a span whose results never arrived (the turn ended first, e.g. a client tool call cut in) had no content item to anchor it and was silently dropped from the assistant replay. Spans that never produced content now replay verbatim; spans whose content was deliberately deleted by a scaffold still drop. - Replay the container id across turns: record each response's container id (keyed by assistant message id, incl. dump/restore and head/tail continuation merge) and pass it as the 'container' request param when the last assistant turn left container work pending. - Capture the container under streaming: the SDK's non-beta stream accumulator drops message_delta.container (fix proposed in anthropics/anthropic-sdk-python#1776), so capture it from the raw event in _capture_compaction_from_stream. This also un-breaks the existing same-turn pause_turn continuation, which reads head_message.container. * Mark SDK stream-container workaround with removal pointer * Deduplicate SDK workaround comment * Handle server tool results that arrive a turn after their use block The live test surfaced a fourth gap: when a turn ends with a server tool call still running, its result arrives at the start of the NEXT response, where the parser raised 'CodeExecutionToolResultBlock without previous ServerToolUseBlock' (pending_tool_uses only covers the current response). Resolve orphaned results against the use block recorded in the prior turn's span, and record only the result in the new message's span (the use block replays with its own message, so it isn't duplicated). Also adds a slow live test driving the full shape end-to-end (warmup exec to create the container, then a parallel exec + client tool call leaving pending work, then the tool-result follow-up), and moves the CHANGELOG entry to the Bugfix convention (prefix + bottom of the Unreleased list). * Live test observes only the public API Replace the private _pending_container_for_input skip-gate with a public-surface check: the mixed turn is an assistant message making a client tool call whose content lacks the sleep's output, and successful resumption is that output surfacing in a later assistant message's code execution content. This also lets the test run against unfixed code. Verified red/green live: with the orphaned-result fix reverted (fixes 1-3 present) the test fails with 'CodeExecutionToolResultBlock without previous ServerToolUseBlock'; with the full fix it passes. * Live test detects silent loss via container-state seed The previous public-surface check (sleep output appearing in a later message) could not distinguish true resumption from silent loss: with the pending call dropped from the replay, the model re-runs the command in a fresh container -- and if it re-runs the warmup too, produces the output anyway. Now the warmup plants a random seed file AND prints the seed, and the pending command echoes it back. Only the original container can produce marker-<planted seed>: after silent loss a re-run yields a bare 'marker-' (no seed file) or a different seed (re-warmed), so the test fails instead of passing vacuously or skipping. Verified live both ways: FAILS against the pre-fix provider (with the SDK #1776 workaround applied, inert here since the test runs non-streaming); PASSES at HEAD with the marker carrying the exact planted seed. * Address review: verify resolved-span replay on later turns, document limits - extend unit + live tests to a turn after the pending work resolves (use block replays with its message, result with its own, no container) - document why the prior span deliberately stays open after add_prior_turn_result (history truncation re-pends the work) - document bridge limitation for containers (no bridge-surviving key exists for a pending span) - note pending-span replay position is unanchored by design - trim call-site comments that restated the docstring
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When streaming with
client.messages.stream(), the final message'scontainerfield is alwaysNone, even though the API sends it in themessage_deltaevent. The streaming code copiesstop_reason,stop_sequence,stop_details, and usage from that event, but notcontainer. The equivalent code forclient.beta.messages.stream()does copy it.Without the container id, a caller can't continue a conversation that has unfinished code execution — the next request fails with
400: container_id is required when there are pending tool uses generated by code execution with tools.This adds the missing copy, plus sync and async tests that fail without it.
Fixes #1424.