Account for Anthropic thinking tokens in usage details#554
Open
PratikDhanave wants to merge 4 commits into
Open
Account for Anthropic thinking tokens in usage details#554PratikDhanave wants to merge 4 commits into
PratikDhanave wants to merge 4 commits into
Conversation
assistantUsageUpdate mapped input, output, total, and cache-read tokens but dropped AssistantUsageData.ReasoningTokens, so the framework's dedicated ReasoningTokenCount field stayed zero for reasoning models. ReasoningTokens is a subset of the output tokens, so TotalTokenCount is unaffected; only the breakdown field was missing. Map ReasoningTokens to UsageDetails.ReasoningTokenCount, matching the OpenAI and Gemini providers.
toUsageDetails mapped input, output, total, and cache tokens but dropped Usage.OutputTokensDetails.ThinkingTokens (the tokens the model generated as internal reasoning), so the framework's dedicated ReasoningTokenCount field stayed zero for extended-thinking requests. toUsageDetailsDelta also reconstructed the Usage without OutputTokensDetails, so the streaming path could not recover it either. Map ThinkingTokens to UsageDetails.ReasoningTokenCount and carry OutputTokensDetails through the delta path, matching the OpenAI, Gemini, and Copilot providers. Thinking tokens are a subset of output tokens, so TotalTokenCount is unchanged.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves token-usage accounting by ensuring “reasoning/thinking” tokens emitted by providers are surfaced via UsageDetails.ReasoningTokenCount, including in Anthropic streaming deltas, so telemetry/OTel breakdowns are accurate.
Changes:
- Anthropic: map
Usage.OutputTokensDetails.ThinkingTokensintoUsageDetails.ReasoningTokenCount. - Anthropic streaming: preserve
OutputTokensDetailswhen reconstructinganthropic.UsagefromMessageDeltaUsage. - Copilot: surface reasoning tokens from
assistant.usageevents and add a regression test; add an Anthropic non-streaming regression test.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| provider/copilotprovider/copilot.go | Adds ReasoningTokenCount mapping in Copilot usage updates. |
| provider/copilotprovider/copilot_test.go | Adds coverage asserting Copilot usage events expose reasoning tokens. |
| provider/anthropicprovider/agent.go | Maps Anthropic thinking tokens into ReasoningTokenCount and carries details through streaming delta reconstruction. |
| provider/anthropicprovider/agent_test.go | Adds a non-streaming Anthropic regression test for thinking tokens. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
210
to
214
| OutputTokens: usage.OutputTokens, | ||
| CacheCreationInputTokens: usage.CacheCreationInputTokens, | ||
| CacheReadInputTokens: usage.CacheReadInputTokens, | ||
| OutputTokensDetails: usage.OutputTokensDetails, | ||
| }) |
Contributor
Author
There was a problem hiding this comment.
Added TestUsageReasoningTokens_Streaming, which covers the toUsageDetailsDelta path (thinking tokens on the streamed message_delta → ReasoningTokenCount); red on the old code, green with the fix.
Covers the toUsageDetailsDelta path: thinking tokens arriving on the streamed message_delta usage must surface as ReasoningTokenCount.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Anthropic SDK exposes
Usage.OutputTokensDetails.ThinkingTokens— "the number of output tokens the model generated as internal reasoning."toUsageDetailsmapped input, output, total, and both cache fields but never read it:For any extended-thinking request (this provider explicitly supports it — it emits
ThinkingBlock/ThinkingDeltaasTextReasoningContent), those tokens were silently dropped: the framework’s dedicatedUsageDetails.ReasoningTokenCount(populated by the OpenAI, Gemini, and Copilot providers, and surfaced as an OTel telemetry attribute) stayed0.toUsageDetailsDeltaadditionally rebuilt theUsagefrom only four scalar fields, omittingOutputTokensDetails, so the streaming path could never recover it.Fix
toUsageDetails:ReasoningTokenCount: usage.OutputTokensDetails.ThinkingTokens.toUsageDetailsDelta: carryOutputTokensDetailsthrough when reconstructing theUsage.Thinking tokens are a subset of output tokens, so
TotalTokenCountis unchanged — only the missing breakdown field is added, matching the OpenAI providers.Test
TestUsageReasoningTokensreturns a non-streaming response withoutput_tokens_details.thinking_tokens: 35and assertsReasoningTokenCount == 35. It fails on the old code (0) and passes with the fix.