Stop pinning temperature, which the current model generation refuses - #698
Merged
Conversation
#694 pinned temperature to 0.0 in all three chat-request builders, on the reasoning that these tasks pick from a fixed verdict set so sampling variance buys nothing. That reasoning holds. The parameter does not. Anthropic has deprecated temperature, and newer models reject a request carrying it. Measured against the Argo gateway on 2026-08-20: Sonnet 4.5, Haiku 4.5, Sonnet 4.6, Opus 4.6 accept it Opus 4.7, Sonnet 5, Opus 5 400, "`temperature` is deprecated for this model" So the pin does not steady a modern model's answer, it prevents one. Every agent shipped today declares a model on the accepting side of that line, so nothing in place broke; what the pin did was silently foreclose every upgrade, and fail in the least legible way when one was attempted. The refusal arrives as an invalid-request error, the debrief defers, and nothing in the deferral names sampling. It was found by pointing a new RunDebriefer at Sonnet 5 and watching the preflight refuse on call one. The port keeps both dials and the record still writes whatever a caller sets, so the provenance work stands. What is withdrawn is the judgement that these tasks should set one. A caller that sets nothing records nothing, which is the honest state, and honesty is the point of that column. There is a wider consequence worth stating for anyone reasoning about replay: what can be recorded about how a model was asked is bounded by what the vendor still permits to be specified, and that surface is contracting as sampling knobs give way to adaptive reasoning. Recording sampling faithfully does not arrest that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Coverage reportClick to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#694 (mine) pinned
temperature = 0.0in all three chat-request builders. The reasoning was that these tasks pick from a fixed verdict set under a provider-enforced schema, so sampling variance buys nothing. That reasoning holds. The parameter does not.The measurement
Anthropic has deprecated
temperature. Measured directly against the Argo gateway on 2026-08-20, one call per model:temperature: 0.0`temperature` is deprecated for this modelSo the pin does not steady a modern model's answer, it prevents one.
Why it looked fine
Every agent shipped today declares a model on the accepting side of that line: RunDebriefer defaults to Haiku 4.5, CautionDrafter to Sonnet 4.6. Nothing in place broke. What the pin did was silently foreclose every model upgrade, and fail illegibly when one was attempted: the refusal arrives as an invalid-request error, the debrief records
DebriefDeferred, and nothing in the deferral names sampling as the cause.Found by pointing a new RunDebriefer at Sonnet 5 on the 2-BM rehearsal and watching the walk preflight refuse on call one. It would not have been found by the test suite, which stubs the provider.
What is and is not withdrawn
Kept: the port's
temperature/top_pfields, both adapters sending only what a caller set, and the trace writing it toentries_decision_inferences. The provenance work stands.Withdrawn: the judgement that these three tasks should set a value. A caller that sets nothing records nothing, which is the honest state and was the design principle #694 argued for in the first place.
A consequence worth naming
What can be recorded about how a model was asked is bounded by what the vendor still permits to be specified, and that surface is contracting as sampling knobs give way to adaptive reasoning. Recording sampling faithfully does not arrest that. The JSR paper's reproducibility section is being updated to say so.
Gates
pytest tests/unit tests/architecture: 45199 passed, 631 skippedThe one test asserting the pinned value now asserts
request_temperature is None, with the reason in a comment rather than the bare change.🤖 Generated with Claude Code