Skip to content

Stop pinning temperature, which the current model generation refuses - #698

Merged
xmap merged 1 commit into
mainfrom
fix/unpin-sampling-temperature
Aug 20, 2026
Merged

Stop pinning temperature, which the current model generation refuses#698
xmap merged 1 commit into
mainfrom
fix/unpin-sampling-temperature

Conversation

@xmap

@xmap xmap commented Aug 20, 2026

Copy link
Copy Markdown
Owner

#694 (mine) pinned temperature = 0.0 in all three chat-request builders. The reasoning was that these tasks pick from a fixed verdict set under a provider-enforced schema, so sampling variance buys nothing. That reasoning holds. The parameter does not.

The measurement

Anthropic has deprecated temperature. Measured directly against the Argo gateway on 2026-08-20, one call per model:

Model temperature: 0.0
Sonnet 4.5, Haiku 4.5, Sonnet 4.6, Opus 4.6 accepted
Opus 4.7, Sonnet 5, Opus 5 400 `temperature` is deprecated for this model

So the pin does not steady a modern model's answer, it prevents one.

Why it looked fine

Every agent shipped today declares a model on the accepting side of that line: RunDebriefer defaults to Haiku 4.5, CautionDrafter to Sonnet 4.6. Nothing in place broke. What the pin did was silently foreclose every model upgrade, and fail illegibly when one was attempted: the refusal arrives as an invalid-request error, the debrief records DebriefDeferred, and nothing in the deferral names sampling as the cause.

Found by pointing a new RunDebriefer at Sonnet 5 on the 2-BM rehearsal and watching the walk preflight refuse on call one. It would not have been found by the test suite, which stubs the provider.

What is and is not withdrawn

Kept: the port's temperature/top_p fields, both adapters sending only what a caller set, and the trace writing it to entries_decision_inferences. The provenance work stands.

Withdrawn: the judgement that these three tasks should set a value. A caller that sets nothing records nothing, which is the honest state and was the design principle #694 argued for in the first place.

A consequence worth naming

What can be recorded about how a model was asked is bounded by what the vendor still permits to be specified, and that surface is contracting as sampling knobs give way to adaptive reasoning. Recording sampling faithfully does not arrest that. The JSR paper's reproducibility section is being updated to say so.

Gates

  • pytest tests/unit tests/architecture: 45199 passed, 631 skipped
  • pre-commit: pyright, tach, ruff, architecture fitness all pass

The one test asserting the pinned value now asserts request_temperature is None, with the reason in a comment rather than the bare change.

🤖 Generated with Claude Code

#694 pinned temperature to 0.0 in all three chat-request builders, on
the reasoning that these tasks pick from a fixed verdict set so sampling
variance buys nothing. That reasoning holds. The parameter does not.

Anthropic has deprecated temperature, and newer models reject a request
carrying it. Measured against the Argo gateway on 2026-08-20:

    Sonnet 4.5, Haiku 4.5, Sonnet 4.6, Opus 4.6   accept it
    Opus 4.7, Sonnet 5, Opus 5                    400, "`temperature`
                                                  is deprecated for
                                                  this model"

So the pin does not steady a modern model's answer, it prevents one.
Every agent shipped today declares a model on the accepting side of that
line, so nothing in place broke; what the pin did was silently foreclose
every upgrade, and fail in the least legible way when one was attempted.
The refusal arrives as an invalid-request error, the debrief defers, and
nothing in the deferral names sampling. It was found by pointing a new
RunDebriefer at Sonnet 5 and watching the preflight refuse on call one.

The port keeps both dials and the record still writes whatever a caller
sets, so the provenance work stands. What is withdrawn is the judgement
that these tasks should set one. A caller that sets nothing records
nothing, which is the honest state, and honesty is the point of that
column.

There is a wider consequence worth stating for anyone reasoning about
replay: what can be recorded about how a model was asked is bounded by
what the vendor still permits to be specified, and that surface is
contracting as sampling knobs give way to adaptive reasoning. Recording
sampling faithfully does not arrest that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/agent/prompts
  caution_drafter.py
  run_debrief.py
  apps/api/src/cora/operation/adapters
  _llm_decide_prompt.py
Project Total  

This report was generated by python-coverage-comment-action

@xmap
xmap merged commit aca8966 into main Aug 20, 2026
19 checks passed
@xmap
xmap deleted the fix/unpin-sampling-temperature branch August 20, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant