fix(*): keep migration notices off stdout, and stop planting defaults - #346
Conversation
Review on #341 caught this after it landed. The notice is drained in the `run()` wrapper, which means it prints after every command -- including the two whose stdout is a machine-readable document: `raven doctor --json` and `raven import --json`, both advertised for automation. Reproduced: a config still carrying the retired pin makes `raven doctor --json` emit JSON with a sentence after it, and `jq` refuses the result. stderr is where the rest of this class of message already goes -- the `ConfigReadError` branch a few lines up in the same wrapper, and the loader's malformed-config warning. Moving it there fixes both commands and any future `--json` one without either having to know this exists, which allow-listing the two would not. Narrow but sharp: the migration fires exactly once per install, so the run it can spoil is a first run after upgrading -- and a CI job is precisely the caller that will not be watching. The stale section comment the same review flagged is corrected too: the watermark moved to a sidecar before that PR merged, and the comment still described it living in the config. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
The context-window migration only runs at load, and only once. That leaves no way to ask "does my config still carry something that caps me?" -- by the time anyone wonders, the migration has either already run or decided not to, silently, one launch ago. So `raven doctor` grows a Config section. It reports a `contextWindowTokens` pinned below what the configured model actually holds, and says what rides on it: the per-turn history budget, when the Curator starts paying for a slow path, and when memory consolidation archives. `--fix` removes the pin, so the window follows each model again. Reported without `--fix`, never written. That pin is a real configuration for a local model or an endpoint served smaller than the catalogue thinks; the flag is the user saying which of the two cases theirs is. The write carries the file mode across (config.json holds providers.*.apiKey) and uses a per-PID temp name, the same rules the loader's migration writer follows. One check, not two: a config naming no provider is the other thing worth asking about, but it reads differently on either side of the explicit-provider rule, so it belongs to that change. A model that routes to nothing is already reported above it. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
The bootstrap dumped the whole `Config()` to disk, so a new install started life with eight kilobytes of settings its owner had never heard of -- and, worse, with today's defaults frozen into their file. A default we improve later then never reaches anyone who already has one. That is not a hypothetical cost. `contextWindowTokens: 65536` got onto disk exactly this way, and every upgraded install stayed capped at 64k on a 1M-token model until #341 went and took it back out. The migration was the cure; this is the cause. `save_config` now writes only what differs from the defaults. It stays lossless on reload -- a value equal to its default reloads as that default -- and a fresh config drops from 7,985 bytes to 2, with the seeded extension blocks bringing it to 634. What is left is the handful of lines a person actually chose, which is also a config they can read. One visible consequence, and it is an improvement: the onboarding picker used to lead with the schema's default model, because the bootstrap had written it into the config and the wizard read it back as "the current model" -- a value nobody had chosen, offered as though somebody had. With nothing written, the wizard leads with the provider's own recommendation. The test that pinned the old list now pins the new one and says why. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
|
Claude finished @arelchan's task in 4m 36s —— View job Review complete
Blocking: the network-call-in-a-zero-network-path issue flagged in the earlier review on this PR is still unresolved. The latest commit ( Everything else I checked is solid:
One new nit posted inline (dead/untested branch in the Config-section renderer). |
| model = defaults.model | ||
|
|
||
| if pinned and model: | ||
| real = resolve_context_window(model) |
There was a problem hiding this comment.
[blocker] This breaks the module's own "zero-network" contract for plain raven doctor (see the docstring at the top of this file: "Default mode is zero-network, millisecond-fast"; _gather_static_checks is explicitly "Strictly zero-network").
resolve_context_window(model) defaults to allow_fetch=True, and unlike resolve_max_output_tokens (used a few lines up in _gather_static_checks, which only ever consults LiteLLM's static table), resolve_context_window falls through to _lookup_openrouter_entry → _fetch_openrouter_models, which does a live httpx GET against OpenRouter (10s timeout) whenever the model isn't in LiteLLM's static table and no fresh disk/in-process cache exists.
_inspect_config_health is called unconditionally in register() whenever report.config_loaded is true — not gated behind --probe — so any config with an explicit contextWindowTokens pin now makes plain raven doctor (and raven doctor --json, the automation path this PR is otherwise protecting) attempt a network call. Worse, the case this feature is built for — a self-hosted/local model or custom endpoint "served smaller than the catalogue thinks" — is exactly the kind of model id LiteLLM's static table won't have, so it's the most likely to hit the network fallback. In a sandboxed/no-egress CI runner this adds a multi-second stall to what's documented as a millisecond-fast, offline check before silently degrading.
Every other synchronous/latency-sensitive caller in the repo passes allow_fetch=False for this reason (e.g. raven/agent/loop/main.py:369,738,1048). Suggest the same here: resolve_context_window(model, allow_fetch=False).
There was a problem hiding this comment.
Confirming this independently -- it reproduces, and the failure is not only the theoretical contract break.
With agents.defaults.model = "openrouter/acme/nonexistent-model-xyz" and contextWindowTokens: 32768 in a throwaway home, _inspect_config_health(load_config(cfg), fix=False) with httpx.Client swapped for a recorder:
network calls attempted: [('GET', 'https://openrouter.ai/api/v1/models')]
So plain raven doctor -- no --probe -- reaches _fetch_openrouter_models, and that client is built with timeout=10.0. Offline or behind a firewall, the documented millisecond-fast default mode now blocks for up to ten seconds, and --json is the flag advertised for CI.
Reachable whenever the pin is set and the LiteLLM tier misses, which is exactly the self-hosted / newer-than-the-table case the finding text is written for. Not every model: _lookup_openrouter_entry returns early unless the id starts with openrouter/.
Worth adding for whoever fixes it: allow_fetch=False costs nothing in coverage here. It also turns off the LiteLLM import, but _gather_static_checks has already called resolve_max_output_tokens(defaults.model) by this point, so LiteLLM is in sys.modules and _try_litellm_context_window(allow_import=False) proceeds exactly as it would with True. The only tier lost is the one that should not be running.
CI's diff-coverage gate came back at 79% against a 90% threshold, naming thirteen lines: both failure branches of the `--fix` writer, and the renderer that prints the Config section. The renderer being untested is the one worth calling out. Every assertion so far was on `_inspect_config_health`, which decides what to say -- so the whole block that actually says it could have been deleted with the suite still green, and a finding nothing prints is a finding the user never gets. The new cases drive it both ways: with a fix pending (the section names it and advertises `--fix`) and with one applied (it says what it did and stops advertising). The two error branches are covered as the situations they are: a read-only home leaves the pin in place, says so, and lets the rest of the report print; a `stat` that will not answer does not stop the fix from being written. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
| health = report.config_health | ||
| if health and (health.findings or health.applied): | ||
| console.print("\n[bold]Config[/bold]") | ||
| for line in health.findings: |
There was a problem hiding this comment.
[nit] line.startswith(" ") (and the else branch that strips/re-indents it) has no producer: every finding built by _inspect_config_health — the pin message and f"could not write the fix: {exc}" — is a plain, unindented string, so this branch is dead and untested. Either drop the conditional or wire an actual indented-finding case if one is coming.
gloryfromca
left a comment
There was a problem hiding this comment.
Blocking: the zero-network break already flagged at doctor_commands.py:187 reproduces -- plain raven doctor issues a live GET to openrouter.ai. Repro and a note on why the fix is free are in that thread. Everything else here I would merge as it stands.
Covered: the full diff against origin/main, the callers of what it changes (save_config's only real caller, read_raw_or_raise, the loader's own migration writer, tui_rpc.methods.config), backward compatibility of the on-disk config, AGENTS.md conventions, and the tests for weakening.
What I checked and am satisfied with
save_config(exclude_defaults=True): the blast radius really is one caller.onboard_commands._bootstrap_empty_configis the only one, and only when the file is absent; every other config write goes through the surgicalupdate_*helpers. The round trip is lossless in the way that matters -- the schema's onlymodel_fields_setconsumer isshould_warn_deprecated_memory_window, andmemory_windowisField(exclude=True), so it never survived a dump anyway.tui_rpcreads raw keys and falls back to its own_DEFAULTS, but its keys (agent.temperature,tui.*) are not schema paths, so a thinner file changes nothing there either.- The onboarding picker change is a genuine behaviour change, and the test was repinned with a reason rather than loosened.
anthropic/claude-sonnet-5is the registry's owndefault_modelfor that provider (providers/registry.py:322), so leading with it is the right answer, not an accident of the diff. - The stderr move follows the pattern already in the file (
_helpers.py:311writes the config-path notice to stderr), and the two new cases assert both directions -- stderr carries it, stdout is empty. - Conventions: commits are conventional, ASCII-clean, and carry
Co-authored-bywith a real model id; the branch name fits section 2.1; no new test files, and the touched ones are the right three per section 5.4. - Tests, run here rather than taken on faith:
tests/test_cli_doctor_commands.py tests/test_cli_helpers.py tests/test_config_loader.py tests/test_cli_onboard_commands.py-> 360 passed, 0 skipped. The other four files that touchsave_config(status/agent/gateway/cli_config_precedence) -> 59 passed. Nothing skipped, xfailed, or env-gated.
I also agree with the dead-branch nit at line 582: _inspect_config_health is the only producer of findings, and neither of its two strings is indented, so that else cannot fire.
Two nits of my own inline. And one forward-looking note that is not a finding: a user who legitimately pins a smaller window now gets the ! on every raven doctor run with no way to say "yes, I meant it". Acceptable while the section holds one check, but if it grows, an acknowledge path will be wanted before the section starts being ignored.
| import os as _os | ||
|
|
||
| tmp = path.with_name(f"{path.name}.doctorfix.{_os.getpid()}") | ||
| tmp.write_text(_json.dumps(raw, indent=2, ensure_ascii=False), encoding="utf-8") |
There was a problem hiding this comment.
[nit] Unlike the loader's writer this says it mirrors, this one does not unlink the temp file when the write fails. That one ends except OSError: ... tmp.unlink(missing_ok=True) (loader.py:258-261); here nothing cleans up.
Concrete: on ENOSPC or a quota, tmp.write_text creates the file, fails partway, and the caller's except Exception turns it into a "could not write the fix" finding -- leaving ~/.raven/config.json.doctorfix.<pid> behind holding a partial copy of the config. The mode is the part that stings: chmod runs after the write, so the orphan keeps the default umask (0644) while containing providers.*.apiKey, sitting next to a config.json the user may have tightened to 0600 -- which is the exact exposure the docstring above exists to prevent. A failing os.replace leaves the same thing, fully written.
tmp.unlink(missing_ok=True) in an except OSError: that re-raises closes it, and makes the "same rule the loader's migration writer follows" claim true.
| """ | ||
| Save configuration to file. | ||
|
|
||
| Only what differs from the defaults is written. A dump of everything is |
There was a problem hiding this comment.
[nit] This makes a sibling comment false. _persist_migrations still justifies itself with "save_config is no use here either -- it dumps every default (~8 KB), re-planting the very kind of fossil this migration exists to pull out" (lines 224-226), and after this change it dumps no defaults at all.
The paragraph's other reason still holds and is now the whole justification: the mapping load_config migrated has the extension blocks popped, and a config that needed no edit must stay byte for byte. So this is a one-line trim, not a rethink. Left as is, the next reader takes a present-tense comment as documenting current behaviour and walks away with the wrong picture of what save_config writes -- the same drift the previous commit in this branch fixed for the watermark comment.
Resolving the conflict with the merged #346 by keeping both sides left the two test groups touching. Formatting only. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
Resolving the conflict with the merged #346 by keeping both sides left the two test groups touching. Formatting only. Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
## Summary The model was two attributes on a process-wide `AgentLoop`, so there was one answer for everyone: two sessions could not run different models, a switch in one moved all of them, and the "default" was whatever the last switch happened to write to `agents.defaults`. This makes the model a property of the conversation, and -- following the same thread down -- makes the credential that serves it something the user says rather than something Raven infers. A `ModelBinding` is a model id, the credential that serves it, and how much that model can hold, as one value. `ProviderPool` is the single place deciding which credential a model id pairs with, cached per (vendor, model). `run_turn` resolves the binding for the turn's session and holds it in a `ContextVar` for the whole turn tree; the loop's `provider` / `model` / `context_window_tokens`, the context engine's LLM-backed segments, the skill gate and rewriter, and the consolidator all read that instead of a reference of their own. | Rule | How it lands | |---|---| | Different sessions, different models | A dict of overrides, read once at turn entry | | A switch moves only the session that asked | Nothing else reads that session's entry | | A new session starts on the configured default | A session-scoped switch does not write `agents.defaults` | | A configured subsystem uses its own model+credentials; otherwise it follows the conversation | The factory resolves each pin through the pool, so a holder has either a complete pair or nothing | | A switch mid-turn takes effect next turn | Free -- the turn holds the binding it entered on | Detached work inherits the context copy `asyncio` makes at task creation, so a subagent finishes on the model it was spawned under. ### The window belongs to the binding Sizing came to a head during the rebase. `refresh_context_window` and its cascade assumed the loop has one current model -- which this PR makes false, since two sessions can be on a 200k and a 1M model at once, and one int on the loop cannot answer for both. That is the same mis-sizing #341 removed, arriving through another door. So the window moved onto `ModelBinding`, beside the credential, and every holder reads the binding of the turn it is running under. `refresh_context_window` is gone: a turn now enters on a binding that already knows its own size, so there is nothing to refresh. It resolves on first read rather than at construction, because building a provider is what imports LiteLLM -- a lazily built one has not done it yet, so an eager window is the catalogue-miss default for every model, which is the defect `on_built` existed to paper over. A miss is deliberately not cached, so a cold read cannot make the process wrong for its lifetime. An explicit `contextWindowTokens` travels on every binding, so a pinned number survives whatever model a session switches to. ### The provider is a word the user says A model id does not name whose credential serves it. `openrouter` serving `anthropic/claude-haiku-4-5` and `anthropic` serving `claude-haiku-4-5` are both real, name different keys, bill different accounts, and look identical on the wire. Everything that used to fill in the blank was guessing which account pays. `agents.defaults.provider: "auto"` was the same guess at the config layer, and it did not detect anything: a prefixed id was answered by the provider it names, but a bare id fell to keyword matching in `PROVIDERS` order and took the first configured claimant -- so with anthropic and openrouter both keyed, `gpt-4.1` went to openrouter over openai because openrouter sits third in that list and openai eighth. - `/model <id>` is refused, and the refusal names the two ways out: `/model` for the picker, or `/model <provider> <id>`. A prefixed id is refused too -- a prefix is LiteLLM routing syntax, not evidence about a credential. - `config.set model` refuses without a provider, which is the boundary where the rule can be enforced rather than asked politely. - `raven provider use` requires `--provider`, and names it alongside `raven provider list` in the error. - `--default` is not an exception: changing what new sessions start on is the same choice about the same thing. - `agents.defaults.provider` carries no default, and a one-shot migration resolves each pre-rule config through the old derivation and writes the answer into it. Behaviour is unchanged by construction -- the value written is what that config was already getting -- but it stops being inferred. It reuses the sidecar watermark from #341, with a floor per migration rather than one shared mark, so raising the generation for this one does not re-open the previous one. A config the derivation cannot answer for is left blank rather than filled with a vendor chosen to have something there. The picker already asked in this order -- provider, then model, with a free-text row for an id no catalogue lists -- so it needed nothing. Both help strings now carry the usage, because a rule the user meets first as an error is a rule we chose not to tell them. With nothing left deriving, `providers/pin.py` is deleted: its whole subject was "which provider to pin when the user changed the model without naming one", and its last caller was the onboarding wizard, which asks for the provider before the model and therefore always had one in hand. ### Config defaults No subsystem ships a vendor default. `context.curator_model`, `token_wise.tool_result_lifecycle.summary_model` and `skill_forge.detect_model` all hardcoded the same Gemini id, and `token_wise.smart_routing.tiers` shipped six models across three vendors -- for users who may hold no key for any of them. All are unset now, which is what "not configured" has to mean for the subsystem rule to be expressible. `context.curator_model` and `skill_forge.llm_gate_model` took a model id and nothing else, so the pool had to guess which credential served it. Each pin now takes a provider alongside the model (`curator_provider`, `llm_gate_provider`). With the provider named, a vendor whose credentials are unusable is logged and dropped rather than silently borrowing the conversation's key. With it unset the pin still binds: a configured gateway takes it, because a gateway serves whatever id it is handed under its own credential, and only without one is the vendor derived from the id. That branch is unchanged from main and is documented as it behaves rather than as the rule would prefer -- an earlier revision of this description and of `CONTEXT.md` claimed the drop applied there too, which was wrong. **Release note.** With `context.curator_model` unset, the Curator's slow path runs on the conversation's model instead of failing on a Gemini id nobody had a key for and dropping to the deterministic plan. That is the rule working as asked, but on a long conversation it is up to 12 tool-calling requests per turn of context housekeeping that previously cost nothing. Set `context.curator_model` (and `curator_provider`) to a small model to keep it cheap. ### Review findings **Round 1 (pre-rebase), five findings.** A pin naming a vendor with no spec crashed the agent at startup -- fixed on main, which now passes the provider name separately instead of reading it off a spec that may be None; verified with the reporter's own repro. `/model` before the first message no longer manufactures a zero-message session record (`session.create` is lazy, and `session.title` guards the identical case the same way). Every rpc call in the TUI's session commands has a `.catch` -- without it, `/model` typed before the first `session.create` resolves wrote a raw stderr line over the Ink render while the transcript showed nothing. The remaining two are the picker's provider row and the gateway pin; see below. **Round 2, three before-merge findings, all reproduced here before being fixed.** - The migration watermark was one high-water mark shared by every stamped migration, so raising it to 2 for the provider migration re-ran the context-window one on every config already stamped at 1 -- deleting a `contextWindowTokens: 65536` a user had put back by hand after our own notice invited them to, and printing that invitation again. Each migration now has its own floor. The test that should have caught it built its precondition from `CURRENT_CONFIG_VERSION`, so it moved with the bump; it now pins the literal a shipped build wrote, and two tests cover the upgrade-from-an-older-generation path in both directions. - Deleting `_adopt_provider` took its `_image_tool_result_ok.clear()` with it. Both capability caches key on a model id but are computed from the provider, so an `apiBase` repointed at a box with different capabilities, or a re-authenticated provider, kept the old endpoint's verdict for the life of the process -- images silently dropped from tool results with nothing in the log. Both binding setters now clear them. - `--default` had no working path through the TUI. The picker footer taught `/model <name> --default`, which is the bare-id spelling the parser refuses; the refusal dropped the flag from the suggestion it told the user to follow; and `/model --default` opened the picker, whose selection sent a session-scoped switch that looked like it had changed the default. The flag now rides into the overlay state and back out through the picker's callback, the refusal keeps it, and the footer writes the spelling that works. **The two carried from round 1 are fixed here rather than dissolved.** An earlier revision of this description said the work had dissolved them; it had not, and both were still reproducible. - `model.options` re-derived the provider from the id. For a vendor with a spec that now reads the prefix `stored_model_id` writes, but `find_by_model` answers None for a passthrough vendor (mistral, xai), so the picker starred `agents.defaults.provider` for a session running on someone else's key -- and the starred row is exactly what a user reads to answer whose key is paying. It now reads the head of the stored id, and stars nothing when that names no configured provider. - The gateway pin is documented as it behaves; see the Config defaults section above. **Round 3, after the rebase onto a main that gained #346.** The conflict was additive and is described in the thread; two follow-on commits closed the two nonblocking notes that survived it. `modelPicker.test.tsx` was rendering `<ModelPicker>` without the `scope` prop this branch made required, so the `scope === 'default'` footer had no coverage -- and `tsconfig.json` excludes `src/__tests__`, which is why `tsc --noEmit` was clean and silent about it. And `onModelSelect`, the callback that carries `--default` back out of the picker, had no test; the rule it applies is now its own function, `modelSelectCommand`, because what a selection becomes on the command line has to satisfy three separate constraints in `parseModelArg` at once. **Also taken, each a defect in what this PR itself added or removed:** `clear_session_binding` was gated on `SessionManager.delete` returning True, which it does not for a session that switched model before its first save, so the entry leaked for the life of the process; `_migrate_auto_provider` validated its probe against a strict `Config` before the shims that relocate legacy top-level blocks, so a config still carrying `skillRouter` failed the probe, was skipped silently, and was stamped anyway -- never retried, on exactly the oldest configs most likely to still say `auto`; `providers/lazy.py` and `AgentLoop.set_provider` carried docstrings naming methods and callers this PR deletes; onboarding still wrote `provider="auto"` when clearing a removed provider's default; and `ui-tui/package-lock.json` is reverted to main, its only change being npm-version drift in `libc` metadata. ## Type - [ ] Fix - [x] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` pytest tests/ 6700 passed, 33 skipped, 13 deselected (TERM=dumb) ruff check / format --check clean, 832 files formatted npx tsc --noEmit (ui-tui) clean npm test (ui-tui, vitest run) 998 passed (86 files) npm run lint:rpc generated.ts in sync commitlint + check_commit_messages.py clean over github/main..HEAD ``` `tests/test_cli_theme.py::test_bold_accent_renders_styled_not_bare` is `TERM`-dependent, not ordering-dependent: it fails under an interactive `TERM` and passes under `TERM=dumb`, which is what the `coverage` target pins and what CI runs. That file is not touched by this PR. Every fix in rounds 2 and 3 was mutation-checked rather than assumed: reverting the migration floor, the cache clear, the flag pass-through, the passthrough-vendor fallback, the picker's `scope` prop and the `--default` command-building each turns the corresponding new test red. Driven by hand in an isolated home against the built TUI bundle, not only by tests: `/model claude-opus-4-5` refused with the two ways out; `/model openrouter/anthropic/claude-opus-4-5` refused as well; `/model openrouter claude-opus-4-5` applied; `/model` opening the picker at the provider level. A real config carrying `provider: auto` and a bare `claude-opus-4-5` migrated to `provider: openrouter` -- the vendor it was already resolving to -- with the notice printed and the watermark written to the sidecar. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ## Risk User-visible behaviour changes, all deliberate: `/model <id>`, `config.set model` without a provider, and `raven provider use` without `--provider` now fail where they used to guess. That is the point, but it is a workflow change for anyone who typed a bare id -- the errors name the fix, and the picker needs nothing new learned. `agents.defaults.provider` is written into every config that did not have one, on the first launch after this lands. The value is what that config was already resolving to, so no request changes vendor; what changes is that it is now visible and editable. With `curator_model` unset out of the box, the Curator's slow path starts actually running (see the release note above). Cost, not correctness. One contract tightening is not backward compatible on its own: `config.set key="model"` now rejects a call with no `provider`, and the `-32009 model_switch_in_turn` code is removed from the error table. The in-repo TUI moves with it, but any client outside this repo that sends a bare model id starts failing on the first request after this lands. Rollback: revert the commits. The config keeps an explicit `provider`, which every published build accepts and treats exactly as it treats a hand-written one. The `config.migrations.json` sidecar is left behind and every build since #341 does read it, so a reverted build sees `{"version": 2}` and treats the config as fully migrated -- which is correct, since it is. Delete the sidecar to force the migrations to be reconsidered. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A --------- Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com> Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
## Summary Six follow-ups from #284's review rounds. Five are defects in what that PR itself added or removed and were deliberately deferred rather than rushed into a diff already under review; the sixth is a check #346 wrote down as belonging here, because it only becomes a real state once the provider is explicit. **A session's model now survives a restart on every surface, not only in the TUI.** The read half of the persistence hung off `session.resume`, so only the surface that calls that handler got it: a conversation arriving from a channel came back on the configured default with its own choice sitting unread in its own session record. The read moves onto the loop, where every entry point already goes, and the handler's duplicate copy is gone -- one reader instead of two. `has_session_binding` restores first as well, or a session that switched before a restart reports as having inherited the default and the picker stars the wrong row. `clear_session_binding` counts as having consulted the record, so returning a session to the default is not undone by the next read. **`raven provider use` refuses an unroutable provider before it writes.** A typo exited 0 with the config already changed, then printed a credential warning suggesting `raven provider set <typo> --api-key ...` -- advice that would have created a section for a vendor nothing routes to. `provider set` already applied the right test before writing a section; it is exposed as `ensure_routable_provider` and called first, which is the order the TUI's `/model` already used. Only an unroutable *name* is refused: a known provider with no credentials yet is still written and reported, because picking a model before configuring its key is a normal order to do things in. "No spec of ours" cannot be the typo test either -- Raven carries none for mistral or xai, and refusing on it would block every passthrough vendor the config documents as supported. **The guard that keeps a model and its provider paired has a home again.** A whole-repo AST scan failing any `set_default_model` without `provider=`, and any raw `agents.defaults.model` write not paired with a provider write, lived in `tests/test_provider_pin.py` and was deleted with that file. Only half its message was obsolete; the rule it enforces became more central, and it is the only place a machine checks it. `test_provider_resolution_invariants.py` is where the other source guards of this kind already live. **The binding cache is keyed on everything it caches.** The fingerprint covered `config.providers`, but `bind` also copies `contextWindowTokens` onto the binding, so editing that number left a stale binding answering with the old one until a restart -- the "I changed it and nothing happened" shape the window ladder exists to prevent, arriving through the cache instead. **A gateway call is priced from the gateway's own table.** Who routes a request is not who bills for it. LiteLLM answered first for every id, including one naming a gateway, and the two disagree: measured on the pinned LiteLLM, `openrouter/z-ai/glm-4.6` is filed at 0.40/1.75 per million where OpenRouter's own current table says 0.50/2.00, so every such call was reported a fifth under its real cost. OpenRouter's catalogue now answers ahead of LiteLLM for ids that name it, fresh-only and never fetching: an expired copy is not better evidence than the router's table and would suppress the refetch the expiry exists to trigger, while a blocking fetch would be paid by the turn this runs after. Unchanged for a direct id, which is what once priced a self-hosted deployment at a hosted model's rate. **`raven doctor` can ask about a provider nothing resolved.** #346 deferred this with its reason: before the explicit-provider rule, the field defaulted to `auto` and blank never happened. Now blank means the load-time migration tried the derivation and had no answer. Two findings -- blank, and a name nothing routes to -- both report-only. Neither is offered to `--fix`, and that is not an omission: only the user knows which vendor they meant to pay, and the migration has already tried. Guessing again under the name `--fix` would be the same guess wearing more confidence. ## Type - [x] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [ ] Other ## Verification ``` pytest tests/ 6477 passed, 43 skipped, 13 deselected (TERM=dumb) ruff check / format --check clean, 832 files formatted npx tsc --noEmit (ui-tui) clean npm test (ui-tui) 998 passed (87 files) commitlint + check_commit_messages.py clean over github/main..HEAD npm run lint:rpc generated.ts in sync commitlint + check_commit_messages.py clean over github/main..HEAD ``` Every fix is mutation-checked rather than assumed -- reverting it turns the new test red. That is worth naming for two of them, where writing the test taught something the fix alone did not: - The doctor check's scope is narrower than it looks. With a configured vendor that serves the model, the migration fills the blank during `load_config` and there is nothing left to report, so the fixture has to be genuinely unresolvable -- which is exactly when the finding is real. - The first attempt at the pricing tier read the any-age cache and made a stale disk copy outrank a refetch. `test_expired_disk_triggers_refetch` caught it, which is why the tier is fresh-only. Three review rounds landed on this PR after it opened, each fixed and each verified rather than taken from the report: - **Blocking:** the doctor provider check read a leftover `auto` as an unroutable vendor name. That is the legacy state the check exists for -- `_migrate_auto_provider` leaves the literal in place when it cannot resolve one -- so the user was told to fix a typo they did not make. It now matches `Config._match_provider`'s own `forced != "auto"` test, and says the word is retired rather than merely ignoring it. Neither neighbouring test used a literal `auto`, which is how the branch between them went unentered. - `_restore_attempted.add` in `clear_session_binding` was inert, and its stated reason did not hold for the only caller: `session.delete` unlinks the record before the clear runs. Dropped rather than relocated -- marking in `set_session_binding` would claim the record had been consulted when it had not. The test that was supposed to guard it is renamed and its docstring now scopes what it does and does not pin. - `_routes_anywhere` caught `Exception` where the sibling check catches `KeyError`, and the new refusal printed a different failure marker from the eleven others in its file. - [x] Relevant tests pass locally - [x] Relevant lint / type checks pass locally - [ ] User-facing docs or screenshots are updated when needed ## Risk Two user-visible changes, both deliberate: `raven provider use` now exits 1 on a provider name nothing routes to, where it used to exit 0 with the config already written. A script passing a misspelled `--provider` starts failing; a script passing a real one is unaffected, including for a passthrough vendor. Cost figures change for calls made through OpenRouter on ids where its catalogue and LiteLLM's copy disagree. The new number is the one the gateway bills, so this corrects a systematic under-report rather than introducing a new figure -- but a user comparing today's total to yesterday's will see the step. Rollback: revert the commits. Nothing here writes a new field or changes a stored shape, so a revert needs no migration. - [x] Security impact considered - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes ## Related Issues N/A -- follow-ups from the review of #284, which is merged. --------- Co-authored-by: arelchan <204152633+arelchan@users.noreply.github.com> Co-authored-by: Claude (claude-opus-5) <noreply@anthropic.com>
Summary
Three follow-ups from #341, which fixed a retired
contextWindowTokens: 65536capping every upgraded install at 64k. Two are its loose ends; the third is the reason it happened at all.The notice no longer prints on stdout. #341 drains config-migration notices in the
run()wrapper so every command tells the user, which caught the two whose stdout is a machine-readable document:raven doctor --jsonandraven import --json, both advertised for automation. Reproduced -- a config still carrying the retired pin makesdoctor --jsonemit JSON with a sentence after it, andjqrefuses the result. Moved to stderr, where the rest of this class of message already goes (theConfigReadErrorbranch a few lines up in the same wrapper, and the loader's malformed-config warning). That fixes any future--jsoncommand without it having to know this exists, which allow-listing the two would not. The window is narrow -- the migration fires once per install -- but the run it can spoil is a first run after upgrading, and a CI job is exactly the caller not watching.raven doctorgains a Config section and a--fix. The premise changed while building it: the migrations run at load, so by the time this command looks, nothing is pending -- they already happened, silently, one launch ago. What is left for a person to ask about is what Raven deliberately will not decide for them. So it reports acontextWindowTokenspinned below what the configured model holds, and names what rides on it: the per-turn history budget, when the Curator starts paying for a slow path, and when memory consolidation archives.--fixremoves the pin.Reported without
--fix, never written: that pin is a real configuration for a local model or an endpoint served smaller than the catalogue thinks, and the flag is the user saying which case theirs is. The write carries the file mode across (config.json holdsproviders.*.apiKey) and uses a per-PID temp name -- the same rules the loader's migration writer follows.One check rather than two. A config naming no provider is the other thing worth asking about, but it reads differently on either side of the explicit-provider rule in #284, so it belongs to that change; a model that routes to nothing is already reported above it.
The bootstrap stops writing every default into the user's config. This is the cause of the bug #341 cured.
save_configdumped the wholeConfig(), so a new install started life with eight kilobytes of settings its owner had never heard of and, worse, with that day's defaults frozen into their file -- after which a default we improve never reaches anyone who already has one.contextWindowTokens: 65536got onto disk exactly this way.It now writes only what differs from the defaults. Lossless on reload, since a value equal to its default reloads as that default. A fresh config drops from 7,985 bytes to 2, and to 634 once the extension blocks are seeded -- what is left is the handful of lines someone actually chose, which is also a config they can read.
That third one is the only change here with a broad blast radius, and it has one visible consequence, which is an improvement: the onboarding picker used to lead with the schema's default model, because the bootstrap had written it into the config and the wizard read it back as "the current model" -- a value nobody had chosen, offered as though somebody had. With nothing written, the wizard leads with the provider's own recommendation instead. The test that pinned the old list now pins the new one and says why.
Type
Verification
Driven by hand as well as by tests, each against a throwaway home:
New cases cover the stderr split, both doctor paths including the mode preservation, and the save round-trip keeping every value the user chose.
Risk
The bootstrap change alters what every new install's config file looks like: nearly empty instead of a full dump. Existing configs are untouched -- nothing rewrites them -- and both shapes load identically, so the risk is confined to anyone reading
config.jsonas documentation of what can be set.raven doctorand the schema remain the answer to that question.The onboarding picker's default moves from the schema default to the provider's recommendation, as described above.
doctor --fixwrites only when asked, and only removes a key. Everything else here is output routing.Rollback: revert the commits. Nothing persists a new format, so a reverted build reads every config written under this one.
Related Issues
N/A