Add first-class Grok 4.6 model support - #1
Open
hffmnnj wants to merge 6 commits into
Open
Conversation
Discovery via GetUsableModels remains authoritative and unchanged. This fallback entry mirrors Cursor's facing 256K default context (not the xAI direct-API 500K), matching how sibling entries mirror Cursor limits, and keeps grok-code-fast-1 intact. normalizeSingleModel needs no Grok-specific branch: it derives reasoning from thinkingDetails presence and name from displayName/aliases generically, so a discovered grok-4.6 flows through the same path as every other model.
MODEL_COST_TABLE gains an exact grok-4.6 key (input $2 / output $6 / cached-read $0.20) so estimateModelCost resolves it via the exact lookup before the generic /grok/i pattern fallback, which previously routed every grok id to the misleadingly-named grok-4.20 key (a real but different xAI model at $1.25/$2.50 with 1M context). The config and provider model hooks emit the entry generically from Task 2.1's fallback model with limits and reasoning; attachment/image stay false (vision out of scope per SPEC section 5).
Prove MH-4 exposure and routing with deterministic fixtures only. The fallback catalog path exposes grok-4.6 with the Cursor-facing 256K context and reasoning flag while grok-code-fast-1 stays untouched; the v2 catalog and config hooks emit the complete entry with explicit $2/$6 cost and cached-read metadata rather than relying on the generic grok pattern; and both the streaming and non-streaming request paths accept the id and route it to Cursor's Run API through the model-agnostic proxy. The fake Cursor server now records the model id of every Run request so the routing assertion observes the wire instead of proxy internals. No new scenario was needed; the capture is additive to the existing stream handler. Smoke assertion call sites grow from 94 to 127 (+35); no quarantines exist on this branch. Vision flags remain asserted false.
Add a Models section covering Grok 4.6: fallback catalog registration, Cursor-facing 256K default context, Agent + Thinking reasoning, $2/$6 short-context pricing, and text-in/text-out behavior (no image/vision input is forwarded for any model, unlike the xAI direct API).
The GetUsableModels protobuf ModelDetails carries no limit fields, so every live-discovered model previously normalized to the generic 200K/64K defaults even when the fallback catalog knew its real limits. A live grok-4.6 would therefore have been reported with a 200K context, contradicting the Cursor-facing 256K catalog and docs surface. normalizeSingleModel now consults a data-driven FALLBACK_MODEL_BY_ID map: known ids reuse their fallback contextWindow/maxTokens (and fallback name and reasoning only when discovery omits them), while discovery fields such as displayName and thinkingDetails stay authoritative where present. Unknown models keep the generic defaults, so sibling and new models are unaffected and no one-off id branch was introduced. Fallback and proxy routing are unchanged. The smoke harness previously accepted a reasoning flag on synthetic discovered models but never serialized it into the response; it now emits a thinkingDetails object for truthy values (the protobuf expresses thinking support by presence), and the Grok catalog test asserts live grok-4.6 normalizes to 256K/64K/reasoning true, unknown models keep generic defaults, and discovery thinkingDetails overrides fallback reasoning.
xAI's short-context pricing below 200K prompt tokens is $2.00 input / $0.50 cached input / $6.00 output per 1M (docs.grok-4-6-provenance.md row 20, fn_20260812_vaamzb8z). The explicit grok-4.6 entry shipped in 9aa12f2 copied the stale generic grok metadata (cache read $0.20) from the grok-4.20 fallback, which is a different xAI model at $1.25/$2.50. Correct the explicit key to cache read $0.50 and align the two Grok 4.6 deterministic cost assertions (v2 catalog and config hooks). The grok-4.20 generic fallback, all other models, routing, and limits are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
npx tsc -p tsconfig.json --noEmitbun test/smoke.tsbun run buildgit diff f3dc19b..HEAD --checkThis change is stacked on upstream PR ephraimduncan#37.