feat: callModel per-turn stall timeouts (DEV-723 2/5)

[LukasParke]

Stack layer 2/5 — callModel integration

Base: #770 (watchdog core). Rest of stack: #774 (typed server-failure errors) → #775 (stall retries) → #776 (raw-stream helpers + docs). Linear: DEV-723.

Changes

Wires the phase-1 watchdog into callModel with an opt-in per-call option:

client.callModel({
  model, input,
  timeout: { firstContentMs: 15_000, contentIntervalMs: 30_000 },
})

• timeout is a client-only field on CallModelInput — stripped from the outgoing API request in both resolution paths (resolveAsyncFunctions, resolveRequestForContext).
• ModelResult.createTurnWatchContext() gives each turn its own AbortController; on stall the watchdog aborts the turn's HTTP request (tests assert request.signal.aborted === true and body teardown).
• Signal merging via combineSignals: caller signal + replicated timeoutMs behavior + watchdog — configuring the watchdog never disables the existing timeoutMs, and a user abort still surfaces as their abort, not a stall.
• All three responsesSend sites (initial request, follow-up tool turns, approval-resume) re-arm deadlines per turn — matching vercel/ai's per-step firstChunkMs semantics.
• ToolEventBroadcaster.complete() made idempotent (first call wins): the pipe's error completion must not be overwritten by the later unconditional .finally() completion, or concurrent consumers would see a clean close instead of the stall error.
• StreamStalledError, StreamStallPhase, StreamTimeoutOptions exported from the package root via the sdk.ts custom region.

Tests

8 integration tests driving the real transport path (scripted SSE bytes → HTTPClient → responsesSend → EventStream → ModelResult): stall before content, keepalive-comment immunity, healthy streams, default-off, mid-generation stall, multi-consumer rejection, per-turn re-arm across a tool loop (turn 1 passes, turn 2 stalls), caller abort signal merging.

Verification

Lint / typecheck / build clean; 222 unit tests passing (8 new).

[devin-ai-integration]

Devin Review found 1 potential issue.

View 1 additional finding in Devin Review.

[devin-ai-integration]

🟡 Overall request time limit is no longer refreshed for retry attempts once stall detection is enabled

The per-attempt request time limit is replaced by a single shared countdown started once (AbortSignal.timeout(timeoutMs) at src/lib/model-result.ts:980-981) instead of being restarted for each retry, so a retried request can be cut off almost immediately.
Impact: Users who enable stall detection and rely on automatic retries can see requests fail with a timeout even though each individual attempt was well within the allowed time.

Why the replicated timeout diverges from the generated retry loop

ClientSDK._createRequest only records context.timeoutMs when no signal was supplied (src/lib/sdks.ts:230-232), and ClientSDK._do then creates a fresh AbortSignal.timeout(timeoutMs) inside the retry callback, i.e. once per attempt (src/lib/sdks.ts:282-289).

createTurnWatchContext always supplies a signal when the watchdog is active, so context.timeoutMs stays unset and _do's per-attempt timer never runs. The replicated AbortSignal.timeout(timeoutMs) is created a single time, before the request is issued, and is merged into the signal used for every retry attempt of that turn. With the default retry config (backoff, retry on 5XX) a turn that gets a 502 after most of the budget has elapsed will have its retry aborted with a timeout error, whereas without timeout configured the retry would have received a full fresh budget.

A secondary, smaller divergence: responsesSend computes the effective value as options?.timeoutMs || client._options.timeoutMs || -1 (src/funcs/responsesSend.ts:282) while the new code uses ??, so an explicit timeoutMs: 0 in the caller's options no longer falls back to the client-level value.

Prompt for agents

In ModelResult.createTurnWatchContext (src/lib/model-result.ts), the code replicates the SDK's timeoutMs behaviour by creating a single AbortSignal.timeout up front and merging it into the request signal. However, the generated client applies its timeout per retry attempt: ClientSDK._createRequest only sets context.timeoutMs when no signal is present (src/lib/sdks.ts:230-232) and ClientSDK._do constructs a fresh AbortSignal.timeout inside the retry callback (src/lib/sdks.ts:282-289). Because the watchdog always supplies a signal, the per-attempt timer is disabled and the replicated one becomes a cumulative deadline spanning all retries of the turn, which can abort a retry that would otherwise have had a full budget. Consider a mechanism that preserves per-attempt semantics — e.g. passing the turn abort signal through a path that still lets _do arm its own per-attempt timeout, or merging the turn signal inside a request hook rather than as options.signal. Also note the effective-value computation uses ?? whereas responsesSend uses || (src/funcs/responsesSend.ts:282), which differs when timeoutMs is 0.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[perry-the-pr-reviewer]

Perry's Review

Verdict: 💬 Comments / questions

Well-structured wiring of the phase-1 watchdog into callModel. Per-turn abort re-arming, signal merging, and the ToolEventBroadcaster.complete() idempotency fix are all correct. The timeout field is properly stripped from the outgoing API request in both resolution paths. Three comments below — the timeout-retry interaction is the one I'd like to see addressed or explicitly acknowledged before this lands.

Full review

What's good

• Per-turn abort re-arming: Each of the three responsesSend sites (initial, follow-up, approval-resume) gets its own AbortController via createTurnWatchContext(). Deadlines are correctly independent per turn — a stall in turn N+1 isn't masked by a generous firstContentMs from turn 1.
• Signal merging: combineSignals(turnAbort.signal, callerSignal, timeoutSignal) correctly preserves the caller's abort (surfaces as the caller's reason, not a stall) and replicates timeoutMs behavior so enabling the watchdog never silently disables the existing timeout.
• Broadcaster idempotency: Making ToolEventBroadcaster.complete() first-call-wins is the right fix — without it, the .finally() completion in startTurnBroadcasterExecution would overwrite the stall error recorded by the pipe's .catch() handler, and late-joining consumers would see a clean close instead of the error.
• Field stripping: timeout is correctly stripped from the API request in both resolveAsyncFunctions (clientOnlyFields set) and resolveRequestForContext (destructuring timeout: _t).
• Tests: 8 integration tests driving the real transport path (scripted SSE → HTTPClient → responsesSend → EventStream → ModelResult) — stall before content, keepalive immunity, healthy streams, default-off, mid-generation stall, multi-consumer rejection, per-turn re-arm, caller-abort merging. Solid coverage of the core scenarios.

Comments

1. Per-attempt timeout becomes cumulative across retries (see inline comment on model-result.ts:979). The SDK's _do method normally creates a fresh AbortSignal.timeout(timeoutMs) inside the retry callback — each retry attempt gets a full timeout budget. When the watchdog is active, createTurnWatchContext supplies options.signal, which causes _createRequest to skip setting context.timeoutMs, disabling _do's per-attempt timer entirely. The replicated AbortSignal.timeout(timeoutMs) created in createTurnWatchContext is a single timer started before the first request — it counts down across ALL retry attempts of the turn. A request that gets a 502 after most of the budget has elapsed will have its retry cut off by the cumulative timeout, whereas without the watchdog the retry would have gotten a fresh budget. Since the default retry config is backoff on 5XX, this affects the default configuration whenever timeoutMs is set. Consider letting _do arm its own per-attempt timeout (e.g., via a request hook or by not setting options.signal when only timeoutMs is active), or document that timeoutMs becomes a per-turn cumulative budget when the watchdog is enabled.

2. ?? vs || for timeoutMs: 0 (same line). createTurnWatchContext uses ?? (baseOptions?.timeoutMs ?? this.options.client._options.timeoutMs), while responsesSend uses || (options?.timeoutMs || client._options.timeoutMs || -1). An explicit timeoutMs: 0 falls back to the client default in responsesSend but stays 0 (no timeout signal) in the watchdog path. This is arguably more correct (0 should mean "no timeout"), but it's an inconsistency — consider aligning the operator.

3. Test gap: multi-turn stall with concurrent stream consumers (see inline comment on test file). The "propagates the stall to all concurrent stream consumers" test covers the no-tools path (direct stream → ReusableReadableStream). For the tools path, a follow-up turn stall propagates through pipeAndConsumeStream → executeToolsIfNeeded → await executionPromise, not through the turnBroadcaster's completionError. This works (all stream methods await executionPromise), but it's untested — a test exercising concurrent getTextStream() + getFullResponsesStream() consumers with a stall in a follow-up turn would verify the error reaches both.

Risk assessment

Dimension	Severity	Risk	Reasoning
Implementation risk	🟨🟨	Medium	Core watchdog wiring is correct; the timeout-retry interaction is a subtle behavioral change not covered by tests.
Premise risk	🟩	Low	Per-turn stall detection is a sound design; the approach matches vercel/ai's per-step semantics.
Estimated impact	🟩	Low	Feature is opt-in (default off); worst case is degraded retry behavior for users who enable the watchdog with timeoutMs and retries.

Risk Factor	Severity	Risk	Reasoning
Reversibility	🟩	Low	Opt-in feature, unset by default.
Detectability	🟩	Low	Retry failures surface as timeout errors, immediately observable.
Blast radius	🟩	Low	Only affects callers who opt into timeout with timeoutMs + retries.
Data integrity	🟩	None	No persisted state is touched by the watchdog.
Financial exposure	🟩	None	No billing or payment paths affected.
Security and privacy exposure	🟩	None	No credentials or tenant isolation involved.
Propagation	🟩	Low	Errors surface to the caller; no downstream systems absorb them.
Availability	🟨🟨	Medium	Premature retry timeouts could cause requests to fail that would otherwise succeed on retry.
Recovery cost	🟩	Low	Disable the watchdog or adjust timeoutMs to recover.
Time to correct	🟩	Low	Fix is localized to createTurnWatchContext.

[perry-the-pr-reviewer]

▶ Suggestion: per-attempt timeout becomes cumulative across retries

The SDK's _do method creates a fresh AbortSignal.timeout(timeoutMs) inside the retry callback (sdks.ts:284-288), so each retry attempt gets a full timeout budget. When the watchdog is active, createTurnWatchContext supplies options.signal, which causes _createRequest to skip context.timeoutMs (sdks.ts:230-231), disabling _do's per-attempt timer. The AbortSignal.timeout(timeoutMs) created here is a single timer started before the first request — it counts down across ALL retry attempts of the turn.

With the default backoff retry on 5XX, a request that gets a 502 after most of the budget has elapsed will have its retry cut off by this cumulative timeout, whereas without the watchdog the retry would have gotten a fresh budget.

Consider one of:

• Let _do arm its own per-attempt timeout (e.g., pass the turn signal through a request hook instead of options.signal, so _createRequest still sets context.timeoutMs).
• Document that timeoutMs becomes a per-turn cumulative budget when the watchdog is enabled.

Minor note on the same line: ?? here diverges from responsesSend's || (responsesSend.ts:282) for timeoutMs: 0 — an explicit 0 stays 0 (no timeout signal) here but falls back to the client default in responsesSend. Consider aligning the operator.

▶ Prompt for agents: In ModelResult.createTurnWatchContext, the replicated AbortSignal.timeout(timeoutMs) is created once and merged into the request signal, but the SDK's _do normally creates a fresh timeout per retry attempt. Because the watchdog always supplies a signal, _do's per-attempt timer is disabled. Consider a mechanism that preserves per-attempt semantics — e.g. passing the turn abort signal through a request hook rather than as options.signal, or documenting the cumulative-budget behavior. Also align the ?? operator with responsesSend's || for timeoutMs: 0.

[perry-the-pr-reviewer]

▶ Suggestion: add a test for multi-turn stall with concurrent stream consumers

The existing "propagates the stall to all concurrent stream consumers" test (line 311) covers the no-tools path (direct stream → ReusableReadableStream). This test covers the multi-turn path but only asserts on getText().

For the tools path, a follow-up turn stall propagates through pipeAndConsumeStream → executeToolsIfNeeded → await executionPromise, not through the turnBroadcaster's completionError (the .finally() in startTurnBroadcasterExecution calls broadcaster.complete() without the error). All stream methods await executionPromise so the error does reach consumers, but this path is untested.

Consider adding a test that runs concurrent getTextStream() + getFullResponsesStream() consumers with a stall in a follow-up turn (tools enabled) and asserts both reject with StreamStalledError.

▶ Prompt for agents: Add a test to tests/unit/call-model-stream-timeout.test.ts that exercises the multi-turn (tools) path: a stall in a follow-up turn with concurrent getTextStream() and getFullResponsesStream() consumers, asserting both reject with StreamStalledError. The existing concurrent-consumers test only covers the no-tools single-turn path.