Skip to content

Suspend sandbox execution deadlines while a tool dispatch awaits the host#1437

Merged
RhysSullivan merged 1 commit into
mainfrom
fix/sandbox-deadline-suspends-during-dispatch
Jul 20, 2026
Merged

Suspend sandbox execution deadlines while a tool dispatch awaits the host#1437
RhysSullivan merged 1 commit into
mainfrom
fix/sandbox-deadline-suspends-during-dispatch

Conversation

@RhysSullivan

Copy link
Copy Markdown
Collaborator

Problem

Every sandbox runtime enforced its execution timeout as absolute wall-clock from execution start — including time spent paused for human approval. An approval-gated tool call is an in-flight tool dispatch awaiting the host, so a pause kept burning the 5-minute budget: when it expired mid-wait the fiber settled, the engine dropped the outstanding pause, and an approval granted inside the host's advertised approval window landed on a dead execution (Execution timed out after 300000ms, then Paused execution is unknown on retry).

Reproduced on production cloud with two chained approval gates: the second approval, granted 1m50s inside its advertised 4-minute window, hit the already-expired sandbox. The first approved step's side effect had committed, so the execution died half-completed. Also reproduced on selfhost (QuickJS's fixed deadline).

The host-side watchdog in runtime-dynamic-worker already had the correct accounting: its reference point pins to now while a dispatch is in flight and advances when it returns, so only autonomous sandbox compute counts. This change adopts that same rule in the three sandbox-side timers.

Fix

Uniform reference-point rule in all three runtimes — the deadline fires only after timeoutMs of continuous autonomous compute; time while ≥1 tool dispatch awaits the host doesn't count, and each dispatch return grants a fresh budget:

  • QuickJS (selfhost, Cloudflare self-host worker, local app): a per-evaluation deadline tracker replaces the fixed deadlineMs; the interrupt handler, job drain, and deferred waits all read it, and the tool bridge marks dispatch start/return. While a dispatch is in flight the deferred wait runs with no timer (a dispatch's deferred always settles, so this cannot hang).
  • Dynamic worker (cloud): the generated module's fixed setTimeout race becomes a 1s-poll watchdog over in-flight/last-returned state, cleared in a finally. State lives inside evaluate() and the clock starts at evaluate entry.
  • Deno subprocess: same reference-point watchdog host-side, with the in-flight bracket structured so the decrement runs even if the invoke or write fails.

The engine's drop-pauses-on-settle behavior is unchanged — with the timers fixed, a paused execution no longer settles out from under its pause. A wedged sandbox still dies: autonomous compute exceeding the budget times out exactly as before, with the same messages, and cloud's host-side timeoutMs + grace backstop is untouched.

Tests

  • Per-runtime behavior tests (QuickJS, dynamic worker, deno): a dispatch outlasting the timeout does not kill the execution; autonomous compute exceeding the timeout still does; the budget resets (not accumulates) after a dispatch returns.
  • Repurposed the core tool-invoker drop-on-settle test to an independently-failing executor fixture (the old fixture relied on QuickJS expiring during a pause — now dead behavior by design); the drop-on-settle assertion is unchanged and now exact-match.
  • New e2e scenario resume-after-sandbox-deadline.test.ts: one execution, two chained approval gates; the first approved at ~3.5 min (late in its window), the second at ~5.75 min after start — past the old absolute deadline, inside its own advertised window. Both approvals reach live pauses and both gated tools run. Passes on selfhost and on cloud when booted with a production-like paused-session idle window; skips (with reason) under the suite's default seconds-scale idle-teardown override, which would evict the paused session for unrelated reasons mid-journey.

Gates: kernel suites (75/18/16), core/execution (53), typecheck, lint, format:check green. Changesets: patch for @executor-js/runtime-quickjs (published) and the two private runtimes.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing 076edba Commit Preview URL

Branch Preview URL
Jul 20 2026, 08:34 AM

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Cloudflare preview

Torn down — the PR is closed.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud 076edba Jul 20 2026, 08:35 AM

@pkg-pr-new

pkg-pr-new Bot commented Jul 20, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@1437

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@1437

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@1437

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@1437

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@1437

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@1437

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@1437

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@1437

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@1437

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@1437

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@1437

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@1437

executor

npm i https://pkg.pr.new/executor@1437

commit: 076edba

@RhysSullivan
RhysSullivan marked this pull request as ready for review July 20, 2026 09:01
@RhysSullivan
RhysSullivan merged commit 99c808f into main Jul 20, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant