Skip to content

feat(sandbox): add Fystash backend (Sandbox protocol) - #1007

Open
fystash wants to merge 1 commit into
benchflow-ai:mainfrom
fystash:feat/sandbox-fystash
Open

feat(sandbox): add Fystash backend (Sandbox protocol)#1007
fystash wants to merge 1 commit into
benchflow-ai:mainfrom
fystash:feat/sandbox-fystash

Conversation

@fystash

@fystash fystash commented Aug 16, 2026

Copy link
Copy Markdown

Adds a Fystash sandbox backend behind the existing Sandbox protocol so BenchFlow can run representative coding / verifier workloads on Firecracker microVMs.

What landed on our side (already live-proved)

We implemented and ran the protocol against production https://api.fystash.ai/v1 on 2026-08-15:

  • start / stop
  • exec (echo + python)
  • upload_file / download_file (verifier test.sh included)
  • snapshot / restore on the same sandbox (supports_snapshot=True means same-Run workspace tar rollback, not a Firecracker VM snapshot)
  • multi-step coding (write function, run check)

Runs: run_06G09E9AEHR01CNQ7W3JWXHTMC (new-Branch start 103335 ms) and run_06G0A455J5R014NP981S5N1T1W (same-Branch Ready 2115 ms). See Fystash docs/design-partners/benchflow-live.md.

What this PR adds

  • src/benchflow/sandbox/fystash.pyBaseSandbox port of the live-proved adapter
  • extra: sandbox-fystash depending on the public fystash Python SDK (>=0.6.0)
  • CLI: --sandbox fystash via the provider registry (same path as Daytona / Modal / AgentCore)
  • env: FYSTASH_API_URL (default https://api.fystash.ai/v1), FYSTASH_ACCESS_TOKEN, FYSTASH_PROJECT_ID (or fystash login + from_cli_context())
  • offline tests for registry drift, manifest-required _validate_definition, and preflight

Honest residuals

  • ImageBuilder does not build arbitrary guest Dockerfiles. Fystash pins python-agent / browser-agent via environment/environment.yaml or environment/sandbox.yaml.
  • Single workspace only — service != main is rejected (same class as Modal / direct Daytona).
  • No DinD, no GPU advertise.
  • expose_ports is declared; Preview grants are a separate Fystash Plan capability, not wired in this first PR.
  • Snapshot is workspace rollback on the same Run, not a vendor VM snapshot API.
  • exec(..., user=) is ignored (catalog actor is agent).
  • host is 127.0.0.1; Preview grants are unwired.
  • Reuse the same Branch for later starts. A new Branch pays Drive initialize (~2 min when healthy).

Happy to iterate on ImageBuilder / compose once this lands. This does not claim public GA.

Made with Cursor

Register --sandbox fystash so BenchFlow can run python-agent workloads on the public Fystash API, with honest residuals for ImageBuilder, DinD, GPU, and workspace-tar snapshots.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants