feat(acp): ship Codex and actor session benchmark#1750
Draft
NathanFlurry wants to merge 57 commits into
Draft
Conversation
fix(sidecar): remove implicit host path execution
# Conflicts: # .gitignore # crates/agentos-actor-plugin/src/actions/shell.rs # crates/client/src/shell.rs # crates/client/tests/shell_e2e.rs # crates/native-sidecar/src/execution.rs # crates/native-sidecar/tests/signal.rs # packages/core/README.md # packages/core/src/agent-os.ts # packages/core/src/runtime-compat.ts # packages/core/src/sidecar/rpc-client.ts # packages/core/src/test/terminal-harness.ts # packages/core/tests/brush-interactive.test.ts # packages/core/tests/native-sidecar-process.test.ts # packages/runtime-core/src/kernel-proxy.ts # packages/runtime-core/src/test-runtime.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
actor-sessionbenchmark for Claude Code, Pi, Codex, and OpenCode, measuring both session spawn and the first mocked LLM turnjust install-gigacodeto rebuild, reinstall, and version-check the sibling Gigacode checkoutWhy
We need a repeatable answer for two different latency boundaries: when an agent process has completed its ACP handshake, and when its first message reaches the provider and completes. Codex also needs to be tested as the two-package combination (
codexagent adapter pluscodex-cli/WASI engine), not just as independently buildable artifacts.Benchmark
Command:
All four agents completed 3/3 measured session creations and 3/3 mocked prompts after one excluded warmup.
The provider is local
llmock, so the first-turn numbers primarily measure agent/runtime startup and protocol overhead rather than network/model latency. The run exposed non-fatal teardown diagnostics after successful measurements (Claude processESRCHand a final VM close-state warning); those do not change the recorded success counts.Validation
rivet-dev/codex@8f4655a9dviamake -C registry/native codex-required(29,929,852-byte optimized artifact)@rivet-dev/agentos-toolchain: build, typecheck, 18 tests passed (2 skipped)@agentos-software/codex-cli: staged and assembled bothcodexandcodex-exec@agentos-software/codex: typecheck, package build, and package projection test passedGIGACODE_ROOT=/home/nathan/sandbox-agent just install-gigacoderebuilt and reinstalledgigacode 0.4.2Notes
mainadvanced with three website-only commits after this stack was validated; the preview is cut from this exact branch before any unrelated rebase.