From 36c5c221794aa97a50910e3dfb34e4a418b7f1cc Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 3 Aug 2026 23:25:19 +0000 Subject: [PATCH 1/3] docs: update README test coverage with current fixture results The README still cited 91,392 tests from a `bun test fixtures.test.ts` command that no longer exists. Replace it with the state/blockchain test split from the latest green CI run on main (49,931 + 59,722 = 109,653, all passing), document how to run the fixtures locally via test/cli.ts, and add Arrow Glacier and Gray Glacier to the supported fork table. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_015Tf5vdNSMushEZtQ2dc77r --- README.md | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 070785a..29637b1 100644 --- a/README.md +++ b/README.md @@ -15,18 +15,33 @@ Other packages in this repo (`@evm-effect/ethereum-types`, `@evm-effect/crypto`, ## Test Coverage -This implementation is extensively tested against the official [Ethereum Execution Specs](https://github.com/ethereum/execution-specs) state tests and blockchain tests: +This implementation is extensively tested against the official [Ethereum Execution Specs](https://github.com/ethereum/execution-specs) state tests and blockchain tests. The full fixture set (`tests-bal@v7.1.1`) runs on every push and pull request via [`.github/workflows/evm-tests.yml`](.github/workflows/evm-tests.yml): + +| Suite | Tests | Result | +|-------|------:|--------| +| State tests | 49,931 | ✅ all passing | +| Blockchain tests | 59,722 | ✅ all passing | +| **Total** | **109,653** | ✅ all passing | + +Numbers from the latest run on `main` (commit `381b062`): state tests in 2h 44m, blockchain tests in 4h 36m, zero failures. + +### Running the fixtures locally ```shell -➜ evm-effect git:(main) ✗ bun test packages/evm/test/fixtures.test.ts -... +cd packages/evm +pnpm run fixtures # download the execution-spec fixtures (requires uv) +pnpm tsx ./test/cli.ts --format=state_test +pnpm tsx ./test/cli.ts --format=blockchain_test +``` - 91392 pass - 0 fail - 2851964 expect() calls -Ran 91392 tests across 1 file. [11202.85s] +The full suite takes several hours, so narrow it down with `--filter` (comma-separated substrings matched against the test id, fixture hash, fork, and format) and `--limit`: + +```shell +pnpm tsx ./test/cli.ts --format=state_test --filter=eip7702 --limit=100 ``` +Add `--trace` to emit EIP-3155 traces per test case, or `--emit-report` to write a pass/fail report file. + ## Supported Forks All released Ethereum forks are supported: @@ -44,6 +59,8 @@ All released Ethereum forks are supported: | Muir Glacier | ✅ | | Berlin | ✅ | | London | ✅ | +| Arrow Glacier | ✅ | +| Gray Glacier | ✅ | | Paris (The Merge) | ✅ | | Shanghai | ✅ | | Cancun | ✅ | From 4929a28bf516e0f6054a7f0fdf4133a5f7344d0a Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 3 Aug 2026 23:36:54 +0000 Subject: [PATCH 2/3] docs: clarify what the 109,653 passing fixtures cover Break down the fixture release so the headline number can't be mistaken for total coverage: 220,583 fixtures in the release, 148,067 in the two formats the runner consumes, 109,653 actually executed. Record what the fork allowlist skips (Amsterdam, Petersburg, transition forks) and which fixture formats are not consumed. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_015Tf5vdNSMushEZtQ2dc77r --- README.md | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 29637b1..fa09736 100644 --- a/README.md +++ b/README.md @@ -15,15 +15,30 @@ Other packages in this repo (`@evm-effect/ethereum-types`, `@evm-effect/crypto`, ## Test Coverage -This implementation is extensively tested against the official [Ethereum Execution Specs](https://github.com/ethereum/execution-specs) state tests and blockchain tests. The full fixture set (`tests-bal@v7.1.1`) runs on every push and pull request via [`.github/workflows/evm-tests.yml`](.github/workflows/evm-tests.yml): +This implementation is extensively tested against the official [Ethereum Execution Specs](https://github.com/ethereum/execution-specs) state tests and blockchain tests, from the `tests-bal@v7.1.1` fixture release. Both suites run on every push and pull request via [`.github/workflows/evm-tests.yml`](.github/workflows/evm-tests.yml): -| Suite | Tests | Result | -|-------|------:|--------| -| State tests | 49,931 | ✅ all passing | -| Blockchain tests | 59,722 | ✅ all passing | -| **Total** | **109,653** | ✅ all passing | +| Suite | Tests | Result | Runtime | +|-------|------:|--------|---------| +| State tests | 49,931 | ✅ all passing | 2h 44m | +| Blockchain tests | 59,722 | ✅ all passing | 4h 36m | +| **Total** | **109,653** | ✅ **0 failures** | | -Numbers from the latest run on `main` (commit `381b062`): state tests in 2h 44m, blockchain tests in 4h 36m, zero failures. +Numbers from the latest run on `main` (commit `381b062`). + +
+What isn't covered by those runs + +The fixture release contains 220,583 test cases in total; 148,067 of them are in the two formats this runner consumes (`state_test` and `blockchain_test`). Of those, 109,653 execute — the rest are skipped by the fork allowlist in [`test/cli.ts`](packages/evm/test/cli.ts): + +| Skipped | Tests | Why | +|---------|------:|-----| +| Amsterdam | 35,480 | Fork still in progress | +| Petersburg (`ConstantinopleFix`) | 2,561 | Not yet in the enabled fork set | +| Transition forks (`ShanghaiToCancunAtTime15k`, …) | 373 | Not yet in the enabled fork set | + +The remaining 72,516 fixtures are in formats the runner does not consume: `blockchain_test_engine` (72,333), `transaction_test` (165), and `blockchain_test_sync` (18). + +
### Running the fixtures locally From b2d46f55010e9789efde0e7aadf1a1330d4c71e9 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 3 Aug 2026 23:43:43 +0000 Subject: [PATCH 3/3] ci: skip EVM fixture tests for docs-only changes The state and blockchain fixture suites take ~2h45m and ~4h35m, and ran on every push regardless of what changed, so documentation edits had to be cancelled by hand. Ignore markdown and LICENSE changes on both the push and pull_request triggers. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_015Tf5vdNSMushEZtQ2dc77r --- .github/workflows/evm-tests.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/evm-tests.yml b/.github/workflows/evm-tests.yml index 0d29630..1a7c356 100644 --- a/.github/workflows/evm-tests.yml +++ b/.github/workflows/evm-tests.yml @@ -3,8 +3,14 @@ name: EVM Tests on: push: branches: [main] + paths-ignore: + - "**.md" + - "LICENSE" pull_request: branches: [main] + paths-ignore: + - "**.md" + - "LICENSE" jobs: state-tests: