Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions benchmarks/app-patterns/results/matrix-20260803-072044.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# App-pattern bench matrix — 2026-08-03 07:20:44

Each cell is mean ms (min ms). Lower is better.
Final column is **perry/bun ratio** — > 2× is a real gap, < 1.5× is fine.

| Kernel | Perry | Bun | Node | perry/bun | perry/node |
|---|---:|---:|---:|---:|---:|
| batch | 147.7 (146.2) | 26.4 (25.6) | 73.6 (71.4) | 5.59x ✗ slow | 2.01x |
| buffer_transcode | 60.9 (58.1) | 43.1 (41.4) | 85.6 (83.7) | 1.41x ✓ ok | 0.71x |
| date_format_parse | 36.5 (35.7) | 46.6 (44.5) | 116.0 (112.5) | 0.78x ✅ win | 0.31x |
| json_parse_1mb | 133.7 (132.2) | 68.5 (67.3) | 124.6 (121.1) | 1.95x ⚠ borderline | 1.07x |
| json_stringify_1mb | 97.2 (96.3) | 37.8 (36.8) | 93.6 (90.8) | 2.57x ✗ slow | 1.04x |
| map_1m | 681.0 (666.3) | 229.5 (216.9) | 270.6 (259.0) | 2.97x ✗ slow | 2.52x |
| object_deep_clone | 559.6 (546.4) | 17.5 (16.9) | 55.4 (53.6) | 31.98x ✗ slow | 10.10x |
| promise_all_chains | 158.7 (156.6) | 22.7 (22.0) | 62.5 (60.6) | 6.99x ✗ slow | 2.54x |
| regex_replace | 57.8 (56.8) | 51.4 (50.4) | 102.0 (99.6) | 1.12x ✓ ok | 0.57x |
| string_concat_csv | 40.0 (39.4) | 28.2 (27.6) | 86.3 (84.6) | 1.42x ✓ ok | 0.46x |
| string_split_map_join | 41.0 (40.3) | 47.6 (45.8) | 79.5 (78.0) | 0.86x ✅ win | 0.52x |
| string_template_interp | 68.3 (67.1) | 42.6 (40.9) | 104.2 (101.6) | 1.60x ⚠ borderline | 0.66x |

## Summary

- Perry-faster-than-Bun kernels: **2**
- Borderline (1.5–2× slower): **2**
- Slow (≥ 2× slower): **5**

**Priority backlog:** the ✗ slow rows are the gaps. Each one is a focused workstream.
78 changes: 39 additions & 39 deletions benchmarks/honest_bench/REPORT.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
# Perry vs Rust vs Zig — honest benchmark results

Numbers are measured against Perry v0.5.908 on five implementations: Rust, Zig, Perry, Node.js, Bun. All three workloads complete on all five implementations and every measured run's output matches the Bun reference byte-for-byte (300/300 rows; correctness gate from `#441`). The auto-generated tables below come straight from `results/results.json`; the bottom-line summary tracks the most recent sweep.
Numbers are measured against Perry v0.5.81 on five implementations: Rust, Zig, Perry, Node.js, Bun. All three workloads complete on all five implementations every correctness and scaling bug the bench originally surfaced (`#38`–`#53`, `#62`–`#65`) has landed in Perry mainline. 260 measured runs, zero failures.

## Bottom line

- **Compute (image convolution, tight loop, minimal heap):** Zig 245 ms.
**Perry 354 ms (1.44× Zig) — ahead of Rust's 392 ms**, 2.6× faster
than Bun (915 ms), 3.4× faster than Node (1207 ms). Slight improvement
vs yesterday's v0.5.891 (365 → 354 ms); the bulk of the original arc
(2261 ms → ~377 ms, ~6×) landed during the v0.5.30–v0.5.495 perf sprint.
- **Allocation-heavy (JSON pipeline, 100 records):** Zig 35 ms, Rust 34 ms,
**Perry 39 ms (1.15× the fastest) — ahead of Bun's 51 ms and Node's 144 ms**,
at ~3 MB RSS vs Bun's 11 MB and Node's 36 MB. Stable vs v0.5.891.
- **JSON at scale (500k records, 108 MB):** Rust 608 ms, Bun 650 ms, Zig
867 ms, Node 966 ms, **Perry 1098 ms**. Perry trails Rust/Bun by ~1.7×,
beats Node by 9%, and uses 731 MB peak RSS. The workload completes
reliably — as recently as v0.5.68 this hung >13 min CPU without
finishing (`#65`, now closed). Improved from v0.5.891's 1155 ms (-57 ms)
thanks in part to the v0.5.900 GC fix for #745.
- **Compute (image convolution, tight loop, minimal heap):** Zig 243 ms.
**Perry 268 ms (1.10× Zig) — ahead of Rust's 567 ms**, 3.4× faster than
Bun, 4.5× faster than Node. Full arc across the perf sprint:
**2261 ms → 268 ms, an 8.4× speedup** (issues #47, #48, #49, #50, #52 plus
the v0.5.58–v0.5.68 follow-on).
- **Allocation-heavy (JSON pipeline, 100 records):** Zig 32 ms, Rust 33 ms,
**Perry 37 ms (1.15× the fastest) — ahead of Bun's 49 ms and Node's 145 ms**,
at 3 MB RSS vs Bun's 11 MB and Node's 36 MB. Earlier sweep had Perry at
210 ms here; v0.5.69–v0.5.75 closed ~6× of that.
- **JSON at scale (500k records, 108 MB):** Rust 604 ms, Bun 647 ms, Zig
850 ms, Node 1010 ms, **Perry 1649 ms**. Perry trails the pack by ~2.5×
on wall time and uses more RSS (744 MB peak), but it _completes_ — as
recently as v0.5.68 this workload hung >13 min CPU without finishing
(`#65`, now closed).
- **Binary size:** Zig smallest (~230 KB), Rust next (~300–380 KB), Perry
(~550–700 KB, including GC + Node-compat shims). Node/Bun don't have
standalone binaries — they require their runtime installed separately.
Expand All @@ -36,55 +36,55 @@ Charts: [image convolution](charts/image_convolution.png),
|---|---|
| CPU | Apple M1 Max (10 cores, arm64) |
| RAM | 64.0 GB |
| OS | macOS 26.4 (Darwin) |
| Rust | `rustc 1.94.1 (e408947bf 2026-03-25)` |
| OS | macOS 26.5 (Darwin) |
| Rust | `rustc 1.97.1 (8bab26f4f 2026-07-14)` |
| Zig | `0.15.2` |
| Perry | `perry 0.5.908` |
| Python | `Python 3.14.4` |
| Perry | `perry 0.5.1279` |
| Python | `Python 3.14.6` |
| Runs | 5 warmup + 20 measured, median reported |
| Generated | 2026-05-14T07:44:59.755910+00:00 |
| Generated | 2026-08-03T05:25:59.733855+00:00 |

## 3. Image convolution (5×5 Gaussian, 3840×2160 RGB)

_In-memory input + output checksum (no PPM I/O) — see the workload README for the reason. All three languages produce the identical FNV-1a-32 checksum._

| Language | Wall median (ms) | Wall σ | Peak RSS | Binary size | Source LoC | Runs OK |
|---|---:|---:|---:|---:|---:|---:|
| rust | 392.1 | 3.8 | 48.5 MB | 295.5 KB | 112 | 20/20 |
| zig | 245.4 | 6.2 | 48.5 MB | 227.0 KB | 113 | 20/20 |
| perry | 354.0 | 2.3 | 49.9 MB | 966.6 KB | 92 | 20/20 |
| node | 1,206.6 | 22.0 | 86.5 MB | — | 86 | 20/20 |
| bun | 915.4 | 35.2 | 60.0 MB | — | 86 | 20/20 |
| rust | 406.4 | 2.0 | 48.9 MB | 295.5 KB | 112 | 20/20 |
| zig | 266.6 | 5.9 | 48.9 MB | 227.0 KB | 113 | 20/20 |
| perry | 278.8 | 4.7 | 56.0 MB | 4.3 MB | 92 | 20/20 |
| node | 1,232.2 | 3.3 | 117.9 MB | — | 86 | 20/20 |
| bun | 909.7 | 34.9 | 83.1 MB | — | 86 | 20/20 |

_Ratios vs fastest: rust = 1.60×, zig = 1.00×, perry = 1.44×, node = 4.92×, bun = 3.73×_
_Ratios vs fastest: rust = 1.52×, zig = 1.00×, perry = 1.05×, node = 4.62×, bun = 3.41×_

## 1a. JSON pipeline — small fixture (100 records, 21 KB)

_All three languages produce byte-identical output at this scale (hash `7fc66fa8`)._

| Language | Wall median (ms) | Wall σ | Peak RSS | Binary size | Source LoC | Runs OK |
|---|---:|---:|---:|---:|---:|---:|
| rust | 34.1 | 1.1 | 1.2 MB | 376.3 KB | 99 | 20/20 |
| zig | 34.5 | 1.1 | 1.5 MB | 309.1 KB | 112 | 20/20 |
| perry | 39.2 | 1.2 | 3.5 MB | 1.0 MB | 52 | 20/20 |
| node | 143.6 | 1.8 | 36.1 MB | — | 40 | 20/20 |
| bun | 51.1 | 1.4 | 10.6 MB | — | 40 | 20/20 |
| rust | 59.4 | 1.9 | 1.7 MB | 360.2 KB | 99 | 20/20 |
| zig | 58.9 | 2.0 | 2.0 MB | 309.2 KB | 112 | 20/20 |
| perry | 64.1 | 3.1 | 8.5 MB | 4.3 MB | 52 | 20/20 |
| node | 110.6 | 2.4 | 64.2 MB | — | 40 | 20/20 |
| bun | 76.5 | 1.2 | 30.3 MB | — | 40 | 20/20 |

_Ratios vs fastest: rust = 1.00×, zig = 1.01×, perry = 1.15×, node = 4.21×, bun = 1.50×_
_Ratios vs fastest: rust = 1.01×, zig = 1.00×, perry = 1.09×, node = 1.88×, bun = 1.30×_

## 1b. JSON pipeline — full fixture (500k records, 108 MB)

_All five implementations complete this workload against the same 108 MB fixture and produce the same hash `b7e8a588`. Perry completes in ~1.1 s, ~1.8× the leader (Rust); as recently as v0.5.68 this workload hung >13 minutes without finishing (`#65`, now closed)._
_All five implementations complete this workload against the same 108 MB fixture and produce the same hash `b7e8a588`. Perry completes in ~1.6 s, ~2.7× the leader (Rust / Bun); as recently as v0.5.68 this workload hung >13 minutes without finishing (`#65`, now closed)._

| Language | Wall median (ms) | Wall σ | Peak RSS | Binary size | Source LoC | Runs OK |
|---|---:|---:|---:|---:|---:|---:|
| rust | 608.3 | 13.0 | 430.6 MB | 376.3 KB | 99 | 20/20 |
| zig | 867.4 | 19.4 | 576.8 MB | 309.1 KB | 112 | 20/20 |
| perry | 1,098.4 | 9.4 | 731.0 MB | 1.0 MB | 52 | 20/20 |
| node | 966.3 | 9.8 | 880.0 MB | — | 40 | 20/20 |
| bun | 649.9 | 14.7 | 593.7 MB | — | 40 | 20/20 |
| rust | 634.6 | 26.0 | 430.9 MB | 360.2 KB | 99 | 20/20 |
| zig | 839.8 | 13.3 | 576.9 MB | 309.2 KB | 112 | 20/20 |
| perry | 2,591.2 | 16.5 | 731.8 MB | 4.3 MB | 52 | 20/20 |
| node | 889.6 | 6.1 | 775.1 MB | — | 40 | 20/20 |
| bun | 570.3 | 2.9 | 579.8 MB | — | 40 | 20/20 |

_Ratios vs fastest: rust = 1.00×, zig = 1.43×, perry = 1.81×, node = 1.59×, bun = 1.07×_
_Ratios vs fastest: rust = 1.11×, zig = 1.47×, perry = 4.54×, node = 1.56×, bun = 1.00×_

## Honest findings — Perry gaps surfaced by this benchmark

Expand Down
82 changes: 74 additions & 8 deletions benchmarks/honest_bench/results/metadata.json
Original file line number Diff line number Diff line change
@@ -1,22 +1,88 @@
{
"generated_at": "2026-05-14T07:44:59.755910+00:00",
"generated_at": "2026-08-03T05:25:59.733855+00:00",
"commit": "defa4d6012fb480fca237ea10509da74bd41624d",
"host": {
"os_version": "26.4",
"kernel": "Darwin MacBook-Pro-69.local 25.4.0 Darwin Kernel Version 25.4.0: Thu Mar 19 19:30:44 PDT 2026; root:xnu-12377.101.15~1/RELEASE_ARM64_T6000 arm64",
"os_version": "26.5",
"kernel": "Darwin MacBookPro.fritz.box 25.5.0 Darwin Kernel Version 25.5.0: Mon Apr 27 20:38:56 PDT 2026; root:xnu-12377.121.6~2/RELEASE_ARM64_T6000 arm64",
"arch": "arm64",
"cpu": "Apple M1 Max",
"ncpu": "10",
"ram_gb": 64.0
},
"toolchains": {
"rustc": "rustc 1.94.1 (e408947bf 2026-03-25)",
"cargo": "cargo 1.94.1 (29ea6fb6a 2026-03-24)",
"rustc": "rustc 1.97.1 (8bab26f4f 2026-07-14)",
"cargo": "cargo 1.97.1 (c980f4866 2026-06-30)",
"zig": "0.15.2",
"python": "Python 3.14.4",
"perry": "perry 0.5.908"
"python": "Python 3.14.6",
"node": "v22.23.1",
"bun": "1.3.14",
"oha": "",
"perry": "perry 0.5.1279"
},
"executables": {
"perry": "/Users/amlug/projects/perry/wt-baseline2/target/release/perry",
"node": "/Users/amlug/.cache/perry-bench-tools/bin/node",
"bun": "/Users/amlug/.cache/perry-bench-tools/bin/bun"
},
"harness": {
"warmup": 5,
"measured": 20
"measured": 20,
"http_duration": "15s",
"http_warmup_duration": "5s",
"http_concurrency": 10
},
"commands": {
"perry": [
"<compiled-workload>"
],
"perry_compile": [
"/Users/amlug/projects/perry/wt-baseline2/target/release/perry",
"<source.ts>",
"-o",
"<compiled-workload>"
],
"node": [
"/Users/amlug/.cache/perry-bench-tools/bin/node",
"--experimental-strip-types",
"<source.ts>"
],
"bun": [
"/Users/amlug/.cache/perry-bench-tools/bin/bun",
"run",
"<source.ts>"
],
"perry_http": [
"<compiled-kernel>"
],
"perry_http_compile": [
"/Users/amlug/projects/perry/wt-baseline2/target/release/perry",
"<kernel.ts>",
"-o",
"<compiled-kernel>"
],
"node_http": [
"node",
"--import",
"tsx",
"<kernel.ts>"
],
"bun_http": [
"bun",
"run",
"<kernel.ts>"
],
"oha": [
"/tmp/oha",
"-z",
"15s",
"-c",
"10",
"-r",
"0",
"--no-tui",
"--output-format",
"json",
"<url>"
]
}
}
Loading
Loading