Skip to content

test: deterministic test infrastructure - #10

Merged
sderrow merged 2 commits into
mainfrom
sean/fix-flaky-tests
Jul 12, 2026
Merged

test: deterministic test infrastructure#10
sderrow merged 2 commits into
mainfrom
sean/fix-flaky-tests

Conversation

@sderrow

@sderrow sderrow commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Empirical soak runs (20× full suite, 15× isolated) surfaced flakey tests, mostly around timing. This PR rebuilds the test infrastructure so timing flakes are structurally impossible rather than individually patched:

  • The local project runs on vitest fake timers (auto-advancing tick mode): timing assertions are now exact virtual times, and the project's wall time dropped ~9s → ~1.6s.
  • Redis-backed tests assert causality and state (deferred job holds, state polling) instead of wall-clock bounds, which real network time can't honor.
  • A test.extend() fixture layer guarantees limiter/connection teardown even when assertions throw — leaked heartbeats were amplifying the flakes — plus custom matchers that are exact under fake time and lower-bound under real time.

Validated with repeated 50-run soaks: zero failures.

sderrow commented Jul 12, 2026

Copy link
Copy Markdown
Owner Author

@sderrow
sderrow force-pushed the sean/fix-flaky-tests branch from 6558f90 to 12eb8a7 Compare July 12, 2026 03:09
@sderrow sderrow changed the title fix(tests): reduce flakiness test: deterministic test infrastructure — fake clocks, fixtures, causality Jul 12, 2026
@sderrow sderrow changed the title test: deterministic test infrastructure — fake clocks, fixtures, causality test: deterministic test infrastructure Jul 12, 2026
@sderrow
sderrow marked this pull request as ready for review July 12, 2026 04:15

sderrow commented Jul 12, 2026

Copy link
Copy Markdown
Owner Author

Merge activity

  • Jul 12, 4:25 AM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Jul 12, 4:25 AM UTC: @sderrow merged this pull request with Graphite.

@sderrow
sderrow merged commit cda53df into main Jul 12, 2026
4 checks passed
@sderrow
sderrow deleted the sean/fix-flaky-tests branch July 12, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant