Skip to content

fix(tests): serialize AffordanceRegistry tests to remove flake - #20

Merged
hanrw merged 1 commit into
tddworks:mainfrom
lswang6:fix/affordance-registry-test-isolation
Aug 4, 2026
Merged

fix(tests): serialize AffordanceRegistry tests to remove flake#20
hanrw merged 1 commit into
tddworks:mainfrom
lswang6:fix/affordance-registry-test-isolation

Conversation

@lswang6

@lswang6 lswang6 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Problem

Two tests fail intermittently under parallel scheduling:

  • booted simulator apiLinks include plugin stream when registered
  • booted simulator affordances include plugin stream when registered
✘ Expectation failed: (sim.apiLinks["stream"] → nil) != nil
   SimulatorTests.swift:187

Root cause

AffordanceRegistry (Sources/Domain/Shared/AffordanceRegistry.swift) is process-global mutable state. Two suites in DomainTests touch it:

  • AffordanceRegistryTests@Suite(.serialized), calls AffordanceRegistry.reset() in init() before every test
  • SimulatorTests — plain parallel @Suite; three tests register() a provider and then assert on the merged affordance

.serialized only orders tests within a suite — separate suites still run concurrently. So AffordanceRegistryTests.init()'s reset() lands between SimulatorTests' register() and its assertion, and the affordance reads back nil.

The .serialized trait on the individual test functions does not help here: it scopes parameterized cases, not cross-test isolation. That's why the three Simulator plugin tests were also racing each other — both affected tests register a provider for Simulator and reset at entry.

Fix

Move the three registry-touching Simulator tests into the serialized AffordanceRegistryTests suite, so every test that mutates the global registry shares one serial scope.

  • Assertions are unchanged — the tests only moved.
  • The redundant inline reset() calls are dropped; the suite's init() already resets.
  • Comments added at both sites explaining why registering a provider from any other suite reintroduces the flake.

Test-only change; no production code touched.

Tests/ASCCommandTests/.../SimulatorsListTests.swift also calls reset(), but it never registers a provider and nothing else in that target does either, so it is unaffected and left alone.

Verification

swift test --filter "SimulatorTests|AffordanceRegistryTests", 20 runs each:

failed runs
before 15 / 20
after 0 / 20

Full suite: 5/5 green, 2383 tests — unchanged count.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Added coverage confirming stream options and API links appear for booted simulators.
    • Verified stream options are not provided for shut-down simulators.
    • Consolidated registry behavior checks into a dedicated test area for more reliable results.

`booted simulator apiLinks include plugin stream when registered` and
`booted simulator affordances include plugin stream when registered`
failed intermittently under parallel scheduling.

AffordanceRegistry is process-global mutable state. AffordanceRegistryTests
is `@Suite(.serialized)` and resets the registry in `init()`, while
SimulatorTests is a plain parallel `@Suite` whose plugin tests register a
provider and then read the result. `.serialized` only orders tests *within*
a suite — separate suites still run concurrently, so the reset lands between
another suite's `register()` and its assertion and the affordance reads back
nil. The `.serialized` trait on the individual test functions does not help:
it scopes parameterized cases, not cross-test isolation, so the three
Simulator plugin tests were also racing each other.

Move the three registry-touching Simulator tests into the serialized
AffordanceRegistryTests suite so every test that mutates the global registry
shares one serial scope. Assertions are unchanged; the redundant inline
`reset()` calls are dropped since the suite's `init()` already resets.

Measured with `swift test --filter "SimulatorTests|AffordanceRegistryTests"`,
20 runs each: 15/20 failed before, 0/20 after. Full suite 5/5 green,
2383 tests unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: cc66c35c-ca93-42f1-bce2-124eac7f2fe8

📥 Commits

Reviewing files that changed from the base of the PR and between 2031f25 and 17c2a76.

📒 Files selected for processing (2)
  • Tests/DomainTests/Shared/AffordanceRegistryTests.swift
  • Tests/DomainTests/Simulators/SimulatorTests.swift

📝 Walkthrough

Walkthrough

The changes move simulator plugin affordance tests to AffordanceRegistryTests. The tests cover stream affordances and API links for booted simulators and confirm their absence for shutdown simulators.

Changes

Simulator affordance registry coverage

Layer / File(s) Summary
Serialized registry tests
Tests/DomainTests/Shared/AffordanceRegistryTests.swift, Tests/DomainTests/Simulators/SimulatorTests.swift
The registry suite verifies stream affordances and POST API links for booted simulators, excludes them for shutdown simulators, and replaces the former simulator test cases with a relocation note.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the test serialization change and its purpose of removing intermittent failures.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.54%. Comparing base (97a60af) to head (17c2a76).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main      #20   +/-   ##
=======================================
  Coverage   81.54%   81.54%           
=======================================
  Files         453      453           
  Lines       13441    13441           
=======================================
  Hits        10960    10960           
  Misses       2481     2481           

see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hanrw
hanrw merged commit 030c463 into tddworks:main Aug 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants