Skip to content

fix: re-enable skipped Heart tests and make non-asserting checks assert#7845

Open
voidmatcha wants to merge 2 commits into
coder:mainfrom
voidmatcha:fix/e2e-non-asserting-checks
Open

fix: re-enable skipped Heart tests and make non-asserting checks assert#7845
voidmatcha wants to merge 2 commits into
coder:mainfrom
voidmatcha:fix/e2e-non-asserting-checks

Conversation

@voidmatcha

@voidmatcha voidmatcha commented Jun 11, 2026

Copy link
Copy Markdown

This PR fixes test checks that cannot fail:

  • it.only committed in Add idle timeout #7539 (test/unit/node/heart.test.ts): silently skipped the other 8 Heart tests in CI for the past 7 months. One of them stopped passing while disabled (its rejection handler runs on the microtask queue, after the assertion), re-enabled and repaired with a microtask drain.
  • expect() with no matcher (login.test.ts, 4×): asserted nothing; the error-message tests passed whether or not the message rendered.
  • Dangling locator (extensions.test.ts): await page.getByText(...) with no assertion.
  • One-shot page.isVisible() reads (downloads/uploads/github/codeServer, 16×): returns immediately with no auto-retry; converted to web-first assertions:
-    expect(await codeServerPage.page.isVisible("text=Show Local")).toBe(false)
+    await expect(codeServerPage.page.locator("text=Show Local")).not.toBeVisible()

No assertions were removed and no behavior added. Every change strengthens an existing check.

Verification: ./ci/dev/test-unit.sh test/unit/node/heart.test.ts - 9/9 pass; tsc --noEmit + repo ESLint clean on changed files. The Playwright e2e suite was not run locally (requires a full code-server build), the e2e changes are mechanical assertion-form conversions. Happy to iterate if CI surfaces anything.


Found while reviewing the test suite with e2e-skills/e2e-reviewer, which flags committed .only leaks, one-shot visibility reads, and matcher-less expect() calls.

- Remove a committed `it.only` (introduced in coder#7539) that silently skipped
  the 8 sibling Heart unit tests in CI, and repair the "isActive rejects"
  test that stopped passing while it was skipped (the timer callback's
  rejection handler runs on the microtask queue; the assertion ran first).
- Convert 16 one-shot `page.isVisible()` reads in the e2e suite to
  web-first `await expect(locator).toBeVisible()/.not.toBeVisible()` —
  the one-shot form resolves immediately with no auto-retry.
- Add the missing matcher to four `expect(await page.isVisible(...))`
  calls in login.test.ts that asserted nothing, and to a dangling
  `getByText()` locator in extensions.test.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@voidmatcha voidmatcha requested a review from a team as a code owner June 11, 2026 15:39

@code-asher code-asher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for catching these!!

I think the .not.toBeVisible() ones will fail because locator will time out (since the element is not present) but otherwise it looks good to me. nvm, locator is the one that returns immediately, I was thinking of the getBy ones.

I feel like there has got to be a less janky way to wait for the microtask queue haha but I can look into that later.

Edit: hmm actually the ones that are visible are also failing. Not really sure why.
Edit 2: oh just the login test ones are failing.

The required password input blocks empty or cleared submissions
client-side, so the missing-password error and the rate limiter were
unreachable from the UI (the CI failure videos show the browser's native
validation bubble at the moment of the assertion):

- assert the native validation guard (input.password:invalid) for the
  missing-password case and rename the test to match the actual behavior
- refill the password before every attempt in the rate limiter test so
  all 15 attempts actually POST, and mark it test.slow() since each
  attempt now costs a real argon2 hash
- replace the 3x Promise.resolve() microtask drain in heart.test.ts with
  a single real-macrotask yield via jest.requireActual("timers")
  .setImmediate, which drains the queue deterministically

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@voidmatcha voidmatcha force-pushed the fix/e2e-non-asserting-checks branch from d81c14d to e9b15b2 Compare June 12, 2026 10:37
@voidmatcha

Copy link
Copy Markdown
Author

The password input is required, so submits with an empty field are blocked client-side and never POST. The failure videos show the native validation bubble at the moment of the assertion, so both login tests were never passable and the matcher-less expect() was hiding it.

image
  • missing-password: assert the guard the user actually gets (input.password:invalid) instead of the unreachable server error
  • rate limiter: refill the password before each attempt so all 15 actually POST (the re-render was clearing the required field), and test.slow() for the argon2 cost
  • microtask drain: swapped for a single deterministic yield via jest.requireActual("timers").setImmediate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants