Skip to content

test(export): the stuck-export census was taken after the worker, so it could not answer its own question - #231

Merged
rubenvdlinde merged 3 commits into
developmentfrom
S49/e2e-triage
Aug 17, 2026
Merged

test(export): the stuck-export census was taken after the worker, so it could not answer its own question#231
rubenvdlinde merged 3 commits into
developmentfrom
S49/e2e-triage

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

The triage this came out of

openbuild's E2E failures were carried on the board as "3 rotating failures, all DECISION-16 (nc-vue CnDetailPage latch)". Measured across three consecutive development Code Quality runs, that is not the shape:

run created collected passed failed skipped failing tests
31955594412 08-16 15:25 257 190 2 65 export-flows-and-agents · app-icon-management
31960043906 08-16 16:55 257 190 2 65 export-flows-and-agents · app-icon-management
31964268323 08-16 18:19 257 189 3 65 those two + builder-host

Read off the embedded playwrightReportBase64 in each run's playwright-report artifact, not off log greps. So two failures are persistent and identical, and only the third rotates.

failure verdict
builder-host.spec.tsnavigates to a hello-message detail page the rotating one. Already filed as #187 (3 failures in 9 valid runs, on development and on PRs alike). A detail page whose sibling tests either side of it pass in the same run — the DECISION-16 signature. Not touched here.
spec-coverage/app-icon-management.spec.tsRemove in the dark slot… persistent, and NOT proven to be DECISION-16. The tab button resolves and never becomes visible/stable (228 retries, 120 s) — #188's "resolves N× and stays hidden" signature. But a passing sibling settles the surface question: iconUpload.spec.ts"the Icons tab mounts the upload section with light + dark variants and an SVG-only picker" passes in the same run, on the same Icons tab of the same detail page. So the tab is not permanently latched, and this needs a repro I could not get. Escalated, not guessed at.
export-flows-and-agents.spec.tsan operator binds a flow… persistent, and definitively not DECISION-16 — a background-job/lifecycle problem with no Vue in it. That is what this PR is about.

What this PR changes

The export failure reads:

the export job must finish — last status "queued";
last worker pass: worker ok: (no output); job list: 112 total, 0 RunExportJob

and 0 RunExportJob has been read as "the enqueue never happened" — the helper's own comment says the census exists to prove exactly that. It proves nothing, because the census was taken after background-job:worker ran, and a QueuedJob is deleted from oc_jobs once it executes. 0 RunExportJob is what you see whether the job never existed or ran to completion. A check that cannot distinguish its two hypotheses is the exact shape this spec was written to close, reproduced inside the helper written to close it.

  • The census is taken BEFORE the worker pass, and both are reported. before non-zero + after zero ⇒ the worker consumed our job, so a status still on queued means the transition did not fire. before zero against a non-zero total (proving occ shares the web server's database) ⇒ the enqueue genuinely did not happen.
  • On a non-terminal status the failure also reports the deployed exportJob schema — its version, and whether it carries x-openregister-lifecycle. Without that block OR's TransitionEngine finds no state machine and returns silently: the job row is consumed and the object sits at queued with no log line anywhere (fix(schema): bump exportJob past the version instances already carry #219, still open — exportJob declares 0.1.0, and the dev instance carries 1.0.1, so this repo's lifecycle edits can never reach it).

Read from the schema API rather than the object: this OpenRegister build exposes no available-actions on an object read at all — measured on :8080, including with ?_extend=all — so the probe #219 proposed would report "absent" on a perfectly healthy instance and read as evidence.

What this does NOT claim

It does not claim the red cell goes green. No assertion is relaxed, no skip is added, no timeout is raised; the failure only gets louder and names its own cause on the next run.

I could not settle the root cause from outside the instance, and I am not going to guess at it: CI seeds a fresh instance (ci-seed.shsettings#load (forced import), and the run's own log lists export-job among the provisioned schemas), so the declared-0.1.0 schema should import with its lifecycle and #219 should not bite there — while the dev instance, whose stored schema is at 1.0.1 with the lifecycle, has export-job objects sitting at status: succeeded. Those two facts do not yet compose into an explanation. The two probes added here are chosen to decide it on the next run rather than argue about it.

Verification

  • npx playwright test --list on the changed file: 2 tests in 1 file, compiles.
  • prettier --check on the changed file: clean (base was clean too — checked with the change stashed, so this is not laundering a pre-existing violation).
  • eslint reports 2 pre-existing import-extensions errors on lines 36–37, present identically on the base and on imports this PR does not touch; CI's lint script is eslint src, which does not cover tests/.

… was taken after the worker, so it could not answer its own question

`export-flows-and-agents.spec.ts` fails on `development` with

    the export job must finish — last status "queued";
    last worker pass: worker ok: (no output); job list: 112 total, 0 RunExportJob

and that message has been read as proof that the enqueue never happened. It is
not proof of anything. `runExportJobWorker()` took the census AFTER running
`background-job:worker`, and a QueuedJob is DELETED from `oc_jobs` once it
executes — so `0 RunExportJob` is what you see whether the job never existed or
ran to completion. The helper's own comment says the census exists to
disambiguate exactly those two cases; in that order it cannot.

- census taken BEFORE the worker pass, and both are reported;
- on a non-terminal status the failure now also reports the DEPLOYED exportJob
  schema's version and whether it carries `x-openregister-lifecycle`. Without
  that block OR's TransitionEngine finds no state machine and returns silently,
  the job row is consumed, and the object sits at "queued" with no log line —
  #219.

Read from the schema API, not the object: this OpenRegister build exposes no
`available-actions` on an object read at all (measured, including
`?_extend=all`), so probing for that key would report "absent" on a healthy
instance and read as evidence.

No assertion is relaxed and no skip is added; the failure only gets louder.
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ c905827

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
test-l10n
check-gitignore
check-nc-floor
format
composer ✅ 106/106
npm ✅ 626/626
app:check-code ⏭️
info.xml
REUSE
PHPUnit
Newman
Playwright
Hydra gates

Quality workflow — 2026-08-16 23:51 UTC

Download the full PDF report from the workflow artifacts.

Branch point 4ebc903 predates #229 ('fix(ci): green the E2E, Newman and Hydra
Gates jobs — at their real causes'). Judging this PR against post-#229
development read #229's inherited reds as INTRODUCED by a one-file test diff.
…ycle means now

Merging development brought in #229, which shipped `exportJob` 0.1.0 -> 1.1.0.
The probe's docblock and its failure string still told the reader that a missing
`x-openregister-lifecycle` meant #219 was unfixed. On the merged base that is no
longer true, and a comment that survives the change it describes is the half of
a diff git cannot check.

A MISSING reading now means the instance under test never converged onto the
bumped schema — a narrower fault than the one #219 described. The probe keeps
its value; only the conclusion it licenses changes.
@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

The two INTRODUCED checks are a moved base, not this diff — and neither of the two hypotheses fits

Rebased onto current development; the run above is the one to judge.

What happened. This PR branched from 4ebc9035. I captured that commit's own check-runs before the base moved — 42 rows, total_count == rows, nothing truncated:

4ebc9035  FAIL: quality / Quality Report
          FAIL: quality / Hydra Gates              ← "introduced"
          FAIL: quality / E2E Tests (Playwright)
          FAIL: quality / Integration Tests (Newman)  ← "introduced"

development then moved to 7002e4f8#229, "fix(ci): green the E2E, Newman and Hydra Gates jobs — at their real causes", merged 2026-08-16T23:39:17Z, i.e. after my branch point. So both checks were red on the commit I branched from and were repaired on development forty minutes later. Comparing a pre-#229 branch against post-#229 development reports #229's inherited reds as introduced by whoever has not rebased.

⚠️ This is the inverse of the usual stale-baseline trap, and it is nastier: the base did not drift into red, it was fixed. Every inherited red then reads as INTRODUCED against the newer base, and INTRODUCED: none — normally the trustworthy half — is exactly the half that stops being trustworthy.

The file lists settle it (per "diff the FILE LISTS before believing an attribution"). #229 touched 27 files; this PR touches 1, and the overlap is zero:

the complaint where #229 fixed it
[gate-66] openregister-dependency-shape: FAIL 13 × lib/ — the whole lib/Mcp/Handler/* set, OpenBuildToolProvider, ExportJobService, JobOwnerImpersonator. gate-66 inspects ADR-083 shape in lib/. My diff has no lib/ — your own instinct was right.
"Outsider sees the seeded app in NO list — IDOR closed" tests/integration/openbuild-rbac.postman_collection.json
"newman-added page is reflected" tests/integration/openbuild-page-editor.postman_collection.json
"Invalid manifest is rejected with 4xx" ApplicationsController.php + ApplicationsControllerSaveManifestTest.php
the two unknown-templateId 4xx-not-500 assertions openbuild-templates-marketplace.postman_collection.json

That is why the five failures "do not look like one defect" — they are not one defect, and they are not one bad instance either. They are five separate repairs I was missing, which is exactly the spread you would expect.

Both hypotheses tested and rejected, on their own terms:

  1. State leakage from my spec — structurally impossible, twice over. I audited every added line for a write verb: the diff adds exactly two server interactions, occ background-job:list (read) and GET /api/schemas (read). Zero POST/PUT/PATCH/DELETE. There is nothing to orphan. And Integration Tests (Newman) and E2E Tests (Playwright) are separate jobs, each provisioning its own Nextcloud — a Playwright spec cannot write into Newman's database at all. (The procest shape you cite needs a shared instance and a write; this has neither.)
  2. Flake / 504 — not needed, and it would have been the wrong answer. A rerun would have gone green and I would have logged "flaky", which is worse than the truth: these checks are deterministically red at 4ebc9035 and deterministically green after fix(ci): green the E2E, Newman and Hydra Gates jobs — at their real causes #229. Nothing was flaking.

No Newman assertion is relaxed. I did not touch a postman collection, and the IDOR assertion is untouched — on the rebased branch it is #229's passing version.

One thing the merge did change, and it is worth flagging. #229 shipped the bump I measured as necessary and deliberately declined to guess at: exportJob 0.1.0 → 1.1.0. That made my own prose stale — the docblock and the failure string still told the reader a missing x-openregister-lifecycle meant #219 was unfixed. Fixed in a753e24f: a MISSING reading now means this instance never converged onto the bumped schema, a narrower fault. A comment that outlives the change it describes is the half of a diff git cannot check, so it gets corrected rather than carried.

What to expect from the rebased run: gate-66 and Newman green (they are green on the base); E2E Tests (Playwright) still red at 189 · 3 · 65, unchanged — this PR relaxes nothing and adds no skip. Read its new failure message, not its cell.

@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/openbuild @ 4d56be8

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
check-manifest
test-l10n
check-gitignore
check-nc-floor
format
composer ✅ 106/106
npm ✅ 626/626
app:check-code ⏭️
info.xml
REUSE
PHPUnit
Newman
Playwright
Hydra gates

Quality workflow — 2026-08-17 01:02 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

Rebased run settled — parity, and a correction to my own prediction

All 42 rows, total_count == rows, nothing pending. Each row asserted to exist by name before its conclusion was read (a row that never ran is not a row that passed):

check at branch point 4ebc9035 rebased a753e24f
quality / Hydra Gates (gate-66) failure success
quality / Integration Tests (Newman) failure success
quality / E2E Tests (Playwright) failure failure — at parity, see below
quality / Quality Report failure failure (roll-up of the E2E cell; also red on the base run)

INTRODUCED: none. Both checks you flagged are green, with no lib/ touched and no postman collection touched — the IDOR assertion is untouched and passing.

E2E: parity proven by NAME, not by tally

collected passed failed skipped the failing test
base 7002e4f8 (the #229 merge) 257 191 1 65 version-rollback.spec.ts
PR a753e24f 257 191 1 65 version-rollback.spec.ts

Same test, by name — a matching tally alone would have been compatible with two different failures. That test is the documented flake #188 ("the version-history row resolves 37× and stays hidden"), and it is in neither my scope nor my diff.

⚠️ I predicted this wrong, and it changes this PR's rationale

I told you to expect E2E still red at 189 · 3 · 65. It is 191 · 1 · 65. #229 fixed both persistent failures I triaged — including export-flows-and-agents.spec.ts, the very one this PR was written to diagnose, and app-icon-management.spec.ts, whose spec file #229 also touched. My "these two are persistent, not DECISION-16" triage was correct about the cause and stale about the status: #229 landed the exportJob 0.1.0 → 1.1.0 bump while I was measuring, which is exactly the fix my probe was built to detect the absence of.

So be clear about what merging this now buys, because it is less than my earlier comment implied:

  • The export test passes today. The diagnostic is insurance against recurrence, not a live investigation aid.
  • What remains genuinely broken is the instrument itself. runExportJobWorker() took its job census after running the worker, and a QueuedJob is deleted once it executes — so 0 RunExportJob reads identically whether the enqueue never happened or the job ran to completion. That defect is real, is independent of fix(ci): green the E2E, Newman and Hydra Gates jobs — at their real causes #229, and would misreport the next time an export sticks. It is the reason job list: 112 total, 0 RunExportJob was read as proof of something it cannot prove.

If you would rather not carry a diagnostic for a green test, closing this is a defensible call and I will not argue it. My view is that a measuring instrument that cannot distinguish its own two hypotheses is worth fixing while we know why — but that is a judgement about cost, not a claim about correctness, and it is yours.

@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

Merging at parity. The INTRODUCED: E2E Tests (Playwright) on the latest measurement is the base moving, not a regression — and the evidence is by test name, not tally.

run commit contains #231? version-rollback.spec.ts:314:6 tally
PR a753e24f yes FAILED 191 · 1 · 65
base 7002e4f8 no FAILED 191 · 1 · 65
base 6401e1e8 no passed 192 · 0 · 65

The same test, at the same line — 'rolling back RESTORES the snapshot manifest onto the active version' — failed on a base commit that does not contain this PR, and passed on a later one. Same code, two outcomes: non-deterministic by construction. This PR's diff touches the export census only; it adds no versioning code, relaxes no assertion, adds no skip and raises no timeout. Quality Report is a roll-up of E2E and follows it.

Green on this head, asserted to exist by name before reading the conclusion: Hydra Gates success, Integration Tests (Newman) success — the latter from five failures including the IDOR one, with zero postman collections touched.

Why merge despite the export test now passing. #229 landed the exportJob 0.1.0 → 1.1.0 bump mid-triage, so the diagnostic this PR adds is insurance rather than a live aid — that correction is recorded and it is a fair argument for closing. What justifies merging is the other half: the census was taken after the worker, and a QueuedJob is deleted once it executes, so 0 RunExportJob cannot distinguish 'never enqueued' from 'ran to completion' — the two states it exists to tell apart. That instrument defect is independent of #229 and would misreport the next stuck export.

@rubenvdlinde
rubenvdlinde merged commit d85a036 into development Aug 17, 2026
40 of 42 checks passed
@rubenvdlinde
rubenvdlinde deleted the S49/e2e-triage branch August 17, 2026 01:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants