ci(inferno): prune omit entries that suppress nothing, and report new ones

[aacruzgon]

Stacked on #496, which is stacked on #495. Review those first; this PR's diff is the workflow file alone.

The ordering is load-bearing, not cosmetic: the 12 data_absent_reason entries removed here fail on Postgres until the bare-id reference fix in #495 lands. Merged ahead of it, the next weekly Inferno run would go red on the Postgres legs. The stack enforces that rather than relying on a note.

Summary

OMITTED_TESTS drops test ids from the US Core failure count. An entry that suppresses nothing is worse than no entry at all: it is a test that would go red on a real regression and be silently swallowed instead. #492 found 20 of the 69 entries in that state.

This prunes those 20 and adds a step that reports new ones every run, so the list shrinks as things get fixed rather than accreting.

Refs #492.

Changes

Removed — verified passing on two backends

entries	what	evidence
12	data_absent_reason_extension + _code_system, all six suites	pass on sqlite (#492's audit) and on postgres (six local runs)
3	practitioner_address_test in v6.1.0 / v7.0.0 / v8.0.0	pass in exactly the three suites it was emitted for

The data_absent_reason entries were the ones blocked on #490 — they failed on Postgres until the bare-id reference fix. Their stated reason ("test data does not include DAR extensions/code systems") no longer holds.

Scoped rather than removed — 5 ids the suite never contained

These came from the unconditional block, which assumed every suite has the full capability-statement set and a simple_observation group:

id	was	now
us_core_profile_support	all six	v4.0.0+ (absent from v3.1.1)
us_core_instantiate	all six	v4.0.0+ (absent from v3.1.1)
simple_observation_validation_test	all six	v6.1.0+ (group introduced there)

Deleting them would have removed live omissions in the suites where they do fail, which is why #492 called for scoping.

Added — "Report omissions that suppressed nothing"

Reports two shapes per run, as job-summary sections plus ::warning:: annotations:

• stale — the id ran and did not fail, so the omission changed nothing
• absent — the id is not in this suite at all, so it never could

Deliberately a warning, not a gate. Two reasons, both in the step's comment:

1. Failing on a stale omission would make a fixed test turn CI red — the backwards incentive that grew this list.
2. It cannot gate safely anyway. The list is shared across all five backend legs, and an id that passes on one backend may still be suppressing a real failure on another. Pruning needs agreement across the matrix, which a per-job gate cannot see — the job summary can.

Testing

A workflow change can't be proved by a unit test, so this was validated by extracting the actual run: blocks from the YAML and executing them under bash -e against the six suites' real result JSON (the runs behind the numbers in #492). Not a transcription of the logic — the shipped bash, with ${{ matrix.suite_id }} substituted and $GITHUB_ENV / $GITHUB_STEP_SUMMARY pointed at temp files, exactly as Actions wires them.

• All six suites: both steps exit 0. OMITTED_TESTS round-trips through $GITHUB_ENV and parses as valid JSON in every suite (3 / 5 / 7 / 9 / 11 / 14 entries for v3.1.1 → v8.0.0).
• Zero ABSENT entries remain in any suite — the scoping is complete.
• The detector fires. Feeding the five removed ids back in reports all five as ABSENT, while a genuinely-present id in the same list is correctly not reported. A detector that silently returns nothing looks identical to a clean run, so this was worth pinning.
• Both edge branches exercised. Missing results.json (an early-run failure, reachable via if: always()) prints a notice and exits 0; a list where every omitted id genuinely failed writes the :white_check_mark: summary and exits 0. Neither fails the job.
• The rendered job summary was eyeballed for v6.1.0 — the artifact a human actually reads.
• actionlint delta against main is clean: no new warnings or errors. It adds 4 SC2086 (the >> $GITHUB_STEP_SUMMARY form used by all 50 pre-existing instances in this file) and 4 SC2001 (sed 's/^/- /' prefixing multi-line output, which parameter expansion cannot do). Both are info/style and consistent with the file.

It already found 18 more

Running the new audit against the six suites surfaces 18 further stale candidates among the 49 live omissions this PR does not touch:

suite	stale omissions found
us_core_v311	1
us_core_v400	1
us_core_v501	3
us_core_v610	5
us_core_v700	4
us_core_v800	4

Mostly service_request, observation_pregnancyintent/pregnancystatus, smokingstatus, specimen and observation_adi_documentation validation tests. Not pruned here — that is Postgres-only data, and per the reasoning above an id passing on Postgres may still be suppressing a real failure on MongoDB or Elasticsearch. Surfacing them across the matrix is precisely what the new step is for; the next full nightly will say which are safe.

Notes

• One implementation detail worth flagging for reviewers: the new step passes the results filename to jq rather than echo "$RESULTS" | jq. The results carry \n inside result_message, and an echo that interprets escapes expands them mid-string and makes the JSON unparseable. Bash's builtin echo does not do this, so the existing steps are fine on GitHub runners — but the filename form removes the footgun entirely, and I hit it locally under zsh while testing.
• The 49 live omissions remain unexamined by this PR. Each was added for a stated reason and deserves the same re-check, now much cheaper thanks to the new step.
• No change to gate semantics: Check test results still fails only on fail/error, and still excludes omitted ids. ci(inferno): the US Core gate ignores skip — postgres passed 25 of 544 tests and reported success for six weeks #491 covers the skip-blindness separately.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!