fix(benchmark): check the MockMed encounter type as its own field

[abrichr]

The defect

verify_encounter_saved is the arm-independent success criterion for the
compiled-vs-agent MockMed benchmark. Both arms are judged by it, so the
comparison between arms is unbiased — but every absolute rate it has
produced is too high.

It matched the saved encounter row as one fuzzy candidate string,
<type> — <note[:60]>, at min_ratio = 0.8. One similarity ratio over that
concatenation is decided by whichever field contributes more characters, so a
long correct note outvotes a short wrong type.

Measured on a retained MockMed frame (E4 precondition run, episode 4 of 20):
the agent saved the correct note under encounter type Consult instead of the
requested Triage, and the check passed it.

expected : 'Triage — Follow-up in 2 weeks; BP recheck.'
actual   : 'Consult — Follow-up in 2 weeks; BP recheck.'
ratio    : 0.8471   >=  min_ratio 0.8   ->  MATCHED

Verifying the defect against the actual screenshot found a second,
independent path that the original write-up did not name, and it is the
dominant one. The row check's fallback candidate was the bare
note_text[:BANNER_NOTE_CHARS] — a candidate with no type in it at all.
rapidocr segments MockMed's banner into Encounter saved - plus its own note
line, and that note line satisfies the fallback outright. On the episode-4
frame:

candidate	best line	ratio
Encounter saved — (banner)	'Encountersaved-'	0.8750
Triage — <note> (row)	'Follow-upin2weeks;BPrecheck.'	0.8000
<note> (row fallback)	'Follow-upin2weeks;BPrecheck.'	0.9180
—	'·Consult—Follow-up in2weeks;BPrecheck.' (the real row)	0.7750

Why not raise the threshold

There is no threshold that works. On that frame the false-positive
evidence (0.918) is stronger than the true evidence the check requires
(0.875 for the banner). Swept over all 61 retained trajectories:

min_ratio	TP	FP	TN	FN
0.78 – 0.86	27	2	32	0
0.88 – 0.98	0	0	34	27

Both false positives survive to 0.86; at 0.88 every true positive collapses
into a false negative. The threshold has no separating value.

Removing only the type-free fallback is also not sufficient: the row
candidate still scores 0.8000 on that frame, exactly at the threshold, off a
line that contains no type.

The fix

Structural, not a threshold nudge — each field is judged on its own terms.

• The row line is parsed into its type field and its note field
  (split at the first dash-like character, plus a positional split per known
  type so a dropped separator still parses).
• The type field is categorical: it must name exactly one member of the
  application's enumeration ENCOUNTER_TYPES = ("Triage", "Consult"), and
  that member must be the requested type. A wrong Consult can no longer be
  absorbed by a right note body, at any note length.
• The note field is free text and keeps the unchanged min_ratio
  fuzzy tolerance. Nothing about free-text matching is tightened.
• Both fields must pass on the same parsed candidate.
• A row carrying this run's note under a different known type now sets a new
  wrong_type_row field and fails the run — reporting silent wrong-action, as
  the reliability standard in AGENTS.md §3 requires.

This converges verify_encounter_saved on the discipline
verify_hybrid_final already uses in this same package: that function
scores the type and the note as separate line-level evidence and reports
wrong_type_row, and its docstring already names this exact failure mode.
The MockMed benchmark's criterion was simply never brought along.

Validation — both directions, all 61 trajectories

The regression set is the 61 retained E4 trajectories with screenshots: 40
carry an independent in-page record-oracle label (window.state.encounters
read via page.evaluate — bypasses rendering, OCR and fuzzy matching), and 21
are labelled from the rendered encounter list.

	TP	FP	TN	FN
before (origin/main)	27	2	32	0
after	27	0	34	0

Exactly two episodes change verdict, and both are the known-bad one (the
same seed in both 7B passes). Every known-good episode still passes. There is
no false negative in either direction.

Margins on the 27 true positives, against a 0.8 floor:

field	min	median	max
type	1.0000	1.0000	1.0000
note	0.9000	0.9000	0.9000

And the free-text tolerance is provably unchanged: every one of the 27 true
positives already satisfied the <type> — <note> row candidate at 0.8974, so
dropping the type-free fallback costs no genuine success on this corpus.

Corrected numbers, with denominators

Denominator n = 20 episodes per model per oracle, pre-registered before any
episode ran and never adjusted.

model	oracle	before	after
Qwen2.5-VL-7B	screen (pre-registered pass)	14/20	13/20
Qwen2.5-VL-7B	screen (confirmatory pass)	14/20	13/20
Qwen2.5-VL-7B	in-page record	13/20	13/20 (unchanged)
Qwen2.5-VL-3B	screen and record	0/20	0/20 (unchanged)

Episode 4 moved in both 7B passes. The corrected screen oracle now agrees
with the independent record oracle at 13/20 in both, which strengthens rather
than weakens the run's replication claim.

Tests

Both directions at four note lengths (11 / 22 / 33 / 60 characters), spanning
the band where the old ratio crosses the threshold — 0.7317, 0.8254, 0.8706,
0.9209 against the expected Triage row form. The benchmark's own 32-character
note sits inside the missed band.

• test_right_type_passes_at_every_note_length — the over-halt guard. A
  check strict enough to trip on OCR noise would make a working system look
  broken, which is worse than the over-count it replaces.
• test_wrong_type_fails_at_every_note_length — the regression.
• test_wrong_type_row_beside_right_row_fails
• test_bare_note_line_is_not_an_encounter_row — the second defect path.
• test_unlisted_requested_type_is_still_checked

Mutation-checked: with the fix reverted (old row logic restored, result
fields kept so the failures are behavioural rather than AttributeError),
8 tests fail, including all four wrong-type note lengths. Restored, all pass.

Scope

verify_encounter_saved has exactly one production caller,
benchmark/run_benchmark.py (the MockMed compiled-vs-agent benchmark).
The OpenEMR benchmark uses verify_note_saved, a different criterion with no
categorical field, and is not affected by this defect; it is unchanged
here. verify_hybrid_final was already correct and is unchanged.

VerifyResult gains wrong_type_row with a default, so existing rows in
benchmark/results.json still parse.

normalize_text is now exported from openadapt_flow.vision (it was already
public in vision.ocr).

🤖 Generated with Claude Code

https://claude.ai/code/session_01NyCHrzA1psrKMFfroYbzaM

[abrichr]

Launch-train status (2026-07-28): this PR remains intentionally held behind the exact fresh-frame/OpenEMR campaign gate. Development continues in parallel; this is only a merge-order constraint on Flow main. Before merge, the corrected benchmark verdicts need the independent oracle and artifact-manifest review recorded in the release plan, followed by one fresh-main rebase and one authoritative exact-head matrix. No repeated CI reruns are needed meanwhile.