Skip to content

Cross-check the capture-watch preflight against the live camera selection - #705

Merged
xmap merged 2 commits into
mainfrom
feat/capture-watch-camera-prefix-check
Aug 21, 2026
Merged

Cross-check the capture-watch preflight against the live camera selection#705
xmap merged 2 commits into
mainfrom
feat/capture-watch-camera-prefix-check

Conversation

@xmap

@xmap xmap commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Two independent commits, both prompted by the 2026-08-20 disk-full incident at APS 2-BM.

1. Camera-prefix cross-check in the capture-watch preflight.

capture_watch_pvs's full_file_name role is a hardcoded PV string carrying one camera's prefix, and nothing makes it follow 2bm:MCTOptics:CameraSelect. During the incident the operator switched from camera 2 to camera 1, which would leave that role reading the idle camera's stale filename readback. That value is exactly what RunWitnessRecorder vaults into run_capture_path as the Run's capture path, and it is personal data.

The preflight gains one report line, camera_prefix_check, emitted only when a code declares both full_file_name and a new optional camera_selected role. It reuses readings the existing sweep already took (no second control_port.read()) and never reports a clean match on anything it cannot confirm: an unreadable PV, an empty mapping table, and an unrecognized reading are each their own distinct bad verdict.

The reading-to-prefix mapping is a deployment-declared table (Settings.capture_camera_select_prefixes), not hardcoded. CORA does not know whether the live CameraSelected resolves to a bare index or an ENUM label, and must not guess. The camera-index-to-prefix fact itself is confirmed by deployments/2-bm/beamline.yaml (DET-11, operator-verified 2026-06-19); the wire format of the readback is left for beamline staff to confirm when they configure the role.

Live risk today is zero because run_witness_recording_enabled is off at 2-BM. That is precisely why this lands before it is turned on.

2. DATA-8 gains a third data point.

The row asks whether dropped frames are rare-and-alarming or routine. Its two existing data points are both files that closed cleanly. The 2026-08-20 scan is a different shape: the disk filled mid-acquisition, the saved-frame counter fell behind the collected-frame counter, and the file it left cannot be counted at all. It opens, but reading the frame-index dataset raises an HDF5 address-overflow error with end-of-allocation at 2048 bytes.

That matters for the answer the row is gathering evidence toward. A shortfall check modelled only on "fewer frames than commanded" cannot express this outcome, because there is no readable count to compare against.

Deliberately not touched: the storage-tier account in operations.md and the HOST-2 row. Both were already corrected on main (#684, #696), and their current text, that the write target is an operator-settable PV re-measured to /local1 on 2026-08-19 with older scans still on /local2, is better sourced than anything this incident adds. An operator recollection that both tiers are in concurrent use is worth confirming with the beamline before it is written down as fact.

Testing. 1,011 tests pass (preflight unit tests plus the full deployments tier, including the open-questions alignment guard). The camera tests were rewritten during review: all five originally configured a placeholder PV whose prefix matched nothing in the table, which made the match case fail and, worse, made the mismatch case pass regardless of which camera was selected. They now configure a real camera-prefixed PV so match and mismatch differ only in the camera reading.

xmap and others added 2 commits August 21, 2026 07:49
…he live camera selection

On 2026-08-20 an operator's camera switch at 2-BM left the configured
full_file_name PV (hardcoded to one camera's prefix) reading the idle
camera's stale filename readback. That value is exactly what
RunWitnessRecorder vaults into run_capture_path as the Run's capture
path, which is personal data. The live risk is currently zero because
run_witness_recording_enabled is off at 2-BM, which is exactly why now
is the right time to close the gap before it is turned on.

The preflight gains one more report line, camera_prefix_check, only
when a code declares both full_file_name and a new optional
camera_selected role in capture_watch_pvs: it reuses the readings
already taken by the existing sweep (no second control_port.read()),
and never reports a clean match on anything it cannot actually
confirm, an unreadable camera_selected PV, an empty
capture_camera_select_prefixes table, or a reading the table does not
resolve are each their own distinct bad verdict.

The reading-to-prefix mapping is a deployment-declared table
(Settings.capture_camera_select_prefixes), mirroring
capture_status_phases, because CORA does not know, and must not
guess, whether the live CameraSelected PV resolves to a bare index or
an ENUM label; camera-index-to-prefix itself is confirmed via
deployments/2-bm/beamline.yaml (DET-11, operator-verified 2026-06-19),
but the wire format of CameraSelected's own reading is not yet
confirmed against the live IOC and is left for beamline staff to
verify when they configure the new role.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The camera tests configure a REAL camera-prefixed PV name for
full_file_name, not a placeholder: with a placeholder the match case
cannot pass and, worse, the mismatch case passes for the wrong reason,
since a prefix absent from the table mismatches whatever camera is
selected. Match and mismatch now differ only in the camera reading.
DATA-8 asks whether dropped frames are rare-and-alarming or routine, and
its two existing data points are both files that closed cleanly: one
early smoke test with a near-total shortfall, one production scan with
none. The 2026-08-20 scan is the first of a different shape. The disk
filled mid-acquisition, the saved-frame counter fell behind the
collected-frame counter, and the file it left behind cannot be counted
at all: it opens, but reading the frame-index dataset raises an HDF5
address-overflow error.

That matters for the answer the row is gathering evidence toward. A
shortfall check modelled only on "fewer frames than commanded" cannot
express this outcome, because there is no readable count to compare
against. An unreadable file has to be its own result, not a zero.

Only the DATA-8 row changes here. The storage-tier account on
operations.md and the HOST-2 row are deliberately untouched: both were
already corrected on main (#684, #696), and their current text, that
the write target is an operator-settable PV re-measured to /local1 on
2026-08-19 with older scans still on /local2, is better sourced than
anything this incident adds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  apps/api/src/cora/api
  capture_watch_preflight.py 467
  apps/api/src/cora/infrastructure
  config.py
Project Total  

This report was generated by python-coverage-comment-action

@xmap
xmap merged commit 5893da8 into main Aug 21, 2026
19 checks passed
@xmap
xmap deleted the feat/capture-watch-camera-prefix-check branch August 21, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant