Skip to content

Add DESY P03 detector adapters (CBF + Lambda NeXus) - #12

Open
Wiebke wants to merge 7 commits into
mainfrom
desy-p03-adapter-walker
Open

Add DESY P03 detector adapters (CBF + Lambda NeXus)#12
Wiebke wants to merge 7 commits into
mainfrom
desy-p03-adapter-walker

Conversation

@Wiebke

@Wiebke Wiebke commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Adds two adapters for DESY beamline P03 detectors, plus the packaging/CI to build and test them:

  • CBFAdapter: reads .cbf files via fabio, merging the CBF header into metadata. Subclasses ArrayAdapter, same pattern as the existing bl733 adapters.
  • LambdaDetectorNexusAdapter + walk: the Lambda detector splits each frame across up to 11 module NeXus files; the walker groups a scan's module files into one node, and the adapter stitches them into a single array using each module's own translation/distance offset. Applies flatfield correction (skipped if already applied upstream), and supports an opt-in pad_to_detector_size to guarantee the full detector extent even when a module was switched off mid-scan.

Tests cover both adapters via round-trip registration + read through the tiled client, plus unit tests for stitching, flatfield, and frame-count logic. Fixtures are synthetic, sized down from the real module geometry.

Requires pip install ".[desy_p03]" (fabio, h5py, ndindex), or pixi run -e desy-p03 ....

To use the example config:

tiled serve config ./tiled_config_desy/config.yaml

The Lambda walker is not picked up by tiled serve config (no walkers slot there) -- registering Lambda data needs an explicit tiled register call:

tiled register <tiled_uri> --verbose \
    --adapter 'application/x-cbf=splash_tiled.external.desy_p03.adapters.cbf:CBFAdapter' \
    --adapter 'multipart/related;type=application/x-hdf5=splash_tiled.external.desy_p03.adapters.lambda_nxs:LambdaDetectorNexusAdapter' \
    --walker 'splash_tiled.external.desy_p03.adapters.lambda_nxs:walk' \
    ./data

Some considerations

  • Module offsets come from each file's own translation/distance, not a fixed table -- a switched-off module writes no file at all, and the remaining files get renumbered, so filenames can't identify which module is missing. pad_to_detector_size is opt-in rather than automatic since a missing module's geometry can't be safely reconstructed from data alone; the caller has to say which known unit ("2M"/"9M", or an explicit shape) applies.
  • Frame count is the minimum across all modules, not just the first file's -- real scans have been observed where one module logs one extra frame; trusting a single module's count risks an out-of-range read on a shorter one.
  • pixel_mask's bit meanings are currently not known, bit 31 looks like an inter-chip gap marker, others are likely hot pixels etc. apply_mask defaults off.

Wiebke and others added 7 commits August 18, 2026 03:12
Adds a desy-p03 pixi environment/feature (fabio, h5py, ndindex),
a desy_p03 pyproject extra, a Containerfile.desy and minimal
tiled_config_desy/config.yaml for the DESY deployment, and a
build-and-push-desy CI job.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Covers the CBF and Lambda NeXus adapters added earlier: header/data
reading for CBF, and module registration, stitching, single-frame
scans, flatfield application, and zero-frame scans for the Lambda
walker/adapter.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
When a physical Lambda module is switched off mid-scan, no file is
written for it at all, and the files that ARE written get renumbered
sequentially -- so a written file's name says nothing about which
physical module it is, and nothing on disk reveals whether the
missing module was the one defining the detector's full extent.
Without any signal of this, assembled_shape silently shrinks instead
of leaving a fill_value gap the way a missing middle module does.

Adds an opt-in pad_to_detector_size constructor parameter (by name,
via LAMBDA_KNOWN_DETECTOR_SIZES, or as an explicit (dim0, dim1)) that
guarantees the assembled shape covers the true full detector
regardless of which modules are present. from_uris now forwards
kwargs so this (and other existing options) can actually reach the
adapter.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
num_frames was taken only from the first module file. Real scans have
been observed where one module logs one more frame than the others;
trusting a single module's count risks asking a shorter module for a
frame index it doesn't have, raising IndexError. Use the minimum
across all modules instead, and add a regression test covering the
vulnerable ordering (the first-opened module has the higher count).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Shortens the fixture's real-data-survey comment block (no need to
enumerate exact scan/file counts and per-pixel gap ranges), and fixes
grammar slips left over from an earlier trim. Also corrects
MASK_BAD_THRESHOLD's comment: it was described as an arbitrary "magic
number", but real data shows it isolates a specific pixel_mask flag
(bit 31) from the other, still-undocumented codes -- not arbitrary,
just unconfirmed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
tiled_config_desy/ as a whole new top-level folder for one file was
unnecessary -- it's not referenced by the CI build or Containerfile.desy
(config gets mounted at runtime, not baked into the image), so this is
a plain rename.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant