Add .edf and .gb (General Binary) adapters for ALS BL7.3.3

[Wiebke]

Builds on the initial adapter skeleton to add two working file adapters for the PILATUS 2M detector at beamline 7.3.3:

• EDFAdapter: reads .edf image files via fabio, merging EDF header fields with metadata from a companion .txt file
• GeneralBinaryPilatus2MAdapter: reads stitched .gb float32 images, combining metadata from paired _hi and _lo EDF companions; selects the later acquisition date
  Both adapters subclass ArrayAdapter.

Includes round-trip integration tests (register, then read via tiled client) and unit tests for metadata parsing and merging logic. Test fixtures use synthetic data derived from real example files, with PI names anonymized.

Requires pip install ".[bl733]" for fabio.

To use the example config, populate a folder .data with .edf .gb and compainion .edf files and run:

tiled serve config ./example_configs/bl733/config.yaml 

and

tiled register <tiled_uri> --verbose \
            --ext '.edf=application/x-edf' \
            --adapter 'application/x-edf=als_tiled.bl733.adapters.edf:EDFAdapter' \
            --ext '.gb=application/x-gb' \
            --adapter 'application/x-gb=als_tiled.bl733.adapters.gb:GeneralBinaryPilatus2MAdapter' \
            ./data

Some considerations:

• Side-car .txt and .edf files are not registered as explicit data sources. This simplifies registration as a single file can serve as the "trigger" to register a scan and no custom detection of mimetime is needed, as the file ending directly translated to the mimetype). This does mean that built-in download capabilities in Tiled will not include these additional files. We have two option to make native download possible: A custom exporter that exports the full set of side-car, or switching to combined mimetypes (edf+txt), (gb+2txt+2edf). test_image_file_with_sidecar_metadata_file
• Tests are inspired by round-trip registration tests for custom formats: bluesky/tiled/tests/test_custom_format.py
• Current core Tiled adapters for images subclass either ArrayAdapter (e.g. bluesky/tiled/adapters/jpeg.py) or Adapter[ArrayStructure] (e.g. bluesky/tiled/adapters/npy.py. I choose ArrayAdapter as Adapter[ArrayStructure] versions seem to be earmarked for refactoring, see Adapter Rationalisation bluesky/tiled#1066

[dylanmcreynolds]

Can you please add an __init__.py in the bl733 folder?

>>> import als_tiled.bl733.edf
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'als_tiled.bl733.edf'

[Wiebke]

__init__.py was indeed missing from als_tiled/bl733/ and als_tiled/bl733/adapters/. They have been added. The ModuleNotFoundError for als_tiled.bl733.edf is correct though: the adapter lives at als_tiled.bl733.adapters.edf.

The adapters/ subdirectory is intentional to leave room for als_tiled.bl733.exporters/ and similar additions later. Happy to flatten if a simpler structure is preferred.

[dylanmcreynolds]

Sorry, misread the structure in an IDE that flattens empty dirs. I like the adapters and exporters packages

[Wiebke]

Since the initial review comments the following changes were made:

• Updated tiled to the most recent release 0.2.8 (base image and core dependency)
• Added .mypy.ini with per-module strictness (strict on adapters, error-checking on tests, loose elsewhere)
• Updated pre-commit hook versions and added mypy. The precommit
• Test dependencies now pull in bl733 and tiled-all via als_tiled[bl733,tiled-all] so pip install ".[test]" installs everything needed
• Added test for GB registration when companion EDF files are missing

[Wiebke]

Some additional notes:

This builds on previous work in mlexchange/workflow-viz, in particular also mlexchange/workflow-viz/pull/2 by @rajasriramoju

fabio.open is an alias to fabio.openimage.openimage which in my current understanding eagerly reads data. This is in contrast to Pillow's Image.open or numpy.load for .npy files.
This lazy loading enables a different initialization scheme for ArrayAdapter that is not followed here.

[dylanmcreynolds]

I left a comment about logging that is pretty unimportant, happy to merge without.

[dylanmcreynolds]

I think I'd like to used the unused logger here to printout what file is being processed.

[Wiebke]

Added a logger.debug call.

[dylanmcreynolds]

What an important Type hint :) . Perhaps we should remove this file.

[dylanmcreynolds]

This might be silly optimization, but what if we define the string in the module and only replace it here?

Also, there so reason to replace the string if debug level isn't set. What about

if logger.getEffectiveLevel(logging.DEBUG):
    logger.debug(LOADING_MESSAGE, filepath)

I wouldn't bother but I think this is code that gets called a lot of times in a server.

[Wiebke]

Constants sound like a good idea, I moved the strings to module-level LOADING_MESSAGE for both edf and gb.

For the getEffectiveLevel check: Logging is already doing lazy evaluation for string replacement guarded by isEnabledFor. That type of check seems to be mostly recommended when the calculating the string to be replaces is expensive (according to https://docs.python.org/3/howto/logging.html#optimization). Since we're just passing filepath here, I would think we are fine without the extra guard?

[Wiebke]

Closes #1.