SQM enhancements by mrosseel · Pull Request #374 · brickbots/PiFinder

mrosseel · 2026-01-26T14:38:35Z

Summary

Comprehensive improvements to the Sky Quality Meter (SQM) module for more accurate and reliable sky brightness measurements.

Changes (vs upstream/main)

Core SQM (sqm/sqm.py) — 153 lines changed

Use static bias_offset as pedestal instead of adaptive noise floor estimation
Load per-camera calibration from ~/PiFinder_data/sqm_calibration_{camera_type}.json
Use ASTAP extinction convention: 0.28 * (airmass - 1), no fixed baseline
Exclude saturated stars from mzero calculation (flux set to -1)
Use flux-weighted mean for mzero
Improved centroid coordinate documentation (row, col = y, x matching Tetra3)

Sweep tool (ui/exp_sweep.py → ui/sqm_sweep.py) — renamed + 119 lines changed

Merged sqm_correction.py into sqm_sweep.py (single tool for validation sweeps)
New save_sweep_metadata.py for persisting sweep results
Reduced exposure sweep from 100 to 20 images
Added bracketed exposures to correction package
Real-time progress UI during sweeps

SQM UI (ui/sqm.py) — 37 lines changed

Show star count in SQM view (moved to top line)
Simplified Bortle class display
Added exposure time display

Calibration UI (ui/sqm_calibration.py) — 45 lines changed

Fix calibration text: "Keep cap on..." for bias/dark captures
UX: Show "Saving..." during correction save

Callbacks (ui/callbacks.py) — 6 lines changed

Fix missing right argument in SQM MarkingMenu

Camera profiles (sqm/camera_profiles.py) — 4 lines changed

Minor profile adjustments

Auto-exposure tests (tests/test_auto_exposure.py) — 34 lines changed

Update tests for asymmetric PID gains API (gains_decrease/gains_increase replacing single gains tuple)

SQM tests (tests/test_sqm.py) — +623 lines

Comprehensive unit test suite covering SQM calculations, photometry, extinction, calibration loading, and edge cases

Diff stat

 python/PiFinder/sqm/__init__.py                   |  10 +-
 python/PiFinder/sqm/camera_profiles.py            |   4 +-
 python/PiFinder/sqm/save_sweep_metadata.py        |  18 +
 python/PiFinder/sqm/sqm.py                        | 153 ++++--
 python/PiFinder/ui/callbacks.py                   |   6 +-
 python/PiFinder/ui/sqm.py                         |  37 +-
 python/PiFinder/ui/sqm_calibration.py             |  45 +-
 python/PiFinder/ui/{exp_sweep.py => sqm_sweep.py} | 119 ++++-
 python/tests/test_auto_exposure.py                |  34 +-
 python/tests/test_sqm.py                          | 623 +++++++++++++++++++++-
 10 files changed, 887 insertions(+), 162 deletions(-)

Test plan

Run pytest -m unit — verify all tests pass
Run SQM measurement on device with IMX296 camera
Verify calibration wizard saves/loads correctly
Confirm extinction correction matches ASTAP values
Test sweep tool end-to-end

🤖 Generated with Claude Code

This reverts commit c872394.

- Show 'Saving...' message immediately before creating zip - Remove +2 margin from noise floor - target exact value for shortest exposure Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Includes: mzero, mzero_std, background, pedestal, extinction, star counts, saturation info, etc. - everything needed for calibration validation. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Captures 3 exposures: base, +1 stop, -1 stop (or base, -1, -2 if at max). Shows progress for each bracket. Re-enables auto-exposure when done. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Faster sweep (~1 min vs ~4 min) while still sampling exposure space. Launch from SQM view → hold square → DEBUG. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Brighter stars have higher SNR so their mzero estimates are more reliable. Weight each star's contribution by its flux. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Without margin, background_corrected ≈ 0 when at noise floor, causing log10(~0) = very negative, making SQM too high. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…tion Two fixes for SQM being ~0.4 mag too high: 1. Use only bias_offset (6.0 ADU) as pedestal instead of full noise_floor (7.46 ADU). Read noise and dark current are random fluctuations, not systematic offsets that should be subtracted. 2. Disable atmospheric extinction correction. When comparing to ground-based SQM meters, both measurements are through the same atmosphere, so no correction is needed. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Use 0.1 mag/airmass to match typical consumer SQM meters instead of the scientific V-band value (0.28). This enables comparing measurements at different altitudes while staying consistent with reference meters. At zenith: +0.10 mag At 45°: +0.14 mag At 30°: +0.20 mag Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…ased for science - Main SQM value uses fixed 0.1 mag extinction (matches consumer SQM meters) - Also calculate altitude-dependent extinction (0.1 mag/airmass) - Display altitude-corrected value on SQM view in small font ("alt: XX.XX") - Enables comparing measurements at different altitudes scientifically Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Main SQM value: raw measurement (no extinction correction) - Altitude-corrected: adds 0.28*(airmass-1) per ASTAP convention - Zenith is reference point (extinction=0), only additional extinction below zenith is added for altitude comparison At different altitudes: - Zenith (90°): extinction = 0 - 45°: extinction = 0.115 - 30°: extinction = 0.28 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Simplify SQMCalculator API by removing unused parameters - Add robust Cedar-to-tetra3 fallback when centroid extraction fails - Fix read noise calculation to use temporal variance (avoids PRNU) - Add calibration indicator (CAL/!CAL) to SQM display - Expand test coverage for SQM functionality Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Fix PID integral crash: Reset integral when error changes sign (prevents -487,500µs crash when going from too many → too few stars) - Start sweep at 400ms instead of 25ms for faster recovery - Add sleep mode logging to diagnose camera blocking issues - Revert default to PID (SNR exposures too long on clear nights) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Tests now reflect the new sweep pattern that starts at 400ms instead of 25ms for faster recovery. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Move inline imports to file tops per project convention: - camera_profiles.py: numpy import - test_sqm.py: NoiseFloorEstimator import - Add unit tests for previously untested functions: - SQM._measure_star_flux_with_local_background() - get_camera_profile(), detect_camera_type() - CameraProfile.crop_and_rotate() - save_sweep_metadata() - Add Pickering airmass test for 5° altitude - Improve __init__.py docstrings - Document centroid (row, col) convention in docstrings Test coverage: 37 → 65 tests, ~92% coverage Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- SQM now uses bias_offset directly from camera profile (no NoiseFloorEstimator at runtime) - Load calibration from ~/PiFinder_data/sqm_calibration_{camera_type}.json on init - Add indoor calibration mode: skip sky frames in calibration wizard (press 0) - Combine exp_sweep and sqm_correction into single sqm_sweep tool - Add full NoiseFloorEstimator output to sweep/correction metadata - Add read_noise, dark_current to SQM details for diagnostics - Menu: CAL (indoor calibration), CORRECT (sweep with full metadata) - Fix bias_offset comment: measured from bias frames, not tuned to SQM-L Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Simplify the PID controller by removing: - Rate limiting (update_interval, _last_adjustment_time) - Asymmetric gains (gains_decrease/gains_increase → single gains tuple) - Unused time import The PID now responds immediately to all errors with uniform gains. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

MarkingMenu dataclass requires down, left, and right arguments. Added empty right option to fix TypeError on SQM screen launch. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Change "Hold steady..." to "Keep cap on..." during bias and dark frame captures, since the telescope cap should be on during these steps. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Non-SQM files: took main's version (solver, state, base UI, camera_interface). SQM files: kept wt-sqm's static bias_offset pedestal (vs adaptive noise floor), calibration file loading, and simplified PID gains. Kept main's sqm_correction.py (discarded sqm_sweep.py). auto_exposure.py taken from main. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Tests referenced old single `gains` parameter but ExposurePIDController now uses `gains_decrease`/`gains_increase` for asymmetric PID control. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

mrosseel and others added 30 commits October 21, 2024 20:48

Adding first version of sqm measurement

a035b80

Added UI screen, some refactoring for jupyter debugging

e7fa2a1

Fix crash on no solution

f1f9b2f

No shared_state crash

5eebb12

Protect against various crash situations

34dc9e4

Swapping centroid y/x

6e111c7

Added bortle classes, radius=3

8429089

Now try radius=2

d7d962e

Raw output for camera in shared state

d8674d9

Introducing bias image

76b26a6

feeding linear image through pipeline

eef32af

Merge remote-tracking branch 'origin/raw_output' into sqm

439d012

Delete and ignore comets file, sqm fixes

d4d5b29

Fix bias image bug

4888e4f

Missing bias image arguments

8dabba8

Another raw_image mention removed

b75f52e

Add debugging output

4b61770

Merged main with the location changes

95d13c5

no ubx yet

c872394

Merge branch 'main' of github.com:brickbots/PiFinder

51f14f4

Revert "no ubx yet"

0ec4de8

This reverts commit c872394.

Merge remote-tracking branch 'origin/main'

bbcad5c

ZMerge branch 'main' of github.com:mrosseel/PiFinder

fd44825

Merge remote-tracking branch 'origin/main'

64f8401

Merge remote-tracking branch 'origin/main'

b46636f

Merge remote-tracking branch 'origin/main'

77f79bd

Merge branch 'main' of github.com:brickbots/PiFinder

30fa0dc

Merge remote-tracking branch 'upstream'

0a71bb5

Merge remote-tracking branch 'upstream/main'

f4738fe

Merge remote-tracking branch 'upstream/main'

9c508bc

mrosseel and others added 26 commits January 18, 2026 23:35

UX: Show 'Saving...' during correction save, target exact noise floor

0cb24c4

- Show 'Saving...' message immediately before creating zip - Remove +2 margin from noise floor - target exact value for shortest exposure Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add noise_floor and imu_delta to correction metadata

4923a1c

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add full SQM calculation details to shared state and correction metadata

2711aa6

Includes: mzero, mzero_std, background, pedestal, extinction, star counts, saturation info, etc. - everything needed for calibration validation. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Show star count in SQM view

fbf6a9e

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Move star count to top line in SQM view

51d7557

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add bracketed exposures to correction package

0900255

Captures 3 exposures: base, +1 stop, -1 stop (or base, -1, -2 if at max). Shows progress for each bracket. Re-enables auto-exposure when done. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Reduce exposure sweep from 100 to 20 images

3dcfb97

Faster sweep (~1 min vs ~4 min) while still sampling exposure space. Launch from SQM view → hold square → DEBUG. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Use flux-weighted mean for mzero calculation

5542e02

Brighter stars have higher SNR so their mzero estimates are more reliable. Weight each star's contribution by its flux. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Restore +2 margin above noise floor

aa2a79a

Without margin, background_corrected ≈ 0 when at noise floor, causing log10(~0) = very negative, making SQM too high. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

better default for camera

b13a0a2

small corrections to sqm calculation and athmospheric correcitons

c30ec0f

Merge branch 'sqm' of github.com:mrosseel/PiFinder into sqm

6f3cea1

Set SNR as default for testing

c180d68

Update tests for sweep starting at 400ms

9b14eb9

Tests now reflect the new sweep pattern that starts at 400ms instead of 25ms for faster recovery. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Fix missing right argument in SQM MarkingMenu

4263806

MarkingMenu dataclass requires down, left, and right arguments. Added empty right option to fix TypeError on SQM screen launch. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Fix calibration UI text for bias/dark captures

2ebfbfb

Change "Hold steady..." to "Keep cap on..." during bias and dark frame captures, since the telescope cap should be on during these steps. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

mrosseel changed the title ~~[WIP] Sqm enhancements~~ SQM enhancements Mar 14, 2026

mrosseel and others added 2 commits March 14, 2026 19:25

fix: update PID tests for asymmetric gains API

36f4e6b

Tests referenced old single `gains` parameter but ExposurePIDController now uses `gains_decrease`/`gains_increase` for asymmetric PID control. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

fix: remove unused variables in SQM tests (ruff F841)

1cffc07

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

brickbots merged commit ca1a8bf into brickbots:main Mar 16, 2026
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

SQM enhancements#374

SQM enhancements#374
brickbots merged 139 commits intobrickbots:mainfrom
mrosseel:sqm

mrosseel commented Jan 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

mrosseel commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes (vs upstream/main)

Diff stat

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mrosseel commented Jan 26, 2026 •

edited

Loading