Coverage nexp masks

[martinkilbinger]

Summary

Implements creation of coverage healsparse masks encoding the exposures on which PSF information is avialable.

The current code, implemented in utilities does the following steps:

1. Read CCD list and download corresponding FITS exposure images headers (header_downloader.py)
2. Extract CCD corner coordinates using WCS from header (field_corners_extractor.py)
3. Create healsparse mask (coverage_map_builder.py)

This can be integrated into a workflow. We also need to implement obtaining the list of valid CCDs with PSF model.

Reviewer Checklist

Reviewers should tick the following boxes before approving and merging the PR.

• The PR targets the develop branch
• The PR is assigned to the developer
• The PR has appropriate labels
• The PR is included in appropriate projects and/or milestones
• The PR includes a clear description of the proposed changes
• If the PR addresses an open issue the description includes "closes #"
• The code and documentation style match the current standards
• Documentation has been added/updated consistently with the code
• All CI tests are passing
• API docs have been built and checked at least once (if relevant)
• All changed files have been checked and comments provided to the developer
• All of the reviewer's comments have been satisfactorily addressed by the developer

[cailmdaley]

Thanks Martin. The merge conflict's resolved — I merged develop in (the branch had fallen well behind), reconciled pyproject.toml, and regenerated uv.lock, so it's MERGEABLE again. Structure looks clean; the main thing is a scope question, then a few specific items inline.

Per-exposure vs per-CCD coverage. The mask is currently built per exposure: get_exposures() collapses the per-CCD valid-PSF IDs down to unique exposure numbers, _megacam_field_corners() draws a single polygon over the whole ~1° MegaCam mosaic, and each exposure adds a flat +1 across it. So an exposure with even one valid-PSF CCD stamps +1 over its entire field — including the sky where its other CCDs had no valid PSF. nexp then over-reports PSF depth exactly where CCD-level failures cluster (chip gaps, flagged/edge CCDs). The per-CCD validity is already computed upstream, so the fix is to carry the exp-shdu identity through and draw one polygon per surviving CCD instead of collapsing to exposures.

Two questions so we're reviewing against the right target: (1) do you intend to build the mask at CCD level? (2) does the valid-CCD-with-PSF selection land in this PR, or a follow-up? The description flags it as still-to-do, so I want to make sure we're not measuring against the wrong scope.

I think this wants test coverage before it merges — there's none yet, and the pure parsing functions (exposure-number regex, header→WCS split, the corner convention) are exactly where silent failures would hide, and are cheap to cover. Happy to write those with you, and to help wire the per-CCD path if that's the direction.

A few specific items inline.

— Fable, on behalf of Cail

[cailmdaley]

Blocker. coverage_pipeline points at coverage_run:run_pipeline, which no longer exists (removed in an earlier commit). The installed console script raises AttributeError on invocation — drop this line, or restore an orchestrator function.

[cailmdaley]

The missing-CCD set is computed at line 226 but never subtracted here, so this returns every CCD as valid — unfiltered. (The v1.3 path does filter on non-nan FWHM, so the two versions disagree.) Likely part of the valid-CCD-with-PSF work still to come; flagging since the current path silently returns unfiltered. Fix is exp_shdu_all -= exp_shdu_missing_all before returning.

[cailmdaley]

No RA-wrap or pole guard. Corners arrive straight from astropy as degrees in [0, 360), so an exposure straddling RA=0 yields non-monotonic corners (e.g. [359.8, 0.2, ...]) and healsparse fills the ~360° complement, corrupting nexp across that RA stripe; near Dec ≈ +90 (the driver runs dechi=90) the four-corner ordering also degenerates. Worth unwrapping RA to a common branch per polygon and guarding the pole.

[cailmdaley]

hpgeom is imported directly but isn't in dependencies — it only resolves today because healsparse pulls it in transitively. Worth declaring explicitly so it doesn't break the day healsparse drops it.

[cailmdaley]

client.copy writes straight to dest, so an interrupted transfer leaves a partial .txt that the exists(dest) check at line 194 treats as complete on the next run — a resumed job then builds on a truncated header. Download to a temp path and os.rename on success (atomic).

[cailmdaley]

No set -euo pipefail — a failed build/plot for one version doesn't stop the loop, and the final success message prints unconditionally.

[cailmdaley]

-i exp_ra_dec_v1.6, but the extract_field_corners step above is run without -o, so it writes the default exp_ra_dec.txt — this filename is never produced, and the example throws FileNotFoundError if followed verbatim. Align the two.

[cailmdaley]

Update — dug into the failing checks and took care of them so this is green for you.

Merging develop in ran its full suite against the branch, surfacing that all six coverage console scripts failed the entry-point smoke test (tests/unit/test_entrypoints.py::test_console_entrypoint_help_runs). Two causes, both fixed:

• The five runners parsed the full sys.argv (including argv[0]) rather than sys.argv[1:], so -h collided with the integer -c option (error: option -c: invalid integer value: '-h'). Fixed to slice argv[1:], matching cs_util.args.parse_options' own convention.
• coverage_pipeline pointed at run_pipeline, which had been removed — dropped the dangling entry point (easy to re-add as a real orchestrator later if you'd like one).

Also declared hpgeom explicitly (imported directly by coverage_map_builder, previously only resolving transitively via healsparse).

And I added tests/module/test_coverage.py — unit + property tests for the pure logic: exposure-number parsing, the CCD-list → unique-exposure reduction, the MegaCam field-corner convention, the multi-HDU header → WCS split, power-of-two nside validation, and a regression guard for the -h handling. Happy to extend these as the feature grows.

CI is green now (357 passed). With that, the one substantive thing still open is the design question at the top: whether the mask should be built per-CCD — so PSF validity is honoured per-CCD footprint — rather than per-exposure. That one's yours; glad to help wire it once you've settled the intent (and whether the valid-CCD-with-PSF selection lands here or in a follow-up).

— Fable, on behalf of Cail