Skip to content

Migrate cosmo_val + Snakemake to the SACC writers #247

Description

@cailmdaley

Make SACC the sole native output format of the cosmology-validation pipeline. Every two-point product (ξ± coarse and fine, pseudo-Cℓ, COSEBIs, pure-E/B, ρ/τ, n(z)) is written through the sacc_io builders into single-statistic SACC part files, and the Snakemake workflow assembles those parts into one terminal SACC file per catalogue version, {version}.sacc. No bespoke writer remains on the data-vector path.

Desired end state

  • Each cosmo_val statistic's write step is replaced by a call to the matching sacc_io builder, producing a part file that holds exactly one statistic plus the self-describing n(z) tracers it references. A producer writes a part with a single save(s, path) call; it never assembles a terminal file and never emits a bespoke format alongside.
  • A dedicated Snakemake assembly rule reads the parts, concatenates them in canonical order, attaches the BlockDiagonal covariance (analysis blocks plus the dense per-pair fine block), and writes the single terminal {version}.sacc. The terminal file is a pure gather; anything that wants only one statistic targets the corresponding part rule, not a second terminal file.
  • Part filenames are DAG internals: consumers bind parts through the workflow's path helpers, never by hardcoding names. {version}.sacc is the only stable name.
  • Blinding rides the DAG here. Blinding is per part, at birth (PRD: SACC data format + Smokescreen blinding #241 §4; CLIs and the assembly-time assertion are delivered by Smokescreen blinding wiring (fork protocol, three theory backends, custody) #252): a blind-init rule fires once per catalogue version, each blindable part rule (coarse ξ±, fine ξ±, pseudo-Cℓ) is followed by blind-part so only blinded parts persist on disk, COSEBIs/pure-E/B are derived from the blinded fine ξ± (born blinded), and the terminal assembly asserts blind_commitment is identical across parts. This issue authors the rules; Smokescreen blinding wiring (fork protocol, three theory backends, custody) #252 supplies the machinery.
  • The bespoke .txt/custom-FITS/.npz writers on the data-vector path are deleted; nothing downstream reads them.

sacc_io is a fixed dependency and the single source of truth for every builder signature. Metadata keys required on every file: catalogue_version, sp_validation_version, created, npatch.

Acceptance

The workflow produces {version}.sacc per version from parts, with only blinded blindable parts persisted and the commitment assertion exercised; the deleted bespoke writers have no remaining readers; the assembled file matches the sacc_io layout contract.

— Fable on behalf of Cail.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions