Skip to content

Address review of #32: shared recipe constants, assert_unit_consistency, units in the repr - #33

Merged
hdrake merged 2 commits into
units-handlingfrom
units-review-fixes
Jul 28, 2026
Merged

Address review of #32: shared recipe constants, assert_unit_consistency, units in the repr#33
hdrake merged 2 commits into
units-handlingfrom
units-review-fixes

Conversation

@hdrake

@hdrake hdrake commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Addresses the five review comments on #32 (two GitHub, three ReviewNB). Targets
units-handling so it merges into that PR rather than main.

Review comment Resolution
GH: "Adding these lines a dozen times throughout the recipe is tedious... can we just add a constants entry at root level?" New top-level constants key
GH: "Rename this throughout to assert_unit_consistency or something similarly descriptive but more concise." strictassert_unit_consistency
RNB: "The recipe's __repr__ should include the units, maybe in square brackets after the term's name?" Every node annotated [units]
RNB: "Are there any var: null left? This might be a legacy comment." Correct — comment rewritten
RNB: "# to-do: add comparable chunking inside the difference and convergence routines — either do this or remove the comment." Done, not removed

Shared constants at the recipe root

constants:
  seawater_density:
    value: 1035.
    units: "kg m-3"

mass:
  lhs:
    sum:
      Eulerian_tendency:
        product:
          thickness_tendency: "dhdt"
          density: "seawater_density"   # -> 1035. kg m-3
          area: "areacello"

Entries take the same two forms an inline constant does (a bare number, or a
{value:, units:} mapping). References are substituted while parsing, so the
typed tree still holds only Constant nodes and nothing downstream — evaluator,
query, display — knows the table existed. A string operand resolves against the
table first and otherwise stays a diagnostic name, so a constant sharing a name
with a dataset variable shadows it; the shipped recipes name theirs distinctly,
and a test pins that every declared constant is actually referenced (a rename
that missed a use would otherwise degrade quietly into a diagnostic never found).

constants is the one top-level key that is not a budget. This removed 36
repeated blocks from ECCOV4r4_native.yaml, 11 from MOM6.yaml, 9 from
MOM6_3Donly.yaml, 2 from MOM6_surface.yaml.

strictassert_unit_consistency

Straight rename across collect_budgets, evaluate_budgets, the error text,
the tests, the docs, and the notebooks. Not deprecated — strict never shipped.

Units in the repr

Each node now carries its units in square brackets, in both the HTML tree and
the ASCII one:

heat  units=W, lambda=thetao, surface_lambda=tos
├─ lhs [Σ] [W]
│  ├─ Eulerian_tendency [×] [W]
│  │  ├─ tracer_content_tendency_per_unit_area: opottemptend [W m-2]
│  │  └─ area: areacello [m2]
│  └─ surface_ocean_flux_advective_negative_lhs [×] [W]
│     ├─ sign: -1.
│     ├─ specific_heat_capacity: 3992. [J kg-1 K-1]
│     ├─ lambda_mass: tos [degC]
│     ├─ thickness_tendency: boundary_forcing_h_tendency [m s-1]
│     ├─ density: 1035. [kg m-3]
│     └─ area: areacello [m2]

Units enter at the leaves, so show_recipe takes an optional third argument —
the grid/dataset to read raw diagnostics' units from — and the whole tree then
resolves. Reading it top-down shows exactly where a unit stops being
inferable, which is the useful diagnostic when assert_unit_consistency fails.
Without it, only the recipe's declared constants are annotated. A BudgetQuery
displayed in a notebook prefers the units its run actually stamped and falls
back to inference, so it is annotated before and after collect_budgets.

This is backed by a new units.infer_units(node, lookup), which runs the
evaluator's arithmetic statically over the typed tree. A test pins that it
agrees with what the evaluator stamps.

Bare numbers (sign: -1.) are deliberately left unannotated: they are
dimensionless by default rather than by declaration, and a [1] on every sign
would be noise.

var: null — a genuinely legacy comment

MOM6.yaml has no var: null entries left (0.7.0 made the placeholders
optional), so the markdown above collect_budgets was describing something the
cell above it no longer shows. Rewritten to explain what the tree does show —
the operator badges, the new units, and that a term without a var of its own
is one reconstructed from the diagnostics beneath it.

The chunking to-do: done, not deleted

lateral_divergence now gets the same allow_rechunk treatment difference
has always had — the differenced dimensions in a single chunk while xgcm works,
the caller's chunking restored afterwards. The two paths now share three helpers
(_dataset_chunks / _single_chunk_along / _restore_chunks) rather than one
having the logic inline and the other not at all.

Verified on the real ECCO V4r4 LLC90 budgets: numerically identical (all
three budgets, byte for byte) and roughly half the wall clock — graph build
16.6 s → 3.4 s, closure compute 44.2 s → 23.5 s. The notebook still chunks its
dataset, but for memory sizing, which xbudget cannot guess; the comment now says
so.

The chunking wrapper is _Evaluator._lateral_divergence (renamed from
_divergence per review); it wraps the bare collect.lateral_divergence, which
does the xgcm work.

lateral_divergence also gains its first CI test, on a two-tile face-connected
synthetic grid — a miniature of the LLC topology it exists for. It previously
ran only against the data-gated ECCO notebooks, so nothing in CI touched it.

Verified

  • Full suite green, 136 → 157 tests. Run locally with the MOM6 example
    dataset present, so the characterization golden is included rather than
    skipped — it still matches, unchanged.
  • assert_unit_consistency=True passes on the real MOM6 dataset (all six
    sides) and on the real ECCO V4r4 (LLC90) dataset; ECCO budgets still close
    to the same residuals (mass 2.6e-4, heat 5.1e-6, salt 1.2e-4, relative).
  • All three affected notebooks re-executed against the real datasets.
  • Docs build clean under python -m sphinx -b html -W --keep-going.

Note that GitHub Actions does not run on this PR: ci.yml triggers only on pull
requests targeting main, so a stacked PR into units-handling gets no test
job. The Actions matrix (Python 3.11-3.14) will cover this once #32 itself is
merged into main. Read the Docs does build it.

Review follow-ups

  • Root-level constants table (GH review)
  • strict -> assert_unit_consistency (GH review)
  • Units in the recipe repr, in square brackets after the term name (ReviewNB)
  • Stale var: null markdown in the MOM6 notebook (ReviewNB)
  • The ECCO notebook's chunking to-do — implemented, not deleted (ReviewNB)
  • _divergence -> _lateral_divergence (GH review on this PR)

🤖 Generated with Claude Code

…the repr

Five changes, one per review comment on the units-handling PR.

- Shared constants at the recipe root. A new top-level `constants` key declares
  a scalar once (`seawater_density: {value: 1035., units: "kg m-3"}`) and terms
  refer to it by name (`density: "seawater_density"`), instead of repeating the
  three-line mapping at every use — 36 of them in ECCOV4r4_native.yaml alone.
  References are substituted during parsing, so nothing downstream knows the
  table existed; a string operand resolves against it first and otherwise stays
  a diagnostic name.
- `strict` is now `assert_unit_consistency`, which says what it asserts.
- The recipe display annotates every node with its units in square brackets
  (`density: 1035. [kg m-3]`, `lateral [×] [W]`). Units enter at the leaves, so
  `show_recipe` takes an optional dataset/grid to read raw diagnostics' units
  from and the whole tree resolves; a `BudgetQuery` shown in a notebook prefers
  the units its run actually stamped. Backed by `units.infer_units`, which runs
  the evaluator's arithmetic statically over the typed tree.
- Dropped the "legacy comment" in the MOM6 notebook: MOM6.yaml has no `var:
  null` entries left, so the text now explains what it is really showing.
- Did the ECCO notebook's chunking to-do rather than deleting it.
  `lateral_divergence` now gets the same `allow_rechunk` treatment `difference`
  has always had, via three helpers the two now share. Verified identical to the
  byte on the real ECCO LLC90 budgets, and roughly half the wall clock.

`lateral_divergence` also gains its first CI test, on a two-tile face-connected
synthetic grid; previously it ran only against the data-gated ECCO notebooks.

Verified: full suite green (136 -> 157) with the MOM6 example dataset present,
so the characterization golden is included; `assert_unit_consistency=True`
passes on both the real MOM6 and ECCO V4r4 datasets; all three affected
notebooks re-executed; docs build clean under `-W`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Comment thread xbudget/evaluate.py Outdated
Review feedback on #33: `_divergence` was vaguer than the operation it wraps.
Its body calls the bare `collect.lateral_divergence`, which the docstring now
says explicitly so the two are not mistaken for each other.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@hdrake
hdrake merged commit 088ab07 into units-handling Jul 28, 2026
1 check passed
@hdrake
hdrake deleted the units-review-fixes branch July 28, 2026 20:07
hdrake added a commit that referenced this pull request Jul 29, 2026
#28) (#32)

* Add automatic units handling with UDUNITS inference and strict mode

Every derived variable now carries an inferred `units` attribute, composed
through the typed tree with real UDUNITS arithmetic via cf-units: products
multiply units, sums require compatible summands, difference preserves,
reciprocal inverts, lateral_divergence carries the flux unit. Constants may
declare units via a {value, units} mapping; a bare number is dimensionless.

`collect_budgets(strict=True)` enforces that each budget declaring a top-level
`units` reconciles its lhs/rhs roots to it (else UnitError); non-strict warns.
BudgetQuery gains .units() and .budget_units(). All shipped recipes are
annotated and declare budget units; verified strict passes on the MOM6 example.

psu is treated as dimensionless (PSS-78); degC is dimensionalized in products
so heat-content terms reconcile to W.

Closes #28 (mechanism; ECCO pipeline units follow).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Re-execute example notebooks under strict units mode

Attach UDUNITS units to the derived budget inputs the ECCO notebooks build
(dt, volcello, WVELMASS_interior, boundary_forcing_*) so unit inference has a
complete picture, and switch the three budget notebooks (MOM6 + both ECCO) to
collect_budgets(..., strict=True). Verified: strict passes on both the MOM6 and
ECCO V4r4 (LLC90) datasets; derived variables now display their inferred units.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Address review of #32: shared constants, clearer flag name, units in the repr

Five changes, one per review comment on the units-handling PR.

- Shared constants at the recipe root. A new top-level `constants` key declares
  a scalar once (`seawater_density: {value: 1035., units: "kg m-3"}`) and terms
  refer to it by name (`density: "seawater_density"`), instead of repeating the
  three-line mapping at every use — 36 of them in ECCOV4r4_native.yaml alone.
  References are substituted during parsing, so nothing downstream knows the
  table existed; a string operand resolves against it first and otherwise stays
  a diagnostic name.
- `strict` is now `assert_unit_consistency`, which says what it asserts.
- The recipe display annotates every node with its units in square brackets
  (`density: 1035. [kg m-3]`, `lateral [×] [W]`). Units enter at the leaves, so
  `show_recipe` takes an optional dataset/grid to read raw diagnostics' units
  from and the whole tree resolves; a `BudgetQuery` shown in a notebook prefers
  the units its run actually stamped. Backed by `units.infer_units`, which runs
  the evaluator's arithmetic statically over the typed tree.
- Dropped the "legacy comment" in the MOM6 notebook: MOM6.yaml has no `var:
  null` entries left, so the text now explains what it is really showing.
- Did the ECCO notebook's chunking to-do rather than deleting it.
  `lateral_divergence` now gets the same `allow_rechunk` treatment `difference`
  has always had, via three helpers the two now share. Verified identical to the
  byte on the real ECCO LLC90 budgets, and roughly half the wall clock.

`lateral_divergence` also gains its first CI test, on a two-tile face-connected
synthetic grid; previously it ran only against the data-gated ECCO notebooks.

Verified: full suite green (136 -> 157) with the MOM6 example dataset present,
so the characterization golden is included; `assert_unit_consistency=True`
passes on both the real MOM6 and ECCO V4r4 datasets; all three affected
notebooks re-executed; docs build clean under `-W`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Rename the divergence chunking wrapper to _lateral_divergence

Review feedback on #33: `_divergence` was vaguer than the operation it wraps.
Its body calls the bare `collect.lateral_divergence`, which the docstring now
says explicitly so the two are not mistaken for each other.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Act on independent review: unit scale, grid guard, lenient display inference

An independent reviewer went through the units feature hunting for bugs and
tried to write a recipe for their own model from the docs alone. Four findings
were worth fixing, plus the doc gap that stopped them cold.

Correctness

- The stamped `units` on the ECCO salt budget were wrong by 1000x. ECCO's salt
  fluxes carry the salinity scale in their own units string (`1e-3 m3 s-1`), and
  the recipe's compensating `unit_conversion: 0.001` was a bare number — so the
  scale survived into the composed units while the values were already kg/s.
  Anyone reading the attribute got a thousandth of the real flux. The conversion
  constants now declare the reciprocal scale (`{value: 0.001, units: "1e3"}`,
  and `kg g-1` for the g->kg one), so value x units == 1 and the scale cancels.
  Values are untouched; only the label changed.

- `assert_unit_consistency` now compares magnitude, not just dimension, so it
  catches that class of error instead of passing it. `is_convertible` cannot see
  a numeric scale, which is exactly what a mis-declared conversion factor leaves
  behind. Both shipped models reconcile exactly under the stricter check. This
  reverses the "dimension, not exact units" decision documented in #32; the
  motivating case there (salinity's `1e-3`) is better handled by making the
  recipe cancel the scale than by declining to look at it.

- `lateral_divergence` without an `xgcm.Grid` raised `AttributeError: 'NoneType'
  object has no attribute 'axes'` — a regression from the rechunking in #33,
  which reads the grid before the guarded helper is called. Both grid-needing
  operations now share one `_require_grid`, and the divergence has the
  grid-guard test `difference` already had.

- `show_recipe` blanked the units of every term above a missing diagnostic,
  including the budget roots — on the raw ECCO dataset, all three rhs roots.
  `infer_units` takes an optional `available` predicate and, given one, mirrors
  what the evaluator builds (dropping operands it would drop) rather than what
  the recipe describes. A term that genuinely will not be built is still blank.

Smaller fixes

- A `constants` entry missing `value` raised a bare `KeyError`; a budget named
  `constants` raised the same. Both now say what is wrong, with the path.
- The `reciprocal` operand was the one leaf the HTML tree left unannotated.
- Missing diagnostics are reported before unit issues: an absent input causes
  unknown units, so `on_missing="raise"` now names the diagnostic, not the
  symptom.
- `units:` on a term gets its own warning instead of the generic "missing an
  enclosing product" hint, which pointed at the wrong fix.
- Rechunking is a no-op on unchunked data rather than silently returning dask.

Docs

- `recipes.md` gained the `xgcm.Grid` section it never had. The reviewer got
  through everything else from the docs and then had to read `tests/conftest.py`
  to learn that xgcm >= 0.10 wants `padding=`, which every older tutorial spells
  `boundary=`/`periodic=`.
- The "conversion factors are dimensionless" guidance led them straight into the
  bug above; it now says the opposite, with the worked pattern.
- Documented what the assertion does not check, that constants substitute only
  in sum/product operands, and that `aggregate()` returns metadata beside sides.

Verified: 178 tests pass (was 157) with both example datasets present;
`assert_unit_consistency=True` passes on real MOM6 and ECCO under the stricter
magnitude check, with ECCO closure residuals unchanged; three notebooks
re-executed; docs build clean under `-W`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Document per-worktree conda env convention in CLAUDE.md

Editable installs bind a conda env to a single checkout, so sharing
docs_env_xbudget across worktrees silently tests the wrong tree. Require
a dedicated docs_env_xbudget_<branch-or-worktree-name> env instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant