Address review of #32: shared recipe constants, assert_unit_consistency, units in the repr - #33
Merged
Merged
Conversation
…the repr
Five changes, one per review comment on the units-handling PR.
- Shared constants at the recipe root. A new top-level `constants` key declares
a scalar once (`seawater_density: {value: 1035., units: "kg m-3"}`) and terms
refer to it by name (`density: "seawater_density"`), instead of repeating the
three-line mapping at every use — 36 of them in ECCOV4r4_native.yaml alone.
References are substituted during parsing, so nothing downstream knows the
table existed; a string operand resolves against it first and otherwise stays
a diagnostic name.
- `strict` is now `assert_unit_consistency`, which says what it asserts.
- The recipe display annotates every node with its units in square brackets
(`density: 1035. [kg m-3]`, `lateral [×] [W]`). Units enter at the leaves, so
`show_recipe` takes an optional dataset/grid to read raw diagnostics' units
from and the whole tree resolves; a `BudgetQuery` shown in a notebook prefers
the units its run actually stamped. Backed by `units.infer_units`, which runs
the evaluator's arithmetic statically over the typed tree.
- Dropped the "legacy comment" in the MOM6 notebook: MOM6.yaml has no `var:
null` entries left, so the text now explains what it is really showing.
- Did the ECCO notebook's chunking to-do rather than deleting it.
`lateral_divergence` now gets the same `allow_rechunk` treatment `difference`
has always had, via three helpers the two now share. Verified identical to the
byte on the real ECCO LLC90 budgets, and roughly half the wall clock.
`lateral_divergence` also gains its first CI test, on a two-tile face-connected
synthetic grid; previously it ran only against the data-gated ECCO notebooks.
Verified: full suite green (136 -> 157) with the MOM6 example dataset present,
so the characterization golden is included; `assert_unit_consistency=True`
passes on both the real MOM6 and ECCO V4r4 datasets; all three affected
notebooks re-executed; docs build clean under `-W`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
hdrake
commented
Jul 28, 2026
Review feedback on #33: `_divergence` was vaguer than the operation it wraps. Its body calls the bare `collect.lateral_divergence`, which the docstring now says explicitly so the two are not mistaken for each other. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hdrake
added a commit
that referenced
this pull request
Jul 29, 2026
#28) (#32) * Add automatic units handling with UDUNITS inference and strict mode Every derived variable now carries an inferred `units` attribute, composed through the typed tree with real UDUNITS arithmetic via cf-units: products multiply units, sums require compatible summands, difference preserves, reciprocal inverts, lateral_divergence carries the flux unit. Constants may declare units via a {value, units} mapping; a bare number is dimensionless. `collect_budgets(strict=True)` enforces that each budget declaring a top-level `units` reconciles its lhs/rhs roots to it (else UnitError); non-strict warns. BudgetQuery gains .units() and .budget_units(). All shipped recipes are annotated and declare budget units; verified strict passes on the MOM6 example. psu is treated as dimensionless (PSS-78); degC is dimensionalized in products so heat-content terms reconcile to W. Closes #28 (mechanism; ECCO pipeline units follow). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Re-execute example notebooks under strict units mode Attach UDUNITS units to the derived budget inputs the ECCO notebooks build (dt, volcello, WVELMASS_interior, boundary_forcing_*) so unit inference has a complete picture, and switch the three budget notebooks (MOM6 + both ECCO) to collect_budgets(..., strict=True). Verified: strict passes on both the MOM6 and ECCO V4r4 (LLC90) datasets; derived variables now display their inferred units. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Address review of #32: shared constants, clearer flag name, units in the repr Five changes, one per review comment on the units-handling PR. - Shared constants at the recipe root. A new top-level `constants` key declares a scalar once (`seawater_density: {value: 1035., units: "kg m-3"}`) and terms refer to it by name (`density: "seawater_density"`), instead of repeating the three-line mapping at every use — 36 of them in ECCOV4r4_native.yaml alone. References are substituted during parsing, so nothing downstream knows the table existed; a string operand resolves against it first and otherwise stays a diagnostic name. - `strict` is now `assert_unit_consistency`, which says what it asserts. - The recipe display annotates every node with its units in square brackets (`density: 1035. [kg m-3]`, `lateral [×] [W]`). Units enter at the leaves, so `show_recipe` takes an optional dataset/grid to read raw diagnostics' units from and the whole tree resolves; a `BudgetQuery` shown in a notebook prefers the units its run actually stamped. Backed by `units.infer_units`, which runs the evaluator's arithmetic statically over the typed tree. - Dropped the "legacy comment" in the MOM6 notebook: MOM6.yaml has no `var: null` entries left, so the text now explains what it is really showing. - Did the ECCO notebook's chunking to-do rather than deleting it. `lateral_divergence` now gets the same `allow_rechunk` treatment `difference` has always had, via three helpers the two now share. Verified identical to the byte on the real ECCO LLC90 budgets, and roughly half the wall clock. `lateral_divergence` also gains its first CI test, on a two-tile face-connected synthetic grid; previously it ran only against the data-gated ECCO notebooks. Verified: full suite green (136 -> 157) with the MOM6 example dataset present, so the characterization golden is included; `assert_unit_consistency=True` passes on both the real MOM6 and ECCO V4r4 datasets; all three affected notebooks re-executed; docs build clean under `-W`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Rename the divergence chunking wrapper to _lateral_divergence Review feedback on #33: `_divergence` was vaguer than the operation it wraps. Its body calls the bare `collect.lateral_divergence`, which the docstring now says explicitly so the two are not mistaken for each other. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Act on independent review: unit scale, grid guard, lenient display inference An independent reviewer went through the units feature hunting for bugs and tried to write a recipe for their own model from the docs alone. Four findings were worth fixing, plus the doc gap that stopped them cold. Correctness - The stamped `units` on the ECCO salt budget were wrong by 1000x. ECCO's salt fluxes carry the salinity scale in their own units string (`1e-3 m3 s-1`), and the recipe's compensating `unit_conversion: 0.001` was a bare number — so the scale survived into the composed units while the values were already kg/s. Anyone reading the attribute got a thousandth of the real flux. The conversion constants now declare the reciprocal scale (`{value: 0.001, units: "1e3"}`, and `kg g-1` for the g->kg one), so value x units == 1 and the scale cancels. Values are untouched; only the label changed. - `assert_unit_consistency` now compares magnitude, not just dimension, so it catches that class of error instead of passing it. `is_convertible` cannot see a numeric scale, which is exactly what a mis-declared conversion factor leaves behind. Both shipped models reconcile exactly under the stricter check. This reverses the "dimension, not exact units" decision documented in #32; the motivating case there (salinity's `1e-3`) is better handled by making the recipe cancel the scale than by declining to look at it. - `lateral_divergence` without an `xgcm.Grid` raised `AttributeError: 'NoneType' object has no attribute 'axes'` — a regression from the rechunking in #33, which reads the grid before the guarded helper is called. Both grid-needing operations now share one `_require_grid`, and the divergence has the grid-guard test `difference` already had. - `show_recipe` blanked the units of every term above a missing diagnostic, including the budget roots — on the raw ECCO dataset, all three rhs roots. `infer_units` takes an optional `available` predicate and, given one, mirrors what the evaluator builds (dropping operands it would drop) rather than what the recipe describes. A term that genuinely will not be built is still blank. Smaller fixes - A `constants` entry missing `value` raised a bare `KeyError`; a budget named `constants` raised the same. Both now say what is wrong, with the path. - The `reciprocal` operand was the one leaf the HTML tree left unannotated. - Missing diagnostics are reported before unit issues: an absent input causes unknown units, so `on_missing="raise"` now names the diagnostic, not the symptom. - `units:` on a term gets its own warning instead of the generic "missing an enclosing product" hint, which pointed at the wrong fix. - Rechunking is a no-op on unchunked data rather than silently returning dask. Docs - `recipes.md` gained the `xgcm.Grid` section it never had. The reviewer got through everything else from the docs and then had to read `tests/conftest.py` to learn that xgcm >= 0.10 wants `padding=`, which every older tutorial spells `boundary=`/`periodic=`. - The "conversion factors are dimensionless" guidance led them straight into the bug above; it now says the opposite, with the worked pattern. - Documented what the assertion does not check, that constants substitute only in sum/product operands, and that `aggregate()` returns metadata beside sides. Verified: 178 tests pass (was 157) with both example datasets present; `assert_unit_consistency=True` passes on real MOM6 and ECCO under the stricter magnitude check, with ECCO closure residuals unchanged; three notebooks re-executed; docs build clean under `-W`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Document per-worktree conda env convention in CLAUDE.md Editable installs bind a conda env to a single checkout, so sharing docs_env_xbudget across worktrees silently tests the wrong tree. Require a dedicated docs_env_xbudget_<branch-or-worktree-name> env instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the five review comments on #32 (two GitHub, three ReviewNB). Targets
units-handlingso it merges into that PR rather thanmain.constantsentry at root level?"constantskeyassert_unit_consistencyor something similarly descriptive but more concise."strict→assert_unit_consistency__repr__should include the units, maybe in square brackets after the term's name?"[units]var: nullleft? This might be a legacy comment."# to-do: add comparable chunking inside the difference and convergence routines— either do this or remove the comment."Shared constants at the recipe root
Entries take the same two forms an inline constant does (a bare number, or a
{value:, units:}mapping). References are substituted while parsing, so thetyped tree still holds only
Constantnodes and nothing downstream — evaluator,query, display — knows the table existed. A string operand resolves against the
table first and otherwise stays a diagnostic name, so a constant sharing a name
with a dataset variable shadows it; the shipped recipes name theirs distinctly,
and a test pins that every declared constant is actually referenced (a rename
that missed a use would otherwise degrade quietly into a diagnostic never found).
constantsis the one top-level key that is not a budget. This removed 36repeated blocks from
ECCOV4r4_native.yaml, 11 fromMOM6.yaml, 9 fromMOM6_3Donly.yaml, 2 fromMOM6_surface.yaml.strict→assert_unit_consistencyStraight rename across
collect_budgets,evaluate_budgets, the error text,the tests, the docs, and the notebooks. Not deprecated —
strictnever shipped.Units in the repr
Each node now carries its units in square brackets, in both the HTML tree and
the ASCII one:
Units enter at the leaves, so
show_recipetakes an optional third argument —the grid/dataset to read raw diagnostics'
unitsfrom — and the whole tree thenresolves. Reading it top-down shows exactly where a unit stops being
inferable, which is the useful diagnostic when
assert_unit_consistencyfails.Without it, only the recipe's declared constants are annotated. A
BudgetQuerydisplayed in a notebook prefers the units its run actually stamped and falls
back to inference, so it is annotated before and after
collect_budgets.This is backed by a new
units.infer_units(node, lookup), which runs theevaluator's arithmetic statically over the typed tree. A test pins that it
agrees with what the evaluator stamps.
Bare numbers (
sign: -1.) are deliberately left unannotated: they aredimensionless by default rather than by declaration, and a
[1]on every signwould be noise.
var: null— a genuinely legacy commentMOM6.yamlhas novar: nullentries left (0.7.0 made the placeholdersoptional), so the markdown above
collect_budgetswas describing something thecell above it no longer shows. Rewritten to explain what the tree does show —
the operator badges, the new units, and that a term without a
varof its ownis one reconstructed from the diagnostics beneath it.
The chunking to-do: done, not deleted
lateral_divergencenow gets the sameallow_rechunktreatmentdifferencehas always had — the differenced dimensions in a single chunk while xgcm works,
the caller's chunking restored afterwards. The two paths now share three helpers
(
_dataset_chunks/_single_chunk_along/_restore_chunks) rather than onehaving the logic inline and the other not at all.
Verified on the real ECCO V4r4 LLC90 budgets: numerically identical (all
three budgets, byte for byte) and roughly half the wall clock — graph build
16.6 s → 3.4 s, closure compute 44.2 s → 23.5 s. The notebook still chunks its
dataset, but for memory sizing, which xbudget cannot guess; the comment now says
so.
The chunking wrapper is
_Evaluator._lateral_divergence(renamed from_divergenceper review); it wraps the barecollect.lateral_divergence, whichdoes the xgcm work.
lateral_divergencealso gains its first CI test, on a two-tile face-connectedsynthetic grid — a miniature of the LLC topology it exists for. It previously
ran only against the data-gated ECCO notebooks, so nothing in CI touched it.
Verified
dataset present, so the characterization golden is included rather than
skipped — it still matches, unchanged.
assert_unit_consistency=Truepasses on the real MOM6 dataset (all sixsides) and on the real ECCO V4r4 (LLC90) dataset; ECCO budgets still close
to the same residuals (mass 2.6e-4, heat 5.1e-6, salt 1.2e-4, relative).
python -m sphinx -b html -W --keep-going.Note that GitHub Actions does not run on this PR:
ci.ymltriggers only on pullrequests targeting
main, so a stacked PR intounits-handlinggets no testjob. The Actions matrix (Python 3.11-3.14) will cover this once #32 itself is
merged into
main. Read the Docs does build it.Review follow-ups
constantstable (GH review)strict->assert_unit_consistency(GH review)var: nullmarkdown in the MOM6 notebook (ReviewNB)_divergence->_lateral_divergence(GH review on this PR)🤖 Generated with Claude Code