Skip to content

Latest commit

 

History

History
539 lines (419 loc) · 23 KB

File metadata and controls

539 lines (419 loc) · 23 KB
myst
html_meta
description keywords
The eOn release workflow: cog-driven version bump, towncrier changelog, PyPI publish, GitHub release, conda-forge feedstock, and incomplete-release recovery.
eOn release, cocogitto, towncrier, conventional commits, PyPI, conda-forge feedstock, atomistic-cookbook

Release workflow

eOn releases use cocogitto (cog) 7+ to derive the next semver tag from the conventional-commit log, run the mechanical bump steps, create an atomic signed tag, and emit the source tarball. Towncrier owns CHANGELOG.md. Tag push triggers .github/workflows/release.yml (GitHub Release + tarball asset + optional PyPI). Maintainers then open a conda-forge/eon-feedstock PR.

This page is the maintainer source of truth. Patterns are adapted from rgpot (release assert / release-prepare / towncrier CI / staged release.yml) to eOn surfaces (pyproject.toml, pixi.toml, condaEnvs/, meson via client/get_version.py).

Version surfaces (single semver)

Surface Role
pyproject.toml project.version Source of truth for Python/meson-python; docs conf.py, client/get_version.py, meson/CMake read here
pixi.toml version Lockstep mirror (cog sed on bump)
CHANGELOG.md Towncrier-only; ## [X.Y.Z](...) section per cut
docs/source/releases/vX.Y.Z/ Curated Sphinx release notes (not auto-generated by cog)
Git tag vX.Y.Z Annotated (prefer signed) tag; drives release.yml
GH asset eon-vX.Y.Z.tar.xz git archive tarball (not GitHub auto-archive; hash stability)
PyPI project eon-akmc Wheel/sdist via meson-python; eon on PyPI is taken (EoN epidemics — do not use). Import package stays eon/. Config: ci/gha/pypi.ncl
conda-forge eon Feedstock consumes GH tarball URL + sha256

Do not hand-edit a ## [X.Y.Z] block above the towncrier marker; fragments under docs/newsfragments/ are the only input for the next section.

Gate locally or in CI:

python3 scripts/release_assert.py              # pyproject == pixi semver
python3 scripts/release_assert.py 2.14.0 --require-changelog
python3 scripts/release_assert.py 2.14.0 --notes /tmp/release_notes.md

Tooling map

Tool Role
cocogitto (cog) 7+ Bump, version commit, annotated tag
towncrier CHANGELOG.md from docs/newsfragments/
scripts/release_assert.py Lockstep + optional CHANGELOG section gate (rgpot potctl release assert analogue)
.github/workflows/release.yml On v*: gate → tarball → GH release → PyPI (stable only)
.github/workflows/release-prepare.yml Advisory dry-run on PRs / workflow_dispatch
.github/workflows/towncrier.yml towncrier check when shipped paths change
SECURITY.md Supported versions + report path
.github/CODEOWNERS Release-surface review hints

CI generation (Nickel, rgpot-style)

Release-related Actions workflows are authored in Nickel under ci/gha/ and exported to .github/workflows/ (same contract as rgpot's ci/gha/workflow.nclbuild.yml; eOn applies it to release workflows since we do not yet own a combinatorial build matrix in Nickel). Internal write-up context: rgpot release_process_v1.2.0_lockstep_potctl_cog_towncrier.org (Nickel section: matrix is Nickel-owned; release publish can stay YAML in rgpot — here release publish is Nickel-owned so the process PR cannot drift from hand-edited YAML).

Nickel Generated (commit both; do not hand-edit YAML alone)
ci/gha/common.ncl shared pins/helpers
ci/gha/release.ncl .github/workflows/release.yml
ci/gha/release_prepare.ncl .github/workflows/release-prepare.yml
ci/gha/towncrier_check.ncl .github/workflows/towncrier.yml
./ci/gha/gen.sh
# or: pixi r -e cigen gen-gha   # once cigen env/lock is available
nickel export --format yaml ci/gha/release.ncl -o .github/workflows/release.yml

See ci/gha/README.md. Other ci_*.yml workflows remain hand-maintained until ported.

Maintainer secrets / one-time setup

Do once per GitHub repo (Settings):

Secret / config Used by
PYPI_API_TOKEN or trusted publisher (PyPI → GitHub, environment release) release.yml job pypi
Optional environment release with required reviewers Gate publishes
GPG/signing keys on maintainer machines Prefer git tag -s / signed commits; GHA does not replace maintainer GPG

PyPI first-time (eon-akmc; do not use eon — already taken by EoN epidemics):

  1. Create account at pypi.org; enable 2FA.

  2. Preferred: configure Trusted Publishing for owner/repo TheochemUI/eOn (or your fork), workflow release.yml, environment release, project name eon-akmc (see ci/gha/pypi.ncl). Then release.yml uses pypa/gh-action-pypi-publish with OIDC (no long-lived token).

  3. Fallback: create API token scoped to project eon-akmc, store as repo secret PYPI_API_TOKEN (pypi-... prefix).

  4. Local dry-run (no upload):

    python3 -m pip install build twine
    python3 -m build
    twine check dist/*
  5. Never twine upload by hand unless CI is down; the tag workflow is the source of truth for stable tags.

Multi-package monorepo: fat tarball vs splits

One monorepo, two distribution shapes (both can ship from the same tag train):

Shape Artifact Primary consumer
Fat tree eon-vX.Y.Z.tar.xz (git archive of the full tag) conda-forge eon-feedstock, EasyBuild, source builds
Splits PyPI eon-akmc, pyeonclient, eon-schema, … pip/uv focused installs

Schema/package layout may be cleaned up over time. The release contract is: always produce one complete fat archive for the conda-forge recipe, and also publish splits when those packages have independent consumers.

Fat tarball (feedstock path)

  • Built by release.yml: git archive --format=tar "$TAG" | xz -9 > eon-v….tar.xz
  • Must be a complete monorepo snapshot the feedstock can build (schema/, eon/, client/, tools, meson, and any packages/ the tree uses).
  • Cap’n Proto authoring: schema/eon_params.capnp. After edits: python tools/params_ssot/codegen.py then ./packages/eon-schema/scripts/sync_ssot_into_package.sh (vendors into the split for PyPI). Commit generated outputs so the fat archive is self-contained.
  • Feedstock continues to use only the fat tarball URL + sha256 — no new conda package required when adding PyPI splits.

Split PyPI packages

Project Location Notes
eon-akmc root (import eon) Full-tree tag release job
pyeonclient pyproject-pyeonclient.toml Wheel CI; optional [models]
eon-schema packages/eon-schema/ Independent 0.y.z; vendored SSoT; see package PUBLISHING.md

Detail: packages/eon-schema/PUBLISHING.md.

Monorepo developer docs (package map, CI matrix) can expand later under docs/source/devdocs/ without changing this fat/split release contract.

conda-forge first-time: feedstock already exists at conda-forge/eon-feedstock. New maintainers request access via the feedstock recipe/recipe.yaml maintainers list / staged recipes process.

Windows and conda-forge (issue #15)

win-64 must ship in-tree Fortran (EAM/GAGAFE-class pots), not a Fortran-free stub. See conda-forge/eon-feedstock#15, feedstock build.bat, and Windows Compatibility and Scientific Computing (stack sizes, MinGW/MSVC ABI, xtb import libs).

Platform compiler('fortran') meson (feedstock / GHA Windows)
linux / osx yes defaults (with_fortran on unless opted out)
win-64 yes (m2w64 gfortran; no MSVC Fortran) --default-library=static -Dwith_fortran=true -Dwith_cuh2=true
  • Meson already adds /STACK:16777216 on MSVC links (Windows 1 MB default overflows large Fortran stack arrays; Linux is ~8 MB).
  • MinGW-built xtb still needs the dumpbin / lib /DEF: MSVC import library step in feedstock build.bat (C++/torch stay MSVC).
  • CuH2 is on with the rest of in-tree Fortran (iso_c_binding via m2w64 gfortran).
  • Runtime pot DLLs (LoadLibrary) are complementary, not a substitute for in-tree Fortran in the conda package.
  • Before feedstock go-live after a cut: confirm Azure win-64 is green with with_fortran=true and with_cuh2=true on the version/sha256 PR.

1. Pre-release checklist

Before invoking cog bump, verify:

  • Every PR intended for the release is merged into main.

  • CHANGELOG.md is up to date through the previous release only — towncrier will prepend the new section from docs/newsfragments/*. If fragments are missing for user-facing changes, add them first (uvx towncrier create or see docs/newsfragments/README.md).

  • Preview fragments: uvx towncrier build --draft (or pixi r -e dev towncrier build --draft if the env has towncrier).

  • docs/source/releases/v<X.Y.Z>/release-notes.md and index.md exist and are linked from docs/source/releases/index.md (cog cannot generate these — they are curated).

  • CI is green on main (build, docs, pre-commit, serve/xtb/metatomic as relevant).

  • pixi install --locked succeeds cleanly (so the lockfile regeneration hook has something stable to work from).

  • python3 scripts/release_assert.py passes (surfaces already agree, or you are about to let cog set both).

  • PyPI dry-run considered: python3 -m build && twine check dist/* on a clean tree (optional but recommended before first-ever PyPI cut).

  • Incomplete prior release reconciled — see § Incomplete / stuck releases. Do not start a new semver while vN.M.P is tagged but GH asset / PyPI / feedstock are unfinished unless you explicitly choose finalize-first or supersede.

  • Downstream integration test passes (blocking for metatomic consumers). The lab-cosmo/atomistic-cookbook example eon-pet-neb exercises eOn + PET-MAD on the oxadiazole NEB. Point a cookbook checkout at the prospective release commit (override eon in its environment.yml, or pip install -e the eOn tree) and run:

    nox -e eon-pet-neb

    Pass = nox exits 0, gallery artefacts regenerate, client_traceback.log in the example dir is empty. A regression blocks the release.

2. Running cog bump

Happy path on main (cocogitto 7+):

cog bump --auto --annotated 'Release v<X.Y.Z>

<one-paragraph body listing headline features>
See CHANGELOG.md.'

Override inference with --minor, --major, or --version X.Y.Z when needed (e.g. only feat(docs) since last tag but you want a patch).

Dry-run first (also available via Actions → Release prepare workflow dispatch):

cog bump --auto --dry-run
# or pin: cog bump --version 2.15.0 --dry-run

What cog bump does

Defined in cog.toml:

  1. Picks the next version (semver from conventional commits since last tag, or from the flag).
  2. Runs pre_bump_hooks:
    • sed the new version into pyproject.toml and pixi.toml.
    • uvx pixi-to-conda-lock pixi.lock --output condaEnvs regenerates condaEnvs/*.conda-lock.yml.
    • pixi r -e dev towncrier build --version {{version}} --date $(date +%Y-%m-%d) --yes consumes docs/newsfragments/ and prepends CHANGELOG.md.
    • python3 scripts/release_assert.py {{version}} --require-changelog fails the bump if surfaces diverged or the new section is missing.
  3. Stages the diff and writes the release commit (chore(version): <X.Y.Z> by default).
  4. Creates the annotated tag v<X.Y.Z> (configure local git for signed tags if policy requires).
  5. Runs post_bump_hooks:
    • Prints reminder that release.yml on tag push builds the tarball, creates the GH release, and publishes PyPI (stable only).
    • Optional local tarball (also produced in CI): git archive --format=tar v{{version}} | xz -9 > eon-v{{version}}.tar.xz

cog.toml sets disable_changelog = true (towncrier owns CHANGELOG.md) and branch_whitelist = ["main"] so cog bump refuses non-main branches.

3. Post-bump actions

  1. Inspect the release commit:

    git show HEAD
    python3 scripts/release_assert.py --print-version

    Expect: version bumps in pyproject.toml/pixi.toml, regenerated condaEnvs/*.conda-lock.yml, new CHANGELOG.md section, consumed newsfragments deleted, plus your curated docs/source/releases/vX.Y.Z/ pages if you added them in the same commit or just before.

  2. Push the commit and tag together:

    git push origin main --follow-tags
  3. GitHub Actions (release.yml) on the tag:

    • gate: checkout, release_assert.py $VERSION --require-changelog, emit notes artifact (CHANGELOG section or curated release-notes.md).
    • tarball: git archiveeon-vX.Y.Z.tar.xz artifact.
    • gh-release: create/update GitHub Release with tarball attached and notes body (idempotent if release already exists — uploads asset if missing).
    • pypi (stable tags only, not X.Y.Z-rc.N): build + publish via trusted publisher or PYPI_API_TOKEN.

    Manual fallback if Actions is unavailable:

    git archive --format=tar v<X.Y.Z> | xz -9 > eon-v<X.Y.Z>.tar.xz
    gh release create v<X.Y.Z> eon-v<X.Y.Z>.tar.xz \
      --title v<X.Y.Z> \
      --notes-file docs/source/releases/v<X.Y.Z>/release-notes.md

    The feedstock URL assumes this exact asset name; do not rename it.

  4. Confirm channels before feedstock PR:

    • GitHub release vX.Y.Z exists with asset eon-vX.Y.Z.tar.xz
    • python3 scripts/release_assert.py X.Y.Z --require-changelog on the tagged commit
    • PyPI shows eon-akmc==X.Y.Z (stable only; check https://pypi.org/project/eon-akmc/X.Y.Z/) or document intentional skip for this cut
    • Then proceed to §4 conda-forge

4. conda-forge feedstock bump

Once the GitHub release is published (asset URL resolves), update conda-forge/eon-feedstock:

  1. Fork/clone feedstock; branch release-<X.Y.Z> off main.

  2. In recipe/recipe.yaml, set context.version to "<X.Y.Z>".

  3. Compute tarball sha256 and replace the first source[].sha256:

    curl -sSL "https://github.com/TheochemUI/eOn/releases/download/v<X.Y.Z>/eon-v<X.Y.Z>.tar.xz" \
      | sha256sum
  4. Reset build.number: 0 (or 0 under build: per recipe schema).

  5. Verify patches still apply (fix_vesin_const.patch, fix_capnpc.patch, or current recipe patch set); regenerate against the new tree if not.

  6. Commit, push, open PR against conda-forge/eon-feedstock (title e.g. eon <X.Y.Z>).

  7. Watch Azure Pipelines (linux-64, osx-64, osx-arm64, win-64) via cf-ci watch conda-forge/eon-feedstock#<N> or the PR checks UI.

  8. After merge, verify conda install -c conda-forge eon=<X.Y.Z> in a clean env.

Paste-ready PR body template (fill version/sha256):

## eon <X.Y.Z>

Bump eon to <X.Y.Z> from upstream GitHub release.

- Source: https://github.com/TheochemUI/eOn/releases/download/v<X.Y.Z>/eon-v<X.Y.Z>.tar.xz
- sha256: <PASTE>
- build number reset to 0
- Patches: verified apply / updated as needed

Checklist:
- [ ] context.version == <X.Y.Z>
- [ ] source sha256 matches tarball
- [ ] build.number 0
- [ ] patches apply
- [ ] CI matrix green

5. Incomplete / stuck releases

eOn has had cuts where version / tag / CHANGELOG / release-notes exist but GitHub release asset, PyPI, and/or feedstock were never finished, and/or commits landed on main after the version chore without a new semver.

Example state at process-nail-down time: tip may report 2.14.0 in pyproject.toml/pixi.toml with tag v2.14.0 and a CHANGELOG section, while post-tag commits (fixes/features) sit on main and newsfragments may be empty again. Treat that as incomplete multi-channel state, not a green field.

Decision tree

Is there a tag vN.M.P on origin?
├─ NO  → normal path: fragments + curated notes + cog bump (§1–3)
└─ YES
   ├─ Are ALL channels done for N.M.P?
   │    (GH release + eon-vN.M.P.tar.xz asset + PyPI eon==N.M.P [stable] + feedstock PR/merge)
   │    ├─ YES, and main == tag tip (or only docs/ci after tag)
   │    │    → start next semver (fragments for post-tag user-facing work)
   │    └─ YES, but main has user-facing commits after tag
   │         → next semver (do NOT retag N.M.P; add fragments for post-tag work)
   └─ NO (missing GH asset and/or PyPI and/or feedstock)
        ├─ main tip is still the release commit (or only trivial fixes you accept as N.M.P)
        │    → FINALIZE N.M.P (below); do not cog-bump a new version yet
        └─ main has diverged with commits that must ship as a newer semver
             → SUPERSEDE: document N.M.P as partially published; cut N.M.P+1 (or minor)
               with fragments covering post-tag work; finalize missing N.M.P channels
               only if consumers need that exact tag (usually: finish tarball/GH for
               historical accuracy, skip re-PyPI if you never uploaded, feedstock only
               for the version you intend conda users to get)

Finalize a stuck vN.M.P (no new semver)

On the tagged commit (detached or temporary branch at the tag):

git fetch origin tag vN.M.P
git switch --detach vN.M.P
python3 scripts/release_assert.py N.M.P --require-changelog
git archive --format=tar vN.M.P | xz -9 > eon-vN.M.P.tar.xz
# If GH release missing or asset missing:
gh release create vN.M.P eon-vN.M.P.tar.xz --title vN.M.P \
  --notes-file docs/source/releases/vN.M.P/release-notes.md
# or: gh release upload vN.M.P eon-vN.M.P.tar.xz --clobber

PyPI: only if N.M.P was never uploaded and you still want that exact version on the index (metadata must match the tag tree). If post-tag main differs and you already decided to supersede, skip PyPI for N.M.P and publish only the new semver.

Feedstock: open/update PR only for the version you want conda-forge to ship (usually the semver you fully finalized, often the newest complete cut).

Supersede with a new semver

  1. List post-tag commits: git log vN.M.P..main --oneline.
  2. Add docs/newsfragments/* for user-facing post-tag changes (and any that lacked fragments at N.M.P time if they still lack a changelog voice).
  3. Add docs/source/releases/v<NEW>/ pages; link from releases/index.md.
  4. On main: cog bump --version <NEW> (or --auto / --patch / --minor).
  5. Complete §3 and §4 for NEW only.
  6. Optionally still attach eon-vN.M.P.tar.xz to the old GH release for reproducibility; do not double-publish conflicting PyPI/conda builds.

Anti-patterns

  • Retagging vN.M.P to a different commit (breaks tarball/sha256 consumers).
  • Running cog bump while intending to finalize an incomplete same version (creates N.M.P+1 or fails assert if versions already at next).
  • Opening a feedstock PR before the GH asset URL returns 200.
  • Publishing PyPI from a dirty/unreleased tree that does not match the tag.

6. Pre-release / RC tags

cog bump --version 2.15.0-rc.1
git push origin main --follow-tags
  • release.yml creates a prerelease GitHub Release.
  • Skips PyPI (stable index only; avoid yanking/pollution).
  • Promote with a normal stable bump to 2.15.0 and full fragment set as needed.

7. Manual fallback (cog unavailable)

When cog bump fails mid-run (hook error, unexpected prompt) no commit is written yet — fix and re-run. Hand sequence (reference commits like 1edaf4d8 chore: release v2.12.0):

sed -i 's/^version = "[^"]*"/version = "<X.Y.Z>"/' pyproject.toml pixi.toml
uvx pixi-to-conda-lock pixi.lock --output condaEnvs
pixi r -e dev towncrier build --version <X.Y.Z> \
  --date "$(date +%Y-%m-%d)" --yes
# Curate docs/source/releases/v<X.Y.Z>/{index.md,release-notes.md} + toctree
python3 scripts/release_assert.py <X.Y.Z> --require-changelog
git add -A
git commit -m "chore: release v<X.Y.Z>"
git tag -s -a v<X.Y.Z> -m "Release v<X.Y.Z>

<body>"
git archive --format=tar v<X.Y.Z> | xz -9 > eon-v<X.Y.Z>.tar.xz

Then §3 push and channel completion.

8. API / Doxygen documentation gate

rgpot maintains docs/source/Doxyfile_*.cfg and integrates Doxygen output into the Sphinx site. eOn does not yet ship an equivalent in-tree Doxygen pipeline for the C++ client/ tree; Python API docs use Sphinx autodoc/autodoc2 in the existing docs/ build (ci_docs.yml / docs/source/devdocs/docbuild.md).

Release gate (current policy):

  • Sphinx docs build is green on main (ci_docs.yml or local pixi r -e docs … / project-documented doc target).
  • Curated release notes exist under docs/source/releases/vX.Y.Z/.
  • No broken “run doxygen” step is required for a cut.

Deferred (explicit non-goal until a dedicated docs PR): full Doxygen + doxyrest/breathe wiring for client/**/*.h analogous to rgpot. Track as follow-up; do not block semver cuts solely on Doxygen absence. When added, extend this section with the generate command and wire it into ci_docs.yml plus this checklist.

9. Paper tags

Preprints and paper-related tags do not need the release machinery; lightweight annotated tags only:

git tag -a arXiv_2510.06030v3 45c077cb
# feat(arxiv): descriptive one-line summary
#
# body, including the arXiv link

10. Changelog fragments (every user-facing PR)

uvx towncrier create --content "Short user-facing note." 42.added.md
# or interactive:
uvx towncrier create

Types (towncrier.toml): security, removed, deprecated, added, dev, changed, fixed. Preview: uvx towncrier build --draft.

CI runs towncrier check on PRs when shipped paths change (towncrier.yml). Pure docs / ci / chore commits may omit fragments if they avoid those paths.

11. How to verify this release-process PR (no live cut required)

python3 scripts/release_assert.py
python3 scripts/release_assert.py 2.14.0 --require-changelog
python3 -m pytest tests/release/ -q
uvx towncrier build --draft   # ok if no fragments (empty output / skip)
python3 -c "import tomllib, pathlib; tomllib.loads(pathlib.Path('cog.toml').read_text()); print('cog.toml ok')"
# optional if build deps present:
# python3 -m build && twine check dist/*

Confirm docs name GH release/tarball, PyPI, and conda-forge/eon-feedstock; incomplete-release recovery (§5); and Doxygen deferral (§8).