From 989ce66d9c2226cc4b0e9de888e8f917216aa873 Mon Sep 17 00:00:00 2001 From: Bishwas Jha Date: Sun, 19 Apr 2026 07:45:06 +0200 Subject: [PATCH] ci(publish): add PyPI publish pipeline via Trusted Publishing (OIDC) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a two-stage publish workflow that ships python_maithili to PyPI without storing long-lived API tokens in GitHub. Three triggers, three behaviours: - release.published (non-prerelease) -> publishes to production PyPI (environment: pypi, with optional manual-approval gate) - release.published (prerelease) OR workflow_dispatch target=testpypi -> publishes to TestPyPI (environment: testpypi, staging/verification) - workflow_dispatch target=build-only -> builds + twine-checks, no network publish (credential-free dry run) Every path runs the same build job first: python -m build, twine check --strict, install the wheel in a clean venv, run the full pytest suite against the installed wheel. Only if the wheel itself passes the test suite does anything get uploaded. The production job additionally verifies that the GitHub Release tag (v0.X.Y) matches __version__ in the package and refuses to publish on mismatch — preventing accidental version drift. Trusted Publishing was chosen over API tokens because: - No secrets to rotate or leak. - Scoped per environment: the pypi environment can be gated with required reviewers and tag-only deployment branches. - PyPI binds the trust to (owner, repo, workflow filename, environment), so a fork or a different workflow cannot publish on your behalf even if it runs on this repo. The one-time setup (register Trusted Publisher on PyPI and pending publisher on TestPyPI, create pypi + testpypi GitHub Environments) is documented in the new docs/RELEASE.md, along with the routine release checklist, version-bump convention, and recovery procedure for bad releases. build + twine added to requirements-dev.txt so contributors can reproduce the build locally before opening a release PR. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/publish.yml | 192 ++++++++++++++++++++++++++++++++++ CHANGELOG.md | 14 ++- CONTRIBUTING.md | 8 ++ docs/RELEASE.md | 163 +++++++++++++++++++++++++++++ requirements-dev.txt | 4 + 5 files changed, 380 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/publish.yml create mode 100644 docs/RELEASE.md diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..0c12493 --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,192 @@ +name: Publish to PyPI + +# Trusted Publishing (OIDC) — no long-lived API tokens stored. +# +# Triggers: +# - release.published on a non-prerelease GitHub Release +# → publishes to PyPI (environment: pypi) +# - release.published on a pre-release GitHub Release +# → publishes to TestPyPI (environment: testpypi) +# - workflow_dispatch with target=testpypi +# → publishes to TestPyPI (staging verification) +# - workflow_dispatch with target=build-only +# → builds + twine-checks, no network publish (dry run) +# +# One-time setup required before the publish jobs succeed: +# See docs/RELEASE.md for Trusted Publisher configuration on +# pypi.org and test.pypi.org. + +on: + release: + types: [published] + workflow_dispatch: + inputs: + target: + description: "Publish target" + required: true + default: "build-only" + type: choice + options: + - build-only + - testpypi + +concurrency: + group: publish-${{ github.event_name }}-${{ github.ref }} + cancel-in-progress: false + +permissions: + contents: read + +jobs: + # ------------------------------------------------------------------ + # Build sdist + wheel, run twine check, run the full test suite + # against the built wheel so we never ship a wheel that fails tests. + # ------------------------------------------------------------------ + build: + name: Build distributions + runs-on: ubuntu-latest + timeout-minutes: 10 + outputs: + version: ${{ steps.version.outputs.value }} + + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + cache: pip + cache-dependency-path: requirements-dev.txt + + - name: Install build tooling + run: | + python -m pip install --upgrade pip + pip install build twine + + - name: Extract version from package + id: version + run: | + v=$(python -c "import re, pathlib; print(re.search(r'^__version__\s*=\s*[\"\']([^\"\']+)', pathlib.Path('maithili_dsl/__init__.py').read_text(), re.M).group(1))") + echo "value=$v" >> "$GITHUB_OUTPUT" + echo "Package version: $v" + + - name: Build sdist + wheel + run: python -m build + + - name: twine check + run: twine check --strict dist/* + + - name: Install built wheel in clean venv + run: | + python -m venv /tmp/smoketest + /tmp/smoketest/bin/pip install dist/*.whl + /tmp/smoketest/bin/python -m maithili_dsl --version + /tmp/smoketest/bin/python_maithili examples/hello.dmai + + - name: Run tests against the installed wheel + run: | + /tmp/smoketest/bin/pip install -r requirements-dev.txt + /tmp/smoketest/bin/pytest -q --no-cov + + - name: Upload distributions artifact + uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + if-no-files-found: error + retention-days: 7 + + # ------------------------------------------------------------------ + # TestPyPI — fires on workflow_dispatch (target=testpypi) or on + # a pre-release GitHub Release. Safe to re-run with a different + # version number. + # ------------------------------------------------------------------ + publish-testpypi: + name: Publish to TestPyPI + needs: [build] + runs-on: ubuntu-latest + timeout-minutes: 10 + if: | + (github.event_name == 'workflow_dispatch' && inputs.target == 'testpypi') || + (github.event_name == 'release' && github.event.release.prerelease == true) + environment: + name: testpypi + url: https://test.pypi.org/p/python-maithili + permissions: + id-token: write # OIDC token for Trusted Publishing + steps: + - name: Download distributions + uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + + - name: Publish to TestPyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + skip-existing: false + verbose: true + + - name: Summary + run: | + echo "### Published to TestPyPI :rocket:" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "Version: **${{ needs.build.outputs.version }}**" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "\`\`\`bash" >> "$GITHUB_STEP_SUMMARY" + echo "pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ python-maithili==${{ needs.build.outputs.version }}" >> "$GITHUB_STEP_SUMMARY" + echo "\`\`\`" >> "$GITHUB_STEP_SUMMARY" + + # ------------------------------------------------------------------ + # Production PyPI — fires only on a non-prerelease GitHub Release. + # This is the single authoritative way to publish a real version. + # ------------------------------------------------------------------ + publish-pypi: + name: Publish to PyPI + needs: [build] + runs-on: ubuntu-latest + timeout-minutes: 10 + if: | + github.event_name == 'release' && github.event.release.prerelease == false + environment: + name: pypi + url: https://pypi.org/p/python-maithili + permissions: + id-token: write # OIDC token for Trusted Publishing + steps: + - name: Download distributions + uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + + - name: Verify release tag matches package version + env: + TAG: ${{ github.event.release.tag_name }} + VERSION: ${{ needs.build.outputs.version }} + run: | + expected="v${VERSION}" + if [ "$TAG" != "$expected" ]; then + echo "::error::Release tag '$TAG' does not match package version '$expected'. Refusing to publish." + exit 1 + fi + echo "Tag $TAG matches package version $VERSION — proceeding." + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + skip-existing: false + verbose: true + + - name: Summary + run: | + echo "### Published to PyPI :tada:" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "Version: **${{ needs.build.outputs.version }}**" >> "$GITHUB_STEP_SUMMARY" + echo "" >> "$GITHUB_STEP_SUMMARY" + echo "\`\`\`bash" >> "$GITHUB_STEP_SUMMARY" + echo "pip install python-maithili==${{ needs.build.outputs.version }}" >> "$GITHUB_STEP_SUMMARY" + echo "\`\`\`" >> "$GITHUB_STEP_SUMMARY" diff --git a/CHANGELOG.md b/CHANGELOG.md index b3c4cc3..222e0b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -_Nothing yet. Open a PR to add your change here._ +### Added +- **PyPI publish pipeline** (`.github/workflows/publish.yml`) using + Trusted Publishing (OIDC). Triggers: GitHub Release for production + PyPI, pre-release or `workflow_dispatch` for TestPyPI, `build-only` + dispatch for a credential-free dry run. The workflow builds sdist + + wheel, runs `twine check --strict`, installs the wheel in a clean + venv, runs the full pytest suite against the installed wheel, and + verifies the release tag matches `__version__` before publishing. +- **`docs/RELEASE.md`** — step-by-step release process including + one-time Trusted Publisher setup on PyPI + TestPyPI, routine + release flow, version-bump convention, and recovery procedure for + bad releases. +- **`build` and `twine`** added to `requirements-dev.txt`. ## [0.3.0] — 2026-04-17 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8a9c50a..ea79560 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -87,4 +87,12 @@ import whitelist (`MAITHILI_MODULES`), or the safe-builtins list 4. Open a Pull Request targeting `development`. The PR template will prompt you for test evidence and any security considerations. +## 📦 Releasing + +Releases are cut by a project maintainer following the checklist in +[`docs/RELEASE.md`](docs/RELEASE.md). The short version: +publishing is triggered by creating a **GitHub Release** and uses +**Trusted Publishing (OIDC)** — no long-lived PyPI tokens are stored +in the repo. + Thanks for your interest! ❤️ diff --git a/docs/RELEASE.md b/docs/RELEASE.md new file mode 100644 index 0000000..3c15485 --- /dev/null +++ b/docs/RELEASE.md @@ -0,0 +1,163 @@ +# Release Process + +This document describes how to cut a new release of `python_maithili` +to PyPI. The workflow is **`.github/workflows/publish.yml`** and uses +**Trusted Publishing (OIDC)** — no API tokens are stored in GitHub. + +--- + +## One-time setup (required before the first publish succeeds) + +### 1. Create GitHub Environments + +In the repo, go to **Settings → Environments** and create two +environments — they give us a place to pin the OIDC trust and (for +production) an optional manual-approval gate. + +#### `testpypi` +- No required reviewers. +- Deployment branches: all branches (we publish to TestPyPI from + feature branches too). + +#### `pypi` +- **Required reviewers**: add yourself. Every production publish will + wait for explicit approval. This is the final safety gate. +- Deployment branches: protected tags matching `v*` only + (`v[0-9]+.[0-9]+.[0-9]+` and `v[0-9]+.[0-9]+.[0-9]+-*`). + +### 2. Register the Trusted Publisher on PyPI + +Go to +(owner only) and add a new "Trusted publisher": + +| Field | Value | +|---------------------|--------------------------------| +| Owner | `alphacrack` | +| Repository | `python-maithili-dsl` | +| Workflow filename | `publish.yml` | +| Environment name | `pypi` | + +### 3. Register the Trusted Publisher on TestPyPI + +TestPyPI doesn't have the project yet, so use a **pending publisher**. +Go to and under +"Pending trusted publishers" add: + +| Field | Value | +|---------------------|--------------------------------| +| PyPI Project Name | `python-maithili` | +| Owner | `alphacrack` | +| Repository | `python-maithili-dsl` | +| Workflow filename | `publish.yml` | +| Environment name | `testpypi` | + +The pending publisher converts to a real one the first time a workflow +run successfully creates the project on TestPyPI. + +--- + +## Routine release workflow + +### A. Dry-run build (no publish) + +Push a PR or run **Actions → Publish to PyPI → Run workflow → `build-only`**. +This executes the full pipeline except the publish step: + +- Build sdist + wheel. +- `twine check --strict` metadata validation. +- Install the wheel in a clean venv. +- Run the full pytest suite against the installed wheel. +- Upload the `dist/` artifact for inspection. + +No credentials are required and nothing touches the network package +indexes. Use this whenever you want to confirm the package will build. + +### B. Publish to TestPyPI (staging verification) + +Two ways to trigger: + +1. **Manual**: Actions → Publish to PyPI → Run workflow → `testpypi`. +2. **Automatic**: create a GitHub **pre-release** (check "This is a + pre-release" when making the Release). Tag format: `v0.X.Y-rc1`, + `v0.X.Y-dev1`, etc. + +Version numbers you publish to TestPyPI are **separate** from +production PyPI — TestPyPI has its own index. Verify the install: + +```bash +pip install \ + --index-url https://test.pypi.org/simple/ \ + --extra-index-url https://pypi.org/simple/ \ + python-maithili==0.X.Y +python_maithili --version +``` + +Any version on TestPyPI cannot be re-used. If you need to iterate, +bump the version (`0.3.0.dev0`, `0.3.0.dev1`, ...) in +`maithili_dsl/__init__.py` before re-triggering. + +### C. Publish to production PyPI + +**Pre-flight checklist:** + +- [ ] `CHANGELOG.md` has a section for the new version with date filled in. +- [ ] `maithili_dsl/__init__.py` has `__version__` set to the new version. +- [ ] `pyproject.toml` has matching `version = "..."`. +- [ ] `pytest` runs green on `main` CI. +- [ ] The same version was successfully published to TestPyPI and installed + cleanly (step B). + +**Cut the release:** + +1. Ensure `main` has the final release commit on it (all three version + strings updated, CHANGELOG finalized). +2. Create a new **GitHub Release** (not a pre-release): + - Tag: `v0.X.Y` (must match `__version__` exactly; the workflow + enforces this and refuses to publish on mismatch). + - Target: `main`. + - Title: `v0.X.Y`. + - Body: the corresponding CHANGELOG section. + - Leave "This is a pre-release" **unchecked**. +3. Click **Publish release**. The workflow fires. +4. If the `pypi` environment has required reviewers, approve the + deployment in the Actions run. +5. After the run finishes, verify: + + ```bash + pip install python-maithili==0.X.Y + python_maithili --version + ``` + +--- + +## Version bump convention + +This project uses [Semantic Versioning](https://semver.org/): + +| Change | Bump | +|------------------------------------------------|-------| +| Breaking API change (exit code semantics, etc) | MAJOR | +| New keyword / module / feature | MINOR | +| Bugfix, doc change, internal refactor | PATCH | + +Version is tracked in three places that must stay in sync: + +1. `maithili_dsl/__init__.py` — `__version__` (single source of truth). +2. `pyproject.toml` — `version`. +3. `CHANGELOG.md` — section header and date. + +The publish workflow's "Verify release tag matches package version" +step will fail the production publish if these drift. + +--- + +## Recovery: what if a bad version reaches PyPI? + +PyPI does not allow re-uploading the same filename. If you published +a broken release: + +1. **Yank** (do not delete) the broken version on PyPI: + → + pick the version → "Yank release". This hides it from `pip install` + without breaking already-pinned consumers. +2. Bump PATCH and release the fix via the normal flow above. diff --git a/requirements-dev.txt b/requirements-dev.txt index 3a4df92..d3b271c 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -3,3 +3,7 @@ pytest>=7.4,<9 pytest-cov>=4.1,<6 pytest-timeout>=2.2,<3 + +# Build + release tooling (used by docs/RELEASE.md workflow). +build>=1.0,<2 +twine>=4.0,<6