Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ This repo is a Copier template. `template/` holds the generated project as `.jin
## Run every gate (`just ci`)

```bash
just ci # fmt-check + lint + typecheck + test + audit, then "ci: all gates passed"
just ci # fmt-check + lint + typecheck + test + policy + audit, then "ci: all gates passed"
```

`just ci` is the complete local gate — `ci: fmt-check lint typecheck test audit` ending `@echo "ci: all gates passed"`, with `verify` a bare alias (`verify: ci`). It **mirrors** `template/justfile.jinja`'s `ci`: the template chains `audit` when `enable_dependency_audit` is on (it is), so the faithful maintainer recipe includes it. `test` is the heavy generation matrix, so `just ci` needs a roomy `TMPDIR` — the default 4G tmpfs `/tmp` can overflow (export e.g. `TMPDIR=/path/to/roomy/dir` first). The fast inner loop is `just fmt-check lint typecheck` (one invocation, no matrix, no network); there is deliberately **no** `check:` recipe (the template ships none). `audit` is non-hermetic (it queries the OSV/PyPI advisory DB), so `just ci` reaches the network exactly as `just audit` does; scanners (`just scan`) deliberately stay **off** `ci` (CI-only), matching the template.
`just ci` is the complete local gate — `ci: fmt-check lint typecheck test policy audit` ending `@echo "ci: all gates passed"`, with `verify` a bare alias (`verify: ci`). It **mirrors** `template/justfile.jinja`'s `ci`: the template chains `policy` and `audit` when `enable_policy_tests`/`enable_dependency_audit` are on (they are), so the faithful maintainer recipe includes both. `test` is the heavy generation matrix, so `just ci` needs a roomy `TMPDIR` — the default 4G tmpfs `/tmp` can overflow (export e.g. `TMPDIR=/path/to/roomy/dir` first). The fast inner loop is `just fmt-check lint typecheck` (one invocation, no matrix, no network); there is deliberately **no** `check:` recipe (the template ships none). `audit` is non-hermetic (it queries the OSV/PyPI advisory DB), so `just ci` reaches the network exactly as `just audit` does; scanners (`just scan`) deliberately stay **off** `ci` (CI-only), matching the template.

This maintainer `just ci` is distinct from the **downstream's** `just ci` referenced under "Run the tests": `just test` renders each project in the answer matrix and runs *its* `just ci` — a different, generated recipe.

CI does **not** run `just ci`. It keeps parallel per-gate jobs (`test`/`typecheck`/`lint`/`scan`), each inlining its command; only `test` fans out over the OS×Python matrix, and the `scan` job runs `uvx`/`mise exec` (not `uv run`); because every gate is enforced independently (audit via the `scan` job's `pip-audit` step), `just ci` here is the local reproduction of the PR gate, not the CI entry point. Pointing the CI jobs at these recipes (the gap-audit "candidate F") is deferred: the `scan` job aggregates pip-audit + semgrep + gitleaks as three steps that don't map onto the `audit`+`scan` recipe split, so it is a real restructure, not a one-line swap, and the follow-up must reconcile the recipe encoding with the parallel jobs.

**Forward-sync:** when the policy-tests (gap #4) and property-tests (gap #9) layers land, add `policy`/`fuzz` to **both** the `ci` recipe **and** this section, to keep mirroring the template's conditional `ci` (`fmt-check lint typecheck test{% if enable_property_tests %} fuzz{% endif %}{% if enable_policy_tests %} policy{% endif %}{% if enable_dependency_audit %} audit{% endif %}`).
**Forward-sync:** the policy-tests layer (gap #4) has landed — `policy` is now a `ci` member (see "Policy gate"). When the property-tests layer (gap #9) lands, add `fuzz` to **both** the `ci` recipe **and** this section, to keep mirroring the template's conditional `ci` (`fmt-check lint typecheck test{% if enable_property_tests %} fuzz{% endif %}{% if enable_policy_tests %} policy{% endif %}{% if enable_dependency_audit %} audit{% endif %}`).

Deliberate divergences from `template/justfile.jinja`'s `ci`: **none in the gate list** — `ci: fmt-check lint typecheck test audit`, `verify: ci`, and the `@echo` are exact. The divergences are structural: the maintainer's CI never runs `just ci` (parallel per-gate jobs — only `test` matrixed — with inline `uv run`/`uvx`/`mise exec` commands, vs the downstream `ci.yml`'s single `just ci`); there is no `check:` fast split (use `just fmt-check lint typecheck`); and candidate F is deferred. (The `audit` recipe's dropped `--no-dev` is a separate divergence documented under "Dependency audit", not a `ci`-list divergence.)
Deliberate divergences from `template/justfile.jinja`'s `ci`: **none in the gate list** — `ci: fmt-check lint typecheck test policy audit`, `verify: ci`, and the `@echo` are exact. The divergences are structural: the maintainer's CI never runs `just ci` (parallel per-gate jobs — only `test` matrixed — with inline `uv run`/`uvx`/`mise exec` commands, vs the downstream `ci.yml`'s single `just ci`); there is no `check:` fast split (use `just fmt-check lint typecheck`); and candidate F is deferred. (The `audit` recipe's dropped `--no-dev` is a separate divergence documented under "Dependency audit", not a `ci`-list divergence.)

## Run the tests

Expand Down Expand Up @@ -80,6 +80,22 @@ Deliberate divergences from the template's dependency-audit layer: `--no-dev` is

Because nothing here re-derives the pin (no Renovate; the generation drift test reads only the *rendered* downstream), **bump every literal by hand, against the template.** pip-audit (`2.10.1`) has three maintainer sites — the `just audit` recipe, the `pip-audit` step in `test-template.yml`, and the prose above — synced to `template/.github/workflows/…scan.yml….jinja` (the only exact-version template site; the template justfile uses unpinned `uv run pip-audit` and template pyproject floors `pip-audit>=2.10`). No `mise.toml` or `pyproject.toml` pip-audit literal exists (uvx-run, unlike gitleaks). **Sync only the pin *value* — never the export flags:** the template's `uv export` keeps `--no-dev`, but the maintainer must not (it exports 0 packages here — see above), so a mechanical sync against the template would silently neuter the gate. (Mirrors the semgrep/gitleaks pin-sync note and the pre-commit "bump both `rev:` pins together" rule.)

## Policy gate (`just policy`)

```bash
just policy # pins the gate config literals so they cannot be silently weakened (stdlib-only)
```

`just policy` runs `uv run pytest tests/policy` — a stdlib-only (`re`/`tomllib`/`pathlib`) suite pinning this repo's gate **config literals** so they cannot be silently weakened: basedpyright `typeCheckingMode == "recommended"` **and** `failOnWarnings is True`; `"ALL" in ruff select`; the first `just <recipe>` of each **fenced** AGENTS.md example is a real justfile recipe (docs-can't-lie); and every third-party `uses:` is `@<40-hex> # v<ver>` (`test_actions_are_sha_pinned` — this **closes the offline half of the SHA-pin policy**, gap #8). It is a `ci` member (`ci: … test policy audit`) and is also auto-collected by bare `just test` (`uv run pytest`, `testpaths=["tests"]`), so CI enforces it inside the existing `test` job with **no** workflow change.

Changing any of those gates (ruff select, type mode, `failOnWarnings`, an Action pin) requires editing the matching test in `tests/policy/test_gates.py` in the **same commit** — that is the point of the suite. **Scope:** it pins those config *literals* — not the ruff `ignore` / `per-file-ignores` lists (each is separately audited; see "Lint & format this repo"), not the recipe *bodies*, and not the CI workflow that runs the gates (candidate F — see "Run every gate").

Deliberate divergences from `template/tests/…/test_gates.py.jinja`: the **coverage-floor** assertion is dropped (the maintainer has no `[tool.coverage]` / no runtime package — nothing to protect; the `policy` recipe likewise drops `--no-cov`, which is invalid with no `pytest-cov`); a **`failOnWarnings is True`** assertion is added (the template ships none — the maintainer pins the type-gate's teeth explicitly, so the suite guards that pin); and the **docs-can't-lie regex is scoped to fenced code blocks** rather than the whole file. The template's naive whole-doc scan still passes here *today*, but it flips to a hard fail the moment this file names a template-only recipe (`fuzz`/`mutate`) in prose — which gap #9 will do. Fenced-only gates the examples you are told to **run** and lets prose name any recipe freely.

**Two invariants this file must hold** (the test reads AGENTS.md as raw text, and the extractor pairs fences *sequentially*): **(a)** never write a literal triple-backtick run in prose — an odd total mispairs every fence after it and sweeps unrelated prose into a "block"; write "fenced code block", never the delimiter. **(b)** Never write a bare `just <name>` for a non-recipe anywhere, prose included — name it without the `just ` prefix (as `fuzz`/`mutate` are named above). (b) is defence-in-depth: it stops a future (a)-slip from turning into a red gate. The suite also scans `#` comments *inside* fences, so the same rule holds there.

The SHA-pin sub-check overlaps the zizmor job (the security control), so its net-new value is annotation hygiene; the config-literals + docs-can't-lie are the substantive net-new gate. `tests/policy/__init__.py` exists because the maintainer's `tests/` is a package (the template ships no such file). **Known limits:** the recipe allowlist regex matches only `[a-z][a-z-]*` names, so a future parametrized or underscored recipe cited in a fenced example would false-positive (broaden the pattern in the same commit); and nothing here asserts that CI still *runs* the gates (candidate F).

## Add a guardrail layer

1. Add an `enable_*` toggle to `copier.yml`.
Expand Down
Loading