diff --git a/AGENTS.md b/AGENTS.md index 3b209631..09832e32 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,8 +4,11 @@ Universal project instructions for AI coding assistants. ## Project Overview -`big-code-analysis` is a Mozilla-maintained Rust library that extracts -maintainability metrics from source code in many languages. It is built on +`big-code-analysis` is a Rust library that extracts maintainability +metrics from source code in many languages. It is a hard fork of +Mozilla's +[rust-code-analysis](https://github.com/mozilla/rust-code-analysis), +maintained in this repository. It is built on [tree-sitter](https://tree-sitter.github.io/tree-sitter/) and is published on crates.io as a library plus two binaries. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8dd11874..50b65790 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -11,7 +11,7 @@ essentials; the deeper conventions live in under [MPL-2.0](https://www.mozilla.org/MPL/2.0/), matching the rest of the project (declared via `license = "MPL-2.0"` in the root `Cargo.toml`). -- Security-sensitive reports must **not** go in public issues — see +- Security-sensitive reports must **not** go in public issues; see [`SECURITY.md`](SECURITY.md) for the private disclosure channels. ## Getting started @@ -36,24 +36,33 @@ fixture. The two binaries shipped with the workspace are: -- `bca` — the CLI (`big-code-analysis-cli`): +- `bca`, the CLI (`big-code-analysis-cli`): `cargo run -p big-code-analysis-cli --`. -- `bca-web` — the REST API server (`big-code-analysis-web`): +- `bca-web`, the REST API server (`big-code-analysis-web`): `cargo run -p big-code-analysis-web --`. ## Local validation gate `make pre-commit` is the canonical entry point for the full validation -gate. It runs, in one parallel pass: +gate. In one parallel pass it runs, among other stages: - `cargo fmt --check`. - `cargo clippy --workspace --all-targets -- -D warnings` in both default-features and `--all-features` flavours. - `cargo test --workspace --all-features`. +- `cargo doc --no-deps --workspace --all-features` with + `RUSTDOCFLAGS="-D warnings"`. - `cargo +nightly udeps`. -- Markdown / TOML / shell / Makefile lint families - (`rumdl`, `taplo`, `shellcheck`, `shfmt`, `checkmake`). +- Markdown / TOML / shell / Makefile / GitHub Actions lint families + (`rumdl`, `taplo`, `shellcheck`, `shfmt`, `checkmake`, + `actionlint`). +- The man-page drift gate and the `bca` self-scan threshold gates. - `./check-snapshot-anchors.py` (see "Snapshot anchors" below). +- The Python lint / type-check / test stages (see below), skipped + per stage when a tool is absent. + +The authoritative stage list lives in the "Validation gates" section +of [`AGENTS.md`](AGENTS.md). `make ci` runs the same checks without auto-fix, mirroring the GitHub Actions behaviour. @@ -67,7 +76,7 @@ cargo clippy --workspace --all-targets -- -D warnings cargo test --workspace --all-features ``` -If `pre-commit` is installed, also run `pre-commit run --all-files` — +If `pre-commit` is installed, also run `pre-commit run --all-files`; the project's `.pre-commit-config.yaml` wires clippy, `cargo +nightly udeps`, the test suite, `ruff-check` + `ruff-format`, and a local `mypy --strict` hook into the standard hook flow. @@ -79,25 +88,26 @@ plus `pyright` for type checking. `make pre-commit` and `make ci` run these alongside the Rust gates when the tools are present. **Canonical install path: `make py-bootstrap`** (requires -[uv](https://docs.astral.sh/uv/) — `curl -LsSf https://astral.sh/uv/install.sh | sh`, +[uv](https://docs.astral.sh/uv/): `curl -LsSf https://astral.sh/uv/install.sh | sh`, `brew install uv`, or `pipx install uv`). This runs `uv sync --locked --extra dev` against the checked-in `uv.lock`, so the resolved ruff/mypy/pyright/maturin/pytest versions are identical across every contributor on this path. After editing the `dev` extra in `pyproject.toml`, run -`make py-relock` (which runs `uv lock`) to regenerate the lockfile. -`make py-bootstrap` will refuse to silently rewrite `uv.lock` — -intentional, so lockfile churn is always a deliberate, reviewable -commit. Resolve `uv.lock` rebase conflicts by re-running -`make py-relock` rather than hand-merging the file. CI does **not** -yet consume `uv.lock` (workflows pip-install pyproject floors); a -follow-up will move them onto `uv sync --frozen` so the contributor -path and the CI matrix resolve from the same source of truth. +`make py-relock`, which regenerates `uv.lock` **and** the hash-pinned +exports under `big-code-analysis-py/requirements/` (`dev.txt`, +`examples.txt`). `make py-bootstrap` will refuse to silently rewrite +`uv.lock` (intentional, so lockfile churn is always a deliberate, +reviewable commit). Resolve `uv.lock` rebase conflicts by re-running +`make py-relock` rather than hand-merging the file. CI consumes +`uv.lock` through the requirements exports (`pip install +--require-hashes -r …` in the workflows), so a `uv.lock` change and +its regenerated exports must land in the same commit. Alternative install paths (`mise install` via `mise.toml`, direct `pipx install ruff/mypy/pyright/maturin`, `python -m venv .venv && -pip install -e ".[dev]"`) still work but bypass `uv.lock` — resolved +pip install -e ".[dev]"`) still work but bypass `uv.lock`; resolved versions can drift from peers and from CI. They remain documented for contributors with environment constraints that preclude uv. @@ -112,7 +122,7 @@ python -m pytest Or simply `make py-test` from the repo root, which performs the maturin-develop step implicitly. CI runs the same flow across -Linux / macOS / Windows on Python 3.12 and 3.13 (six matrix legs), +Linux / macOS / Windows on every supported Python version, gated by a `dorny/paths-filter` job so Rust-only PRs skip it entirely. @@ -127,7 +137,7 @@ Per-metric tests under `src/metrics/` use [`insta`](https://insta.rs/) snapshot assertions. **Every `insta::assert_json_snapshot!` call must be anchored**: a bare `insta::assert_json_snapshot!(metric.X)` records whatever production -emitted at acceptance time — including bugs. +emitted at acceptance time, including bugs. Each new snapshot assertion must carry one of: @@ -152,11 +162,11 @@ fails on any *increase* against that baseline. Background reading: -- [`AGENTS.md`](AGENTS.md), section "Validation gates" — the full +- [`AGENTS.md`](AGENTS.md), section "Validation gates": the full policy and the bulk-acceptance rules around grammar bumps. - [`docs/development/lessons_learned.md`](docs/development/lessons_learned.md), lesson 2 ("Tree-sitter aliases one rule across many kind_ids") and - lesson 6 ("Snapshot tests pin behaviour, not correctness") — the + lesson 6 ("Snapshot tests pin behaviour, not correctness"): the bug classes the anchor rule exists to catch. For bulk snapshot refresh after a grammar bump or a deliberate @@ -176,8 +186,8 @@ the following have happened in the same parent commit: `tests/repositories/big-code-analysis-output/`). 2. The accepted snapshots are committed and pushed to the submodule's remote (`dekobon/big-code-analysis-output`, `main` branch). -3. The parent records the new submodule SHA — `git add - tests/repositories/big-code-analysis-output` — in the **same +3. The parent records the new submodule SHA (`git add + tests/repositories/big-code-analysis-output`) in the **same parent commit** as the metric/alterator fix. 4. After any rebase, force-push, or long-running batch fix, re-run the integration tests before declaring done. @@ -190,17 +200,17 @@ for why this matters. - **Rust style**: `cargo fmt`, clippy clean with `--workspace --all-targets -- -D warnings`. No `unsafe` code (one - narrow PyO3 FFI exception — see `AGENTS.md`). Avoid + narrow PyO3 FFI exception; see `AGENTS.md`). Avoid `unwrap` / `expect` / `panic!` / `assert!` in non-test code; propagate errors with `?`. - **Visibility**: prefer `pub(crate)` over `pub`; widen visibility only when an item is re-exported from `lib.rs`. -- **Edition**: 2024 — `let-else`, let-chains, and other 2024 features +- **Edition**: 2024; `let-else`, let-chains, and other 2024 features are available. - **Borrowing**: prefer `&str` over `String` parameters unless ownership is required downstream. Never use `to_string_lossy()` on - paths used as identifiers (map keys, JSON output, error correlation) - — use `to_str()` with explicit error handling. + paths used as identifiers (map keys, JSON output, error correlation); + use `to_str()` with explicit error handling. - **Per-language modules mirror each other**: a bug in one `src/languages/language_.rs` typically exists in several. Fix every affected sibling together. @@ -234,13 +244,13 @@ guides under `big-code-analysis-book/src/developers/`: - [Updating tree-sitter grammars](big-code-analysis-book/src/developers/update-grammars.md). External grammar crates are version-pinned (`=0.23.x`, etc.) in the -root `Cargo.toml`. Do not loosen those pins without explicit approval — +root `Cargo.toml`. Do not loosen those pins without explicit approval; a grammar bump is a deliberate, separate change and snapshot tests shift accordingly. ## Code review -Criticism is welcome — point out mistakes, suggest better approaches, +Criticism is welcome: point out mistakes, suggest better approaches, cite relevant standards. Be skeptical and concise. Reviews focus on correctness, API shape, and test coverage before style. diff --git a/README.md b/README.md index 59883d25..6f075589 100644 --- a/README.md +++ b/README.md @@ -10,190 +10,169 @@ [![docs.rs](https://docs.rs/big-code-analysis/badge.svg)](https://docs.rs/big-code-analysis) [![License](https://img.shields.io/crates/l/big-code-analysis.svg)](LICENSE) -**big-code-analysis** is a hard fork of the [rust-code-analysis](https://github.com/mozilla/rust-code-analysis) project. -This project is an unapologetic vibe-coded fork that seeks to add as many features and functions as fast as possible. +**big-code-analysis** measures how maintainable your code is. The `bca` +command line tool computes per-function metrics for +[more than twenty programming languages](https://dekobon.github.io/big-code-analysis/languages.html): +cyclomatic and +[cognitive](https://www.sonarsource.com/docs/CognitiveComplexity.pdf) +complexity, +[Halstead](https://en.wikipedia.org/wiki/Halstead_complexity_measures), +maintainability index, ABC, lines-of-code variants, and the rest of +[the metric suite](https://dekobon.github.io/big-code-analysis/metrics.html). It parses with +[tree-sitter](https://tree-sitter.github.io/tree-sitter/), so it needs +no compiler, build step, or language runtime: point it at a directory +and it prints numbers. + +The project is a hard fork of Mozilla's +[rust-code-analysis](https://github.com/mozilla/rust-code-analysis) +that grows the metric engine into a code-quality toolchain: + +- `bca check`: a threshold gate with baselines, in-source suppression + markers, and CI-friendly exit codes. +- Agent feedback: violations piped back into + [Claude Code](https://code.claude.com/docs/en/overview) or + [opencode](https://opencode.ai/) after every edit + ([below](#feed-metrics-to-your-coding-agent)). +- `bca report`: Markdown and HTML hotspot reports. +- `bca vcs`: change-history metrics over a git tree (churn, ownership + dilution, bug-fix history). +- Library bindings: the same engine as a Rust crate, a + [Python package](https://pypi.org/project/big-code-analysis/), and a + REST server (`bca-web`). + +The full documentation lives in +[**the book**](https://dekobon.github.io/big-code-analysis/): metrics +definitions, command reference, CI recipes, and library guides. + +## Feed metrics to your coding agent + +Coding agents write a lot of code, and nothing in their loop tells +them a function has become too complex to maintain. `bca check` closes +that loop: it checks each file the agent edits and reports the +offending functions back into the model's context the moment the edit +lands. All it needs is `bca` on `PATH` (see +[Quick start](#quick-start)) plus a few lines of config. + +- **Claude Code**: a `PostToolUse` hook runs `bca check` on the edited + file and feeds violations back through stderr. This repository + dogfoods a reference hook at + [`.claude/hooks/bca-check.sh`](.claude/hooks/bca-check.sh). +- **opencode**: a `tool.execute.after` plugin does the same; the + reference copy is at + [`.opencode/plugins/bca-check.js`](.opencode/plugins/bca-check.js). + +The +[agent feedback recipe](https://dekobon.github.io/big-code-analysis/recipes/agent-feedback.html) +has copy-pasteable wiring for both tools, plus the guidance block that +keeps an agent from gaming the metric instead of simplifying the code. + +## Quick start + +Install a prebuilt `bca` from the +[releases page](https://github.com/dekobon/big-code-analysis/releases) +(signed tarballs for Linux, macOS, and Windows, plus `.deb`, `.rpm`, +and `.apk` packages), or install it from a package registry: -Nonetheless, it is still a Rust library to analyze and extract information -from source code written in many different programming languages. -It is based on a parser generator tool and an incremental parsing library -called -Tree Sitter. - -A command line tool called **bca** is provided to interact with the API of the library in an easy way. +```console +cargo install big-code-analysis-cli # or: pip install big-code-analysis-cli +``` -This tool can be used to: +Then, from a project root: -- Call **big-code-analysis** API -- Print nodes and metrics information -- Export metrics in different formats -- Generate a Markdown or HTML quality-metrics report (`bca report markdown` / `bca report html`) +```console +bca metrics src/main.rs # per-function metric tree for one file +bca init # scaffold bca.toml, .bcaignore, .bca-baseline.toml +bca check # exit 2 when a function crosses a threshold +bca report -O html -o report.html +``` -In addition, we provide a **bca-web** tool to use the library through a REST API. +The [Commands](https://dekobon.github.io/big-code-analysis/commands/index.html) +chapter of the book documents every subcommand, flag, and output +format. -## Live example reports +## Quality gates and reports in CI -`bca` runs against its own source on every push to `main` and publishes -the result alongside the documentation: +`bca check` reads thresholds, baselines, and excludes from a committed +`bca.toml`, so CI, local runs, and agent hooks all gate on the same +signal. `bca report` turns the same run into a Markdown comment for a +pull request or an HTML hotspot page. This repository gates itself on +every push and publishes the result: - HTML hotspot report: - Markdown PR/MR comment: -The wiring lives in -[`.github/workflows/pages.yml`](.github/workflows/pages.yml). For -downstream projects, the -[CI integration recipe](https://dekobon.github.io/big-code-analysis/recipes/ci.html) -is the canonical adoption guide — it documents the recommended -pinned-release install path (with `BCA_VERSION` + sha256 pin) plus -a `cargo install` alternative. The in-tree `pages.yml` workflow -builds `bca` from the current checkout because main may carry CLI -artifact schemas that no released `bca` supports yet — see the -schema-compatibility note in the recipe before copying that pattern. - -## Usage - -**big-code-analysis** supports many types of programming languages and -computes a great variety of metrics. You can find up to date documentation at -Documentation. - -On the - - Commands - page, there is a list of commands that can be run to get information -about metrics, nodes, and other general data provided by this software. - -## Using as a library - -`big-code-analysis` is published on crates.io and can be embedded -directly. The crate is on the `1.x` line and ships under a written -stability contract: the public API surface is held stable across -patch and minor bumps, and breaking shape changes are reserved for -the next major bump. Metric *values* may still drift across minor -bumps when a grammar pin moves or a metric definition is fixed — -see [STABILITY.md](./STABILITY.md) for the full versioning contract, -MSRV policy, escape hatches, and exactly what we do and do not -promise within `1.x`. - -For task-oriented walkthroughs — quick start, in-memory analysis, -walking `FuncSpace` results, and error handling — see the -[**Using as a Library**](https://dekobon.github.io/big-code-analysis/library/index.html) -section of the book. - -Python bindings (PyO3) live in -[`big-code-analysis-py/`](./big-code-analysis-py/README.md) and ship -the same metric pipeline as a Python package. See the book's -[**Python Bindings**](https://dekobon.github.io/big-code-analysis/python/index.html) -section for the install matrix, batch / async / SARIF recipes, and -the full error taxonomy. +The [CI integration recipe](https://dekobon.github.io/big-code-analysis/recipes/ci.html) +is the adoption guide: a pinned-release install with checksum +verification, ready-made GitHub Actions and GitLab CI jobs, and the +[baselines](https://dekobon.github.io/big-code-analysis/recipes/baselines.html) +and +[local threshold gates](https://dekobon.github.io/big-code-analysis/recipes/local-gates.html) +recipes for ratcheting an existing codebase. -### Per-language Cargo features +## Use it as a library -Every tree-sitter grammar is gated behind a per-language Cargo -feature. The default feature set is `all-languages`, so a bare +The `big-code-analysis` crate is published on crates.io under a +written stability contract ([STABILITY.md](./STABILITY.md)): the +public API holds stable across patch and minor bumps within `2.x`, +and breaking changes wait for the next major. Metric *values* may +still drift across minor bumps when a grammar pin moves or a metric +definition is fixed; the contract spells out exactly what is and is +not promised. ```toml -big-code-analysis = "2.0.1" +[dependencies] +big-code-analysis = "2" ``` -pulls every grammar in (matching the library's historical behaviour -and what the `bca` / `bca-web` binaries ship). Library consumers that -only need a subset of languages can opt out of the defaults and -re-enable just the grammars they want: - -```toml -big-code-analysis = { version = "2.0.1", default-features = false, features = ["rust", "typescript"] } -``` - -Supported language features: `bash`, `c`, `cpp`, `csharp`, `elixir`, -`go`, `groovy`, `irules`, `java`, `javascript`, `kotlin`, `lua`, -`mozcpp`, `mozjs`, `perl`, `php`, `python`, `ruby`, `rust`, `tcl`, -`typescript`. The `irules` feature adds F5 iRules (a Tcl -dialect; extensions `.irule` / `.irules`). The -`cpp` feature backs the `Cpp` LANG variant with the upstream -community `tree-sitter-cpp` grammar and, with the `Ccomment` and -`Preproc` C-family helper variants, pulls in `tree-sitter-cpp`, -`bca-tree-sitter-ccomment`, and `bca-tree-sitter-preproc` together. -The `c` feature (added in #721) backs the dedicated `C` LANG variant -with upstream `tree-sitter-c` and owns `.c`; it shares the same -`ccomment` / `preproc` C-family helpers. `.h` stays on `Cpp`. -The opt-in `mozcpp` feature adds the `Mozcpp` LANG variant — the -Mozilla/Gecko C++ dialect (vendored `bca-tree-sitter-mozcpp` fork) — -which owns no file extensions and is selected only by name -(`--language mozcpp`, manifest, or API), mirroring `mozjs` for -`.jsm`. Since #720 `cpp` no longer pulls `bca-tree-sitter-mozcpp` -(a breaking dep-set change for `--no-default-features` consumers). - -The `LANG` enum keeps every variant defined regardless of the active -feature set; selecting a [`LANG`] variant whose feature is off -returns `Err(MetricsError::LanguageDisabled(LANG))` from every -dispatch entry point (`analyze`, `metrics_from_tree`, `action`, -`get_ops`, the deprecated `get_function_spaces*` shims, and -`LANG::tree_sitter_language`). The set of compiled-in variants -is queryable via `LANG::is_enabled`. - -## Building - -The repository ships a `Makefile` that wraps every common build, test, -lint, and docs task. Run `make help` for the full list, and -`make check-tools` to verify the optional tools are installed. - -```console -make build # debug build of the entire workspace -make build-release # optimised release build -``` - -If you prefer to run cargo directly, or want to build a single crate: - -```console -cargo build # library only -cargo build -p big-code-analysis-cli # CLI only -cargo build -p big-code-analysis-web # web server only -cargo build --workspace # everything in one shot -``` - -## Testing - -```console -make test # cargo test --workspace --all-features --lib --bins --tests -make test-doc # cargo test --workspace --all-features --doc -make pre-commit # full local gate: fmt-check, clippy, tests, udeps, lint families -``` - -`make pre-commit` is the recommended gate before committing — it is -equivalent to what CI runs. If GNU Make 4 or any of the optional -tools are unavailable, the raw cargo invocation still works: +Every grammar sits behind a per-language Cargo feature; the default is +all of them, and consumers who need a subset can disable default +features and re-enable individual languages. See +[Per-language Cargo features](https://dekobon.github.io/big-code-analysis/library/cargo-features.html) +in the book, and the +[Using as a Library](https://dekobon.github.io/big-code-analysis/library/index.html) +chapter for task-oriented walkthroughs (quick start, in-memory +analysis, walking `FuncSpace` results, error handling). The API +reference is on [docs.rs](https://docs.rs/big-code-analysis). + +Python bindings ([PyO3](https://pyo3.rs/)) live in +[`big-code-analysis-py/`](./big-code-analysis-py/README.md) and ship +the same metric pipeline as the +[`big-code-analysis` package on PyPI](https://pypi.org/project/big-code-analysis/). +The book's +[Python Bindings](https://dekobon.github.io/big-code-analysis/python/index.html) +chapter covers installation, batch and async processing, and +[SARIF](https://docs.oasis-open.org/sarif/sarif/v2.1.0/sarif-v2.1.0.html) +output. -```console -cargo test --workspace --all-features --verbose -``` +For a service, `bca-web` wraps the library in a REST API; see +[Operating bca-web](https://dekobon.github.io/big-code-analysis/commands/web-server.html). -### Updating insta tests +## Building and contributing -We use [insta](https://insta.rs), to update the snapshot tests you should install [cargo insta](https://crates.io/crates/cargo-insta) +The repository is a Cargo workspace with a `Makefile` wrapper for +common tasks. Run `make help` for the full list. ```console -make insta-review # cargo insta test --review +make build # debug build of the entire workspace +make test # full test suite (workspace, all features) +make pre-commit # full local gate, mirrors CI ``` -Will run the tests, generate the new snapshot references and let you review them. - -### Updating grammars - -Have a look at -Update grammars guide -to learn how to update languages grammars. - -## Contributing - -If you want to contribute to the development of this software, have a look at the -guidelines contained in our -Developers Guide. +[CONTRIBUTING.md](./CONTRIBUTING.md) covers the contribution workflow, +and the +[Developers Guide](https://dekobon.github.io/big-code-analysis/developers/index.html) +in the book covers internals: adding a language, implementing a +metric, and updating grammars. ## Licenses -- Mozilla-defined grammars are released under the MIT license. +- The vendored grammar crates (`tree-sitter-ccomment`, + `tree-sitter-mozcpp`, `tree-sitter-mozjs`, `tree-sitter-preproc`, + `tree-sitter-tcl`) are released under the MIT license. -- **big-code-analysis**, **big-code-analysis-cli** and **big-code-analysis-web** -are released under the -Mozilla Public License v2.0. +- **big-code-analysis**, **big-code-analysis-cli**, + **big-code-analysis-web**, and **big-code-analysis-py** are released + under the + [Mozilla Public License v2.0](https://www.mozilla.org/MPL/2.0/). diff --git a/RELEASING.md b/RELEASING.md index 665ea7f8..780677d1 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -4,17 +4,8 @@ This document is the step-by-step procedure for releasing `big-code-analysis`. It describes what to do, in what order, and what to check when something looks wrong. -> **Status.** The release pipeline described here is being built up in -> stages (`S1`–`S8` of the public-release roadmap). The repository -> currently ships with a Cargo workspace, the MSRV declaration, the -> CHANGELOG, and the contributor docs. The signed-artefact pipeline, -> minisign key, packaging matrix, and external taps/buckets land in -> the remaining stages. Sections below that describe in-flight pieces -> say so explicitly. - -The pipeline, once landed, is defined in -`.github/workflows/release.yml`. Everything downstream of `git push ---tags` is automated. +The pipeline is defined in `.github/workflows/release.yml`. +Everything downstream of `git push --tags` is automated. ## MSRV (Minimum Supported Rust Version) @@ -48,36 +39,36 @@ lands). Note the bump in the CHANGELOG under `### Changed`. One push of a `v*` tag will run this end-to-end: -1. **preflight** — validates the tag, checks `Cargo.toml` version +1. **preflight**: validates the tag, checks `Cargo.toml` version parity against `[workspace.package] version`, confirms `minisign.pub` is not a placeholder, and extracts the matching `CHANGELOG.md` section as release notes. -2. **build** — cross-compiles `bca` and `bca-web` for the target +2. **build**: cross-compiles `bca` and `bca-web` for the target matrix: Linux gnu/musl × x86_64/aarch64, macOS aarch64, Windows x86_64/aarch64. `x86_64-unknown-freebsd` is tracked separately (see [#346](https://github.com/dekobon/big-code-analysis/issues/346) - under [Known pipeline issues](#known-pipeline-issues)). Strips + under [Known pipeline limitations](#known-pipeline-limitations)). Strips binaries, captures debug symbols, and produces per-target `.tar.gz` / `.zip` archives. -3. **package-*** — builds `.deb`, `.rpm`, `.apk`, and any other OS +3. **package-***: builds `.deb`, `.rpm`, `.apk`, and any other OS packages from the staged binaries. -4. **smoke-*** — installs each package inside the appropriate +4. **smoke-***: installs each package inside the appropriate container/VM and asserts `bca --version` and `bca-web --version` match the tag. -5. **sign-attest** — flattens every artefact into `release/`, +5. **sign-attest**: flattens every artefact into `release/`, generates CycloneDX SBOMs, computes `SHA256SUMS`, signs it with minisign, and attaches SLSA build provenance. -6. **publish** — creates/updates the GitHub Release, attaches every +6. **publish**: creates/updates the GitHub Release, attaches every artefact + `SHA256SUMS` + `SHA256SUMS.minisig`, and (for non pre-releases, **subject to the gating variables below**) pushes the Homebrew formula and Scoop manifest. -7. **publish-crates** — for non pre-releases, **subject to the gating +7. **publish-crates**: for non pre-releases, **subject to the gating variables below**, runs `cargo publish` for each publishable workspace crate in dependency order: the five `bca-tree-sitter-*` grammar leaves first, then `big-code-analysis` (library), then `big-code-analysis-cli` and `big-code-analysis-web`. Skips idempotently if the version is already on crates.io. -8. **verify** — downloads the published musl tarball back out of the +8. **verify**: downloads the published musl tarball back out of the release, verifies the minisign signature, checksum, and SLSA provenance. @@ -91,15 +82,15 @@ workflows** that are *not* part of `release.yml` and run fully in parallel with it (a failure in one does not block the others): - **`python-wheels.yml`** publishes the importable library bindings - (`big-code-analysis`) — an abi3 extension wheel. See + (`big-code-analysis`), an abi3 extension wheel. See [Python wheels (PyPI)](#python-wheels-pypi). - **`python-cli-wheels.yml`** publishes the `bca` command-line tool - (`big-code-analysis-cli`) — a `-b bin` wheel that drops `bca` onto + (`big-code-analysis-cli`), a `-b bin` wheel that drops `bca` onto `PATH`. See [CLI wheels (PyPI)](#cli-wheels-pypi). Both read the workspace version (`dynamic = ["version"]`), so they -publish in lockstep with the crates above on every bump — no separate -version step. Their one-time Trusted-Publisher setup is in the +publish in lockstep with the crates above on every bump (no separate +version step). Their one-time Trusted-Publisher setup is in the [Post-public-release checklist](#post-public-release-checklist); after that they fire automatically on each tag. @@ -160,7 +151,7 @@ Each leaf manifest sets `[lib] name = "tree_sitter_"` so the *published* package name is `bca-tree-sitter-`. The workspace alias in the root `Cargo.toml` (and `enums/Cargo.toml`) uses Cargo's `package = ...` aliasing so every consumer site reads -`tree-sitter- = { workspace = true }` as before — call sites +`tree-sitter- = { workspace = true }` as before; call sites under `src/`, `enums/`, and feature flags did not change. **Publish order is leaf-first.** The `publish-crates` job in @@ -178,8 +169,8 @@ handles this automatically: it queries the sparse index for runs the parent dry-run if that leaf is already published. On the first tag with `ENABLE_CRATES_PUBLISH=true`, the parent dry-run is skipped with a `::notice::` and the `publish-crates` job uploads the -five leaves *first*, then `big-code-analysis`, then the binaries — -in one workflow run, no manual intervention. From the second tag +five leaves *first*, then `big-code-analysis`, then the binaries (in +one workflow run, no manual intervention). From the second tag onwards the parent dry-run becomes a hard gate. `make release-check VERSION=…` mirrors the same logic: it @@ -187,13 +178,13 @@ unconditionally dry-runs the five leaves, then wraps the parent dry-run in a warning that points back to this section if the bootstrap state is detected. -**Lockstep version policy.** Every crate in this repository — the +**Lockstep version policy.** Every crate in this repository (the library, the CLI, the web crate, the Python crate, the `enums` / `xtask` helpers, and the five `bca-tree-sitter-*` vendored grammar -leaves — shares one version number. There is no per-crate version +leaves) shares one version number. There is no per-crate version drift. A version bump touches: -1. `[workspace.package] version` in the root `Cargo.toml` — this +1. `[workspace.package] version` in the root `Cargo.toml`; this covers every workspace member that declares `version.workspace = true`. 2. `[package] version` in `enums/Cargo.toml` (excluded from the @@ -206,12 +197,15 @@ drift. A version bump touches: 5. The `version = "="` pin on the `big-code-analysis` path-dep in `big-code-analysis-cli/Cargo.toml` and `big-code-analysis-web/Cargo.toml`. -6. The hard-coded version references in user-facing docs +6. Only when the bump is the release-prep commit for the version + being cut: the hard-coded version references in user-facing docs (`README.md`, `STABILITY.md`, the book's `quick-start.md` and `cargo-features.md`) **and** the install snippet in every leaf's `bindings/rust/README.md` (5 files), since those ship inside the published `bca-tree-sitter-*` tarballs and render as - the crates.io landing page. + the crates.io landing page. A post-tag bump to the next + development version leaves them at the published release; see + [Version strings in documentation](#version-strings-in-documentation). 7. The man pages (re-run `cargo run -p xtask`). 8. The SARIF tool-version snapshots (re-run `cargo insta test` and accept). @@ -223,7 +217,7 @@ the `lint` job in `.github/workflows/ci.yml`) after editing to catch any item the human eye missed. A grammar refresh (`recreate-grammars.sh` regenerates the parsers) -is a normal change *under* the current version — bumping the +is a normal change *under* the current version: bumping the grammars does not bump the version on its own. The next workspace release picks up the regenerated grammars at whatever leaf version already matches the workspace version. @@ -255,19 +249,19 @@ respectively. Both are minted at as fine-grained PATs with **Repository access: Only select repositories** (scoped to the single tap or bucket repo) and **Repository permissions -→ Contents: Read and write** — leave every other permission at *No +→ Contents: Read and write**; leave every other permission at *No access*. Store each token under Settings → Secrets and variables → Actions → Secrets on `dekobon/big-code-analysis`. crates.io authentication uses -[Trusted Publishing](https://crates.io/docs/trusted-publishing) — no +[Trusted Publishing](https://crates.io/docs/trusted-publishing): no long-lived `CARGO_REGISTRY_TOKEN` is stored as a secret. The `publish-crates` job mints a GitHub OIDC ID token and exchanges it for a short-lived registry token scoped to that run. -If `HOMEBREW_TAP_TOKEN` or `SCOOP_BUCKET_TOKEN` is missing — or if the +If `HOMEBREW_TAP_TOKEN` or `SCOOP_BUCKET_TOKEN` is missing, or if the target tap/bucket repo is unreachable (deleted, renamed, or the PAT -cannot see it) — the corresponding step emits a GitHub Actions +cannot see it), the corresponding step emits a GitHub Actions warning and skips without failing the release. ### Minisign key @@ -283,7 +277,7 @@ minisign -G -p minisign.pub -s minisign.key ``` Commit `minisign.pub`. Store `minisign.key` as the -`MINISIGN_SECRET_KEY` repo secret via stdin redirection — **do not +`MINISIGN_SECRET_KEY` repo secret via stdin redirection; **do not paste the contents into the web UI**: ```bash @@ -296,9 +290,9 @@ gh secret set MINISIGN_PASSWORD -R dekobon/big-code-analysis A minisign secret key file is two lines and ends with `\n`. Paste-via- UI silently strips the trailing newline (and can introduce other whitespace artefacts) so that `minisign -S` later fails with `Error -while loading the secret key file` — masquerading as a wrong-key / +while loading the secret key file`, masquerading as a wrong-key / wrong-password failure when the bytes are actually one newline short. -Stdin redirection from the file preserves the exact file bytes — +Stdin redirection from the file preserves the exact file bytes, including the trailing newline that the web UI eats. Keep `minisign.key` itself out of the repo. @@ -306,13 +300,13 @@ including the trailing newline that the web UI eats. Keep Stable releases push to (subject to the gating variables above): -- `dekobon/homebrew-tap` — shared Homebrew tap; the release workflow +- `dekobon/homebrew-tap`: shared Homebrew tap; the release workflow commits only `Formula/big-code-analysis.rb` and leaves the other formulae in the tap untouched. -- `dekobon/scoop-bucket` — shared Scoop bucket; the release workflow +- `dekobon/scoop-bucket`: shared Scoop bucket; the release workflow commits only `bucket/big-code-analysis.json` and leaves the other manifests in the bucket untouched. -- crates.io — leaf-first: the five `bca-tree-sitter-*` grammar +- crates.io, leaf-first: the five `bca-tree-sitter-*` grammar crates, then `big-code-analysis` (library), then `big-code-analysis-cli` and `big-code-analysis-web`. See [crates.io ownership](#cratesio-ownership) for the publish loop @@ -323,7 +317,7 @@ Both tap and bucket repos must exist and accept the configured PAT. ### crates.io ownership Before the first automated publish you must manually claim **all eight -crate names** — the five `bca-tree-sitter-*` leaves plus the three +crate names**: the five `bca-tree-sitter-*` leaves plus the three top-level crates. The `publish-crates` job in `release.yml` uses Trusted Publishing which requires the crate to exist before TP can be registered, so the very first publish has to be a hand-rolled @@ -340,16 +334,16 @@ registered, so the very first publish has to be a hand-rolled If any name is taken by someone else, pick a different name and update the matching `[package].name` (and the workspace alias for - leaves) before tagging — `cargo owner --add` only works on crates + leaves) before tagging; `cargo owner --add` only works on crates you already own. 2. **Verify the parent's `include` whitelist is present.** The `[package].include = […]` block in the root `Cargo.toml` restricts the published `.crate` to `src/**`, `Cargo.toml`, `README.md`, `LICENSE`, and `CHANGELOG.md`. Without it, - `cargo publish` packages the entire repo — notably + `cargo publish` packages the entire repo, notably `tests/repositories/` (~130 MiB compressed of snapshot - fixtures) — and the upload fails against crates.io's size + fixtures), and the upload fails against crates.io's size limit with a Varnish `503 backend write error` rather than a useful error message. Verify before the first publish: @@ -412,7 +406,7 @@ one-time setup steps are required on top of the The `publish-crates` job references this environment and the crates.io trusted publisher matches the `environment` OIDC claim against it. Optional protection rules (required reviewers, - deployment branch filters) act as a manual gate on every publish — + deployment branch filters) act as a manual gate on every publish; the environment is the right place to add them, not the workflow. The name must match the TP registration exactly; a typo here is the most common self-inflicted failure mode. @@ -428,7 +422,7 @@ one-time setup steps are required on top of the - Workflow filename: `release.yml` (basename only, not a path). - Environment: `release`. - Every publishable crate needs its own trusted-publisher entry — a + Every publishable crate needs its own trusted-publisher entry: a TP registered on `big-code-analysis` does not cover the CLI, the web crate, or any of the leaves. The workflow still performs a single `auth` exchange for all publishes because crates.io @@ -453,7 +447,7 @@ tagging. Member crates inherit their version from `[workspace.package]`, so edit these in lockstep: -1. Root `Cargo.toml`, `[workspace.package] version = "x.y.z"` — the +1. Root `Cargo.toml`, `[workspace.package] version = "x.y.z"`: the canonical version that every member crate picks up via `version.workspace = true`. 2. Any `[workspace.dependencies]` entries that pin an internal crate @@ -461,7 +455,7 @@ edit these in lockstep: }`). Must match the workspace version, otherwise `cargo publish` on the dependent crate will reject the dependency. 3. The `enums/` helper crate (excluded from the root workspace). - Its own `[package] version` carries the same value — bump it + Its own `[package] version` carries the same value; bump it alongside the workspace bump, never on its own. 4. Each `tree-sitter-/Cargo.toml` (also excluded). Same discipline as `enums/`: bump in lockstep with the workspace. @@ -491,16 +485,16 @@ cargo xtask `man/*.1` embeds both the binary version (`big-code-analysis x.y.z` in the `.TH` line and `vX.Y.Z` in `.SH VERSION`) and the live clap -schema, so any version bump — workspace-wide or CLI-only (e.g. the -`big-code-analysis-cli` version override at #235) — leaves the +schema, so any version bump, workspace-wide or CLI-only (e.g. the +`big-code-analysis-cli` version override at #235), leaves the committed pages stale. The per-PR `man pages up to date` CI job gates against drift; `release.yml` regenerates the pages again per build leg so the shipped artefacts cannot ship with a stale schema, but committing the regenerated pages keeps the gate green between release-prep and tag push. Same rule applies any time a CLI flag is -added or renamed — not just at release time. +added or renamed, not just at release time. -Pick the version using semver. The workspace is on the `1.x` line +Pick the version using semver. The workspace is on the `2.x` line and ships under the [STABILITY.md][stability] contract: the public Rust API surface (`big-code-analysis` library re-exports, the `bca` CLI argument grammar, and the `bca-web` REST schema) is held stable @@ -522,13 +516,132 @@ so the release-prep commit is a single, self-contained change: chore(release): prepare v1.2.0 ``` +## Version strings in documentation + +The docs carry two kinds of version string, and they move at +different times. Reader-facing pins must always reference the +**latest published release** (what a reader can download today), +while the workspace manifests track the **next** version being +developed. Between a tag and the next release the two deliberately +diverge: after the post-tag bump, `Cargo.toml` reads ahead of every +documentation example, and that is correct. When in doubt, apply the +reader test: if someone copies this line today, does it resolve? + +`./check-versions.py` (`make check-versions`, wired into +`make pre-commit` and the CI lint job) enforces the two automatable +categories: dependency-example pins must equal the latest released +`## [X.Y.Z]` section of `CHANGELOG.md`, and the `recipes/ci.md` +install pins may cite that release or the one immediately before it +(they move in the post-publish follow-up). Because release-prep +creates the new changelog section, the gate forces the release-prep +doc bump and rejects a premature one in the same stroke. + +### Bump in the release-prep commit (before tagging vX.Y.Z) + +These strings must read `X.Y.Z` in the tagged commit. They reference +crates.io / PyPI versions, which resolve as soon as `publish-crates` +and the wheel workflows finish, so setting them at release-prep is +safe: + +| File | String | +| --- | --- | +| `STABILITY.md` | `(currently \`X.Y.Z\`)` in the opening paragraph | +| `STABILITY.md` | both exact-pin examples `big-code-analysis = "= X.Y.Z"` | +| `big-code-analysis-book/src/library/stability.md` | the same two shapes as `STABILITY.md` | +| `big-code-analysis-book/src/library/quick-start.md` | the `[dependencies]` example | +| `big-code-analysis-book/src/library/cargo-features.md` | both `[dependencies]` examples | +| `CHANGELOG.md` | `## [Unreleased]` entries move into `## [X.Y.Z]` (see the checklist) | + +Optionally refresh the sample REST responses in +`big-code-analysis-book/src/commands/rest.md` (`"server"`, +`"library"`, `"version"` fields) so captured output matches the +release; they are illustrative, not pins, so this is non-blocking. + +`man/*.1` also embeds the version but is generated: run `cargo xtask` +(already part of the release-prep steps above); never hand-edit. + +Find stragglers with a scoped sweep before tagging. Never run a +blind repo-wide `sed`: the SemVer / Keep-a-Changelog spec URLs, +historical changelog sections, and illustrative examples match +version-shaped patterns and must not be rewritten. Review each hit +against the categories on this page: + +```bash +PREV=2.0.0 # the previous published release +rg -n "$PREV" README.md STABILITY.md docs/ big-code-analysis-py/README.md \ + big-code-analysis-book/src/ --glob '!**/CHANGELOG.md' +``` + +### Update only after the release is actually published + +Every pin in `big-code-analysis-book/src/recipes/ci.md` references a +published GitHub Release or crates.io artifact. They all move in a +single follow-up commit to `main` once the pipeline finishes: + +- `BCA_VERSION` (the GitHub Actions `env:` block and the GitLab + `variables:` block) and their paired `BCA_SHA256` values. The + sha256 is the digest of + `big-code-analysis-X.Y.Z-${BCA_TARGET}.tar.gz` from the release's + `SHA256SUMS` asset; it cannot exist before the `sign-attest` job + runs, which is why this category exists at all. Bump the version + and the sha together, never one without the other. +- The `taiki-e/install-action` pins + (`tool: big-code-analysis-cli@X.Y.Z`, two sites), the + `cargo binstall ... --version X.Y.Z` line, and the cargo cache key + `bca-${{ runner.os }}-X.Y.Z`. These carry no checksum and could + technically move at release-prep, but they mean "a published + release", so they move with their sha-pinned siblings in the one + post-release sweep. + +Fetch the new checksum straight from the release: + +```bash +VERSION=X.Y.Z +gh release download "v${VERSION}" -R dekobon/big-code-analysis \ + -p SHA256SUMS -O - | grep 'x86_64-unknown-linux-gnu\.tar\.gz$' +``` + +This follow-up commit is part of cutting the release, not of the +next development cycle; land it before (or together with) the +post-tag version bump. + +### Leave alone when bumping main to the next version + +The post-tag bump to the next version changes the workspace +manifests (the `Cargo.toml` sites listed under +[Bumping the version](#bumping-the-version), plus `Cargo.lock`) and +the regenerated `man/*.1`, and **nothing else**. In particular do +not touch: + +- Any pin in `recipes/ci.md` (the previous category): readers deploy + the published release, not `main`. +- The dependency examples and `(currently ...)` strings from the + release-prep category: they keep reading the published release + until the next release-prep commit moves them. + +### Never change (outside a major bump) + +- Major-line strings: `2.x` in `STABILITY.md`, `AGENTS.md`, + `README.md`, `docs/conventions/documentation.md`, the book's + stability page, and this file, plus the README dependency example + `big-code-analysis = "2"`. These move only when `3.0` ships. +- Historical records: released `CHANGELOG.md` sections, prose + describing past releases ("shipped in `2.0.0`", "removed in 2.0"), + and `docs/development/lessons_learned.md`. +- Illustrative placeholders: the `v1.2.0` / `v0.1.0` example + versions in this file's command snippets. +- External spec versions that merely look like ours: the + [SemVer 2.0.0](https://semver.org/spec/v2.0.0.html) and + [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/) + URLs. + ## Pre-release checklist Before tagging, on `main`: - [ ] All intended changes are merged and CI is green. - [ ] Workspace version is bumped per - [Bumping the version](#bumping-the-version) — all + [Bumping the version](#bumping-the-version): all `Cargo.toml` sites, plus a refreshed `Cargo.lock`. - [ ] `cargo xtask` has been run and the resulting `man/*.1` edits are committed in the release-prep commit. `git diff man/` @@ -537,12 +650,16 @@ Before tagging, on `main`: notes. The header must match the tag exactly, minus the leading `v`. Move entries out of `## [Unreleased]` into the new section. +- [ ] Reader-facing documentation pins read the new version per the + release-prep category of + [Version strings in documentation](#version-strings-in-documentation), + and the scoped `rg` sweep there shows no stragglers. - [ ] `cargo test --workspace --all-features` passes locally - (including integration snapshots — initialize submodules first). + (including integration snapshots; initialize submodules first). - [ ] `minisign.pub` is a real key (run - `grep '^untrusted comment: placeholder' minisign.pub` — it + `grep '^untrusted comment: placeholder' minisign.pub`; it should print nothing). -- [ ] Parent crate packages to a sane size — `cargo package -p +- [ ] Parent crate packages to a sane size: `cargo package -p big-code-analysis --allow-dirty --no-verify` followed by `ls -lh target/package/big-code-analysis-*.crate` should show well under 10 MiB (the crates.io upload ceiling). If it @@ -567,7 +684,7 @@ git tag -a v1.2.0 -m "v1.2.0" git push origin v1.2.0 ``` -That's it — the push of the tag triggers `release.yml` **and** the two +That's it: the push of the tag triggers `release.yml` **and** the two PyPI wheel workflows (`python-wheels.yml` for the library bindings, `python-cli-wheels.yml` for the `bca` CLI), all in parallel. Watch all three in the Actions tab: @@ -589,7 +706,7 @@ in [Post-release verification](#post-release-verification). ## Cutting a pre-release Pre-release tags match `vX.Y.Z-` where `` is -`[A-Za-z][0-9]*` — e.g. `v1.2.0-rc1`, `v1.2.0-beta2`, +`[A-Za-z][0-9]*`, e.g. `v1.2.0-rc1`, `v1.2.0-beta2`, `v1.2.0-alpha3`. **Do not use dotted forms like `v1.2.0-rc.1`**: Alpine's abuild grammar rejects dots in the pre-release suffix. @@ -609,11 +726,11 @@ pushes. Both PyPI wheel workflows still **build and smoke-test** every wheel on a pre-release tag but **skip the PyPI publish step**, so a pre-release never lands a wheel on PyPI. The CLI wheel (`python-cli-wheels.yml`) -skips publish for *any* hyphenated suffix — `!contains(github.ref, -'-')`, matching `release.yml`'s `*-*` prerelease rule exactly; the +skips publish for *any* hyphenated suffix (`!contains(github.ref, +'-')`, matching `release.yml`'s `*-*` prerelease rule exactly); the library wheel (`python-wheels.yml`) skips the recognised `-rc` / `-beta` / `-alpha` suffixes. For the suffixes this project actually -uses (above), all three pipelines stay aligned — one tag cannot publish +uses (above), all three pipelines stay aligned: one tag cannot publish a prerelease to one registry while skipping another. ## Post-release verification @@ -651,8 +768,8 @@ once the corresponding gating variable is on): Confirm both PyPI wheels published at the new version (these ship on every tag once their Trusted Publishers are registered). Either check -the project pages — and - — or verify the CLI +the project pages ( and +) or verify the CLI end-to-end from a clean environment: ```bash @@ -667,6 +784,13 @@ bca --version # must print the tagged version deactivate ``` +Finally, land the documentation follow-up commit: update every pin +in `big-code-analysis-book/src/recipes/ci.md` (the post-publish +category of +[Version strings in documentation](#version-strings-in-documentation)) +to the new version, with the `BCA_SHA256` values taken from the +release's `SHA256SUMS` asset. + ## Post-public-release checklist The first time the repository goes public and a stable release is @@ -679,7 +803,7 @@ turns into a foot-gun on the *next* release. leaves, `big-code-analysis`, `big-code-analysis-cli`, `big-code-analysis-web`): claim the name with a manual `cargo publish` (leaf-first, retry on the new-crate rate - limit — see [crates.io ownership](#cratesio-ownership) for + limit; see [crates.io ownership](#cratesio-ownership) for the loop), add at least one co-owner via `cargo owner --add`, and register a Trusted Publisher (repo owner `dekobon`, repo `big-code-analysis`, workflow `release.yml`, @@ -739,12 +863,12 @@ turns into a foot-gun on the *next* release. Python bindings ship via `.github/workflows/python-wheels.yml`, not `release.yml`. The two workflows trigger on the same `v*` tag push -but run in parallel — a crates.io publish failure does not block the +but run in parallel; a crates.io publish failure does not block the PyPI upload, and vice versa. What the python-wheels pipeline does: -1. **build** — `PyO3/maturin-action@v1.51.0` builds a manylinux_2_28 +1. **build**: `PyO3/maturin-action@v1.51.0` builds a manylinux_2_28 abi3 wheel on `ubuntu-latest` (x86_64) and `ubuntu-24.04-arm` (aarch64). `[tool.maturin].features` in `big-code-analysis-py/pyproject.toml` pins @@ -752,10 +876,10 @@ What the python-wheels pipeline does: the limited (stable) Python C API and targets CPython 3.12+ forward-compatibly. One wheel per architecture covers every future 3.12+ minor release. -2. **sdist** — `maturin sdist` produces a source distribution as +2. **sdist**: `maturin sdist` produces a source distribution as the PyPI fallback for niche architectures and a reproducibility anchor for the wheels. -3. **smoke-test** — pulls each wheel onto a clean runner of the +3. **smoke-test**: pulls each wheel onto a clean runner of the matching architecture, installs it with `pip install --no-index --find-links=dist big-code-analysis`, and asserts that the public API surface @@ -764,7 +888,7 @@ What the python-wheels pipeline does: shape under both Python 3.12 and 3.13. An abi3 wheel that loaded on 3.12 but failed on 3.13 (the most plausible silent forward-compat regression) trips here. -4. **publish** — gated on a `v*` tag and the `pypi` deployment +4. **publish**: gated on a `v*` tag and the `pypi` deployment environment. Authentication is via PyPI Trusted Publishing (OIDC); the workflow has no `PYPI_API_TOKEN` secret to leak. PEP 740 Sigstore attestations are generated automatically by @@ -794,7 +918,7 @@ on PyPI as the maintainer: The environment name is intentionally distinct from the `release` environment used by the crates.io trusted publisher - in `release.yml` — keeping them separate prevents the OIDC + in `release.yml`; keeping them separate prevents the OIDC `environment` claim from accidentally satisfying the wrong registry's TP entry. @@ -808,14 +932,14 @@ on PyPI as the maintainer: environment with **no protection rules** the first time the workflow runs. Create the environment manually *before* the first `v*` tag if you want the approval gate to apply on the - first publish — otherwise the cutover release goes through + first publish; otherwise the cutover release goes through immediately with no manual checkpoint. 4. **Create the `python-wheels` PR label.** The wheel build / sdist / smoke-test jobs are gated by a `python-wheels` label on PRs so Rust-only PRs that happen to share a path-filter neighbour (e.g. `Cargo.lock`) do not pay the wheel-matrix - cost. GitHub does not auto-create custom labels — until the + cost. GitHub does not auto-create custom labels; until the label exists, contributors cannot opt PRs into wheel verification. One-off via the `gh` CLI: @@ -825,13 +949,13 @@ on PyPI as the maintainer: --description "PR opts in to the manylinux wheel CI matrix" ``` - Tag pushes and `workflow_dispatch` runs ignore the label — + Tag pushes and `workflow_dispatch` runs ignore the label; they always build the full matrix. 5. **First tagged release validates the path.** Trusted Publishing cannot be rehearsed via `workflow_dispatch` (the environment claim mismatches). The first non-prerelease `v*` - tag after registration is the canonical end-to-end test — + tag after registration is the canonical end-to-end test; watch the `publish` job's log for the OIDC exchange and the attestation upload. @@ -841,7 +965,7 @@ on PyPI as the maintainer: `[workspace.package] version` via `version.workspace = true` in its `Cargo.toml`, and `pyproject.toml` reads the same value at build time (`dynamic = ["version"]`). The "Bumping the version" steps -above are therefore sufficient — there is no separate +above are therefore sufficient: there is no separate `big-code-analysis-py/pyproject.toml` version field to keep in sync. ### Testing a release candidate without uploading @@ -855,7 +979,7 @@ To exercise the PyPI side end-to-end against `https://test.pypi.org/`, temporarily change the `pypa/gh-action-pypi-publish` step's `repository-url` input to `https://test.pypi.org/legacy/` and register a matching TP entry -on TestPyPI — keep this off `main` to avoid leaking a real upload +on TestPyPI; keep this off `main` to avoid leaking a real upload into a production-shaped flow. ### Out of scope @@ -864,7 +988,7 @@ The wheel pipeline ships Linux only (x86_64 + aarch64). macOS and Windows wheels are tracked separately under [#103](https://github.com/dekobon/big-code-analysis/issues/103)'s "Out of scope" section. (This Linux-only scope is for the *library* -bindings wheel above; the **CLI** `bca` wheel — see below — does ship +bindings wheel above; the **CLI** `bca` wheel, see below, does ship macOS and Windows.) ## CLI wheels (PyPI) @@ -887,7 +1011,7 @@ onto the user's `PATH`. Key differences from the library wheel: installed command intentionally differs from the dist name (the `bca` name on PyPI is taken; `big-code-analysis` is the library bindings). - **Full grammar set is inherited from the crate** (`all-languages`, via - #252) — no `[tool.maturin] features` wiring. + #252); no `[tool.maturin] features` wiring. - **Wider platform matrix:** Linux `manylinux_2_28` (`x86_64` / `aarch64`), macOS (`x86_64` / `arm64`), Windows (`x86_64`). - **Compliance artefacts ride in the wheel:** the per-binary @@ -919,7 +1043,7 @@ Mirror the library-wheel setup above, with CLI-specific values: 3. **Create the `pypi-cli` GitHub Environment** (Settings → Environments → New environment → `pypi-cli`) before the first `v*` - tag if you want a manual approval gate on the first publish — GitHub + tag if you want a manual approval gate on the first publish; GitHub auto-creates a referenced-but-undefined environment with no protection rules otherwise. @@ -936,7 +1060,7 @@ Mirror the library-wheel setup above, with CLI-specific values: Tag pushes and `workflow_dispatch` runs ignore the label. 5. **First tagged release validates the path**, exactly as for the - library wheel — Trusted Publishing cannot be rehearsed via + library wheel: Trusted Publishing cannot be rehearsed via `workflow_dispatch`. ### Version coupling @@ -952,11 +1076,11 @@ Mirror the library-wheel setup above, with CLI-specific values: `minisign -G -p minisign.pub.new -s minisign.key.new`. 2. Replace `minisign.pub` with the new public key and commit it. 3. Update `MINISIGN_SECRET_KEY` and `MINISIGN_PASSWORD` secrets with - the new values. Use stdin redirection — `gh secret set - MINISIGN_SECRET_KEY -R dekobon/big-code-analysis < minisign.key.new` - — to preserve the trailing newline of the key file; see + the new values. Use stdin redirection (`gh secret set + MINISIGN_SECRET_KEY -R dekobon/big-code-analysis < minisign.key.new`) + to preserve the trailing newline of the key file; see [Minisign key](#minisign-key) for why paste-via-UI bites. -4. Cut a new release — its `SHA256SUMS.minisig` will be signed with +4. Cut a new release; its `SHA256SUMS.minisig` will be signed with the new key, self-documenting the rotation. Users verifying an older release still need the old `minisign.pub` @@ -972,13 +1096,13 @@ if `MINISIGN_SECRET_KEY` is missing, corrupted, or doesn't pair with `MINISIGN_PASSWORD`. `post-publish verify` runs *after* `publish` and is an internal -sanity check — its failure does not invalidate the published +sanity check; its failure does not invalidate the published artefacts and does not roll back any external state. Treat a `verify` red as a CI bug to triage, not as a botched release. If publish itself partially succeeds (e.g. GitHub Release created but tap push failed), the fix is usually to re-run the workflow against -the same tag — **Actions** tab → open the failed run → **Re-run +the same tag: **Actions** tab → open the failed run → **Re-run failed jobs** (top-right of the run page). The pipeline is designed to be idempotent on re-run, and re-runs pick up freshly-set repo secrets without needing a force-retag. @@ -990,7 +1114,7 @@ gh release delete vX.Y.Z --cleanup-tag --yes ``` Then fix the underlying issue, bump to `vX.Y.(Z+1)`, and re-tag. -**Do not re-use a published version number** — Homebrew/Scoop and +**Do not re-use a published version number**: Homebrew/Scoop and crates.io users may have already cached the old artefacts. ### Cutover-only escape hatch: force-moving the tag @@ -999,7 +1123,7 @@ The recovery rule above (bump to the next patch version) is correct for any release that already produced external state. On the *very first* tag for a brand-new repo, before `publish` has touched crates.io / Homebrew / Scoop, no downstream state exists yet to -poison — and bumping the version mid-cutover adds churn (workspace +poison, and bumping the version mid-cutover adds churn (workspace version, man pages, SARIF snapshots, CHANGELOG section). In that narrow window, force-moving the tag is the cheaper recovery: @@ -1023,28 +1147,19 @@ This is **only safe** while: won't roll back). Accept that single noisy red if the wheels are already correctly on PyPI. - crates.io has not yet been told about this version. ANY publish - for the version — workflow-driven *or* manual `cargo publish` from - the maintainer's workstation — makes a force-retag inappropriate, + for the version (workflow-driven *or* manual `cargo publish` from + the maintainer's workstation) makes a force-retag inappropriate, because the published version is irrevocable (yank-able, not delete-able). -Outside that window, never force-move — use `vX.Y.(Z+1)`. - -### Known pipeline issues - -Tracked as GitHub issues; a maintainer triaging a red run should -check these first before deeper debugging: - -- [#346](https://github.com/dekobon/big-code-analysis/issues/346) — - `x86_64-unknown-freebsd` dropped from the binary matrix; cross - v0.2.5 + the vendored grammars' C++ scanners cannot link against - `libcxxrt` without a deeper toolchain change. Restoration via - `vmactions/freebsd-vm` is the queued remediation. While the target - is absent, FreeBSD users install from source. -- [#351](https://github.com/dekobon/big-code-analysis/issues/351) — - `post-publish verify` fails on a brand-new release because - `SHA256SUMS` is emitted with `./`-prefixed filenames and the - verify-step awk filter compares against the unprefixed basename. - The artefacts themselves verify correctly with a manual - `sha256sum -c SHA256SUMS` (sha256sum canonicalises `./X` to `X`). - Will be fixed alongside the producer in v1.0.1. +Outside that window, never force-move; use `vX.Y.(Z+1)`. + +### Known pipeline limitations + +The one standing limitation a maintainer should know about: + +- `x86_64-unknown-freebsd` is absent from the binary matrix + ([#346](https://github.com/dekobon/big-code-analysis/issues/346), + closed): cross v0.2.5 + the vendored grammars' C++ scanners cannot + link against `libcxxrt` without a deeper toolchain change. While + the target is absent, FreeBSD users install from source. diff --git a/SECURITY.md b/SECURITY.md index 4bf02c58..291fd462 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -47,11 +47,10 @@ to release a fix before public disclosure. ## Verifying release artefacts -Every `v*` tag is intended to publish signed release artefacts to -[GitHub Releases](https://github.com/dekobon/big-code-analysis/releases). -Until the release pipeline lands (tracked in -[`RELEASING.md`](RELEASING.md)), the verification recipe below is the -expected post-release shape: +Every `v*` tag publishes signed release artefacts to +[GitHub Releases](https://github.com/dekobon/big-code-analysis/releases) +(the pipeline is described in [`RELEASING.md`](RELEASING.md)). Each +release carries: - **`SHA256SUMS`** — SHA-256 hashes of every artefact in the release. - **`SHA256SUMS.minisig`** — diff --git a/STABILITY.md b/STABILITY.md index 6f3daf0f..91081711 100644 --- a/STABILITY.md +++ b/STABILITY.md @@ -1,12 +1,12 @@ # Stability policy -`big-code-analysis` is on the `2.x` line (currently `2.0.1`). The +`big-code-analysis` is on the `2.x` line (currently `2.0.0`). The crate graduated from the rapid-iteration `0.x` window into a written stability contract and has now made its first major (`2.0`) break: callers can pin a minor version and expect a working tree against any subsequent patch or minor bump, without code edits. This document -spells out exactly what that contract covers across the `2.x` line — -and the narrow seams where it deliberately does not reach. +spells out exactly what that contract covers across the `2.x` line, and +the narrow seams where it deliberately does not reach. ## Versioning scheme @@ -31,8 +31,8 @@ pin bump (any `tree-sitter-*` crate in the root `Cargo.toml`) or a regression fix in a metric definition is by definition a value change but does not break the API surface. Callers who need bit-for-bit reproducibility should pin to an exact version -(`big-code-analysis = "= 2.0.1"`) and store that version alongside -their results — see [What is stable in value](#what-is-stable-in-value) +(`big-code-analysis = "= 2.0.0"`) and store that version alongside +their results; see [What is stable in value](#what-is-stable-in-value) below. [semver]: https://semver.org/spec/v2.0.0.html @@ -46,7 +46,7 @@ section. - **Language identification** - `LANG` enum (generated by the `mk_langs!` macro invoked from - `src/langs.rs`; the macro itself lives in `src/macros.rs`) — + `src/langs.rs`; the macro itself lives in `src/macros/mod.rs`): variants are additive. Adding a new variant in a minor bump is allowed; renaming or removing one is a `3.0` break. Derives `Hash` and implements `Display` (the `name` string) and @@ -57,16 +57,16 @@ section. `LANG::from_str(&lang.to_string()) == Ok(lang)` for every variant. This slug is the single language identifier across every surface (the CLI JSON `language` field, the web `language` field on every - analysis endpoint — `/metrics`, `/comment`, `/function` (#541) — + analysis endpoint (`/metrics`, `/comment`, `/function` (#541)) and the Python bindings). The human-pretty `c/c++` / `c#` - display forms were dropped at `2.0` — a break in the serialized + display forms were dropped at `2.0`, a break in the serialized `language` value. - `get_language_for_file`, `guess_language` in `src/tools.rs`. - **Top-level entry points** - - `analyze` and `Source` in `src/spaces.rs` — the recommended + - `analyze` and `Source` in `src/spaces.rs`: the recommended library entry point. `Source` carries `#[non_exhaustive]` and its fields are private (`pub(crate)`): construct via `Source::new` plus - `with_*` setters. Only that builder API is the contract — the field + `with_*` setters. Only that builder API is the contract: the field set, names, and types (e.g. `code: &[u8]`, `name: String`) are free to change and are **not** SemVer-protected (#533). - `MetricsOptions` carries `#[non_exhaustive]` and, like `Source`, @@ -76,7 +76,7 @@ section. parser-generic `metrics` / `metrics_with_options` functions and the `MetricsCfg` builder were removed at `2.0` in favour of `analyze` / `Ast`. - - `Metric`, `MetricSet` in `src/metric_set.rs` — `Metric` carries + - `Metric`, `MetricSet` in `src/metric_set.rs`: `Metric` carries `#[non_exhaustive]`, so adding variants is additive. `MetricSet` is the opaque bitfield consumed by `MetricsOptions::with_only(&[Metric])` and read back through @@ -114,7 +114,7 @@ section. `pub(crate)`, not part of the public surface, and the parser-generic `operands_and_operators` function was removed in favour of `Ast::ops`. - - `NumJobs` in `src/concurrent_files.rs` (added in 1.x, #560) — the + - `NumJobs` in `src/concurrent_files.rs` (added in 1.x, #560): the shared `` worker-count selector for `ConcurrentRunner`, used by both the `bca` CLI and the `bca-web` server. A clap-agnostic plain `FromStr` + `resolve() -> usize` type; `Auto` @@ -126,7 +126,7 @@ section. `ParseMetricError` / `ParseLangError` convention; it is deliberately exhaustive so callers can match both failure modes without a wildcard. - - `MetricsError` in `src/error.rs` — carries `#[non_exhaustive]`, + - `MetricsError` in `src/error.rs`: carries `#[non_exhaustive]`, so adding variants is additive. Current variants are `LanguageDisabled(LANG)` (the only one produced today) and the reserved-for-future `EmptyRoot`. The previously-reserved @@ -148,18 +148,18 @@ section. - `FunctionSpan` in `src/function.rs`. - **Offender / catalog enums** - `Severity` in `src/output/offenders.rs` (re-exported from the - crate root) — an ordered severity scale carrying + crate root): an ordered severity scale carrying `#[non_exhaustive]`. As of #552 it derives `Hash`, `PartialOrd`/`Ord` keyed on declaration order. Variants are declared least-severe-first (`Warning` then `Error`), so the - derived ordering is `Error > Warning` — callers may rely on + derived ordering is `Error > Warning`; callers may rely on `severities.iter().max()` selecting the worst tier and on `>= Severity::Error` gating. Any future tier (`Info`/`Note`) must be inserted in the correct severity position to preserve this contract. - - `metric_catalog::Direction` — derives `Hash` (#552) so it can key + - `metric_catalog::Direction`: derives `Hash` (#552) so it can key a `HashMap`/`HashSet`. Adding these derives is additive. -- **Per-metric `Stats`** — one `Stats` struct under each +- **Per-metric `Stats`**: one `Stats` struct under each `src/metrics/.rs` (`abc`, `cognitive`, `cyclomatic`, `halstead`, `loc`, `mi`, `nargs`, `nexits`, `nom`, `npa`, `npm`, `wmc`, plus `tokens`). These structs carry `#[non_exhaustive]` @@ -180,21 +180,21 @@ section. Every per-metric `Stats` derives `PartialEq` (#552), so callers can compare two compute-side `Stats` directly without round-tripping - through `to_wire()`. `Eq` is intentionally **not** derived — the + through `to_wire()`. `Eq` is intentionally **not** derived: the float fields (ratios, averages, ABC `magnitude`, the derived Halstead scores) preclude it. Comparing `Stats` from two analyses of the *same* deterministic source is exact-equal-safe (identical code path, identical `f64` bits); do not rely on cross-input float equality. Adding `PartialEq` is additive: it only widens the derived-trait set. -- **Prelude** — `big_code_analysis::prelude` re-exports the +- **Prelude**: `big_code_analysis::prelude` re-exports the recommended entry points (`analyze`, `Ast`, `Source`, `MetricsOptions`, `MetricsError`, `LANG`, `FuncSpace`, `CodeMetrics`, `SpaceKind`, `Metric`). The `metrics_from_tree` entry was removed at `2.0` along with the free function it named; tree adoption now goes through `Ast::from_tree_sitter`. New items may be added; nothing in the set is removed before `3.0`. -- **Per-language Cargo features** — the feature set +- **Per-language Cargo features**: the feature set (`all-languages`, plus per-language features `rust`, `typescript`, `python`, `cpp`, …) is itself part of the contract. Adding a new language feature is additive. Removing one is a @@ -215,8 +215,8 @@ and the matching `bca.toml` `[vcs] file_types` key) and `bca metrics markdown|html --vcs` "Change-history risk" section (#573) are **one-way rendered projections** (like the AST `bca report` output): the page *structure* is stable within `2.x`, but the exact bytes are not a -round-trip format — do not parse them. The 2.0 line restructured the AST -report's presentation deliberately — hotspot section titles now follow one +round-trip format; do not parse them. The 2.0 line restructured the AST +report's presentation deliberately: hotspot section titles now follow one ` hotspots (top N by )` template (#677), the WMC table is labelled "Types" (#687), and the HTML section anchors derive from a stable ` hotspots` slug independent of `--top`, so a deep link minted @@ -226,8 +226,8 @@ no round-trip / wire shape moved. `bca vcs --output` names a single file (a whole-repo report is one document); as of 2.0 `bca metrics`/`bca ops --output` also names a single aggregate file, with the per-file *directory* tree now written by `--output-dir` (#669). The composite -`risk_score` is **ordinal, not cardinal** — only -relative ranks are meaningful — and is formula-versioned +`risk_score` is **ordinal, not cardinal** (only +relative ranks are meaningful) and is formula-versioned (`risk_score_version`): the formula may change within `2.x`, but any change bumps that field, and the serialized field *set* is versioned by `vcs_schema_version`. Per-file score *magnitudes* therefore carry the @@ -237,9 +237,9 @@ In `2.0` the per-file VCS block became a **nested `vcs` object** under each ranked file (and each `/vcs/trend` point), replacing the former `#[serde(flatten)]`-beside-`path` layout, so it now reads like every other metric group (#684). The block is also **always-slim** (#635): -the four constant stamps that hold across an entire response — -`vcs_schema_version`, `risk_score_version`, `long_window_days`, -`recent_window_days` — are carried exactly once on the enclosing +the four constant stamps that hold across an entire response +(`vcs_schema_version`, `risk_score_version`, `long_window_days`, +`recent_window_days`) are carried exactly once on the enclosing envelope (`bca vcs`'s report, `POST /vcs`, `vcs_metrics()`, and the `/vcs/trend` document), never repeated per row or per trend point. The CSV projection stays flat with dotted column names. @@ -250,8 +250,8 @@ surface: `big_code_analysis::vcs::{PerFunctionBlame, LineSpan}`, the and the `CodeMetrics::vcs` field now being populated on nested function spaces (not only the file space). The per-function block shares `wire::Vcs`'s shape and the same `risk_score` / `*_version` contract -above, but its numbers are a `git blame` *current snapshot* — `churn` is -surviving-line count, not the file-level added+deleted churn — so values +above, but its numbers are a `git blame` *current snapshot* (`churn` is +surviving-line count, not the file-level added+deleted churn), so values are not comparable across the file and function levels by construction. Just-in-time (commit-level) risk scoring (#331) adds a further opt-in @@ -286,7 +286,7 @@ VcsAggregate}` plus the `BUS_FACTOR_SCHEMA_VERSION` constant, the `vcs::Error::InvalidBusFactorThreshold` variant. The aggregate is surfaced as a top-level `vcs_aggregate` object by `bca vcs`, `bca report --vcs`, `POST /vcs`, and `vcs_metrics()`, and is gated on the -front end opting in (`compute_bus_factor`), so it is purely additive — no +front end opting in (`compute_bus_factor`), so it is purely additive: no existing field moved and `vcs_schema_version` is unchanged. Like the JIT report, the bus-factor types derive `Serialize` directly; their shape is stable within `2.x` and versioned by `BUS_FACTOR_SCHEMA_VERSION`. The @@ -306,12 +306,12 @@ projection, and the `vcs::Error::InvalidTrend` variant. The trend is surfaced by `bca vcs trend`, `POST /vcs/trend`, and `vcs_trend()`. Each sampled point's metric block is the same `wire::Vcs` shape (so its fields and `risk_score` carry the same ordinal/versioned contract as `bca vcs`); -the *container* shape — `as_of_points`, the per-file point arrays (`null` -where a file did not exist), and the `deltas` summary — is stable within +the *container* shape (`as_of_points`, the per-file point arrays (`null` +where a file did not exist), and the `deltas` summary) is stable within `2.x` and versioned by `trend_schema_version`. The per-point timestamps and the delta magnitudes are derived from the same ordinal `risk_score`, so the "magnitudes are not byte-stable across bumps" caveat applies. The -new module is additive — no existing field moved. +new module is additive: no existing field moved. The persistent change-history **cache** (#334) is an additive, opt-out optimization: `big_code_analysis::vcs::{build_history_index_cached, @@ -321,8 +321,8 @@ by `bca vcs --no-cache` / `--clear-cache` / `--cache-dir` (and reused transparently by `bca metrics --vcs` / `bca report --vcs`), the `POST /vcs` `no_cache` / `cache_dir` fields, and the `vcs_metrics(no_cache=…, cache_dir=…)` parameters. The contract is purely -behavioural — a cache hit produces output **bit-identical** to an uncached -`build_history_index` at the same reference time — so the cache never +behavioural (a cache hit produces output **bit-identical** to an uncached +`build_history_index` at the same reference time), so the cache never changes the `wire::Vcs` shape or any metric value. The **on-disk file format** (the `HistoryCache` / `CommitEvent` JSON, versioned by `CACHE_SCHEMA_VERSION`) is deliberately **not** a stability surface: it is @@ -330,8 +330,8 @@ private machine-local state under the user's cache directory, may change between releases (a stale or unreadable entry is silently recomputed), and must not be parsed or relied on by downstream code. -The VCS front-end **input** config structs — `vcs::Options` and -`vcs::CacheConfig` — are both `#[non_exhaustive]`. Downstream crates +The VCS front-end **input** config structs, `vcs::Options` and +`vcs::CacheConfig`, are both `#[non_exhaustive]`. Downstream crates construct them by starting from `Options::default()` / `CacheConfig::default()` and assigning the `pub` fields they need, **not** by struct literal or `..Default::default()` functional update (both of which `#[non_exhaustive]` @@ -363,7 +363,7 @@ The following are explicitly **not** part of the shape contract: `mk_langs!` macro emits them, but they are intended to be reached through `LANG` rather than referenced by name. - `Parser` and the per-language `Checker` / `Getter` / `Alterator` - trait impls — these are internal plumbing. As of `2.0` they are + trait impls: these are internal plumbing. As of `2.0` they are `pub(crate)`, not a public extension surface: `ParserTrait`, `Parser`, `Filter`, `Cursor`, and the per-metric compute traits (`Cognitive`, `Cyclomatic`, `Halstead`, `Loc`, `Mi`, `Nom`, @@ -372,7 +372,7 @@ The following are explicitly **not** part of the shape contract: `pub(crate)`. `LanguageInfo` was likewise demoted to `pub(crate)`, and the `Callback` trait / `AstCallback` dispatch were removed at `2.0`. None of these are reachable from the public API or appear in - the curated rustdoc — treat them as internal plumbing. + the curated rustdoc; treat them as internal plumbing. [changelog]: ./CHANGELOG.md @@ -388,7 +388,7 @@ versions, even within `2.x`.** Concretely: - A bug fix in a metric definition (cyclomatic, cognitive, ABC, Halstead operator classification, …) is by definition a value change for the files it touches. We do not hold these for major - bumps — fixing a wrong number sooner is more valuable than + bumps: fixing a wrong number sooner is more valuable than freezing it. - Operator / operand classification for Halstead, branch detection for cyclomatic, and exit-point counting continue to be refined @@ -399,14 +399,14 @@ versions, even within `2.x`.** Concretely: This is the one deliberate carve-out in the `2.x` contract. The *shape* of the data is stable; the *numbers in the cells* are not. If you need to compare metric runs across time, pin to an exact -version (`big-code-analysis = "= 2.0.1"`) and store the version +version (`big-code-analysis = "= 2.0.0"`) and store the version alongside the results. ### Float precision and non-finite values The structured serializers (JSON, YAML, TOML, CBOR) emit **full `f64` precision** for the float-valued fields (ratios, averages, -ABC `magnitude`, the derived Halstead scores, and the MI scores) — +ABC `magnitude`, the derived Halstead scores, and the MI scores); they are not rounded. The exact decimal expansion of a float magnitude is therefore **not byte-stable** across versions or platforms: a metric-definition fix or a grammar bump can move the @@ -447,11 +447,11 @@ public `big_code_analysis::wire` module: `wire::FuncSpace`, struct per metric (`wire::Abc`, `wire::Cognitive`, `wire::Cyclomatic`, `wire::Nexits`, `wire::Halstead`, `wire::Loc`, `wire::Mi`, `wire::Nargs`, `wire::Nom`, `wire::Npa`, `wire::Npm`, `wire::Tokens`, -`wire::Wmc`) plus the `wire::CyclomaticModified` sub-record — the +`wire::Wmc`) plus the `wire::CyclomaticModified` sub-record, the `modified` projection nested inside `wire::Cyclomatic`. Each derives `Serialize` + `Deserialize`. These are the **single definition of the -serialized shape** — the compute types' own `Serialize` impls delegate -to them — so the document a compute type emits is exactly the document +serialized shape** (the compute types' own `Serialize` impls delegate +to them), so the document a compute type emits is exactly the document the matching `wire` type parses. **Round-trip contract.** For a value produced by this library and read @@ -470,14 +470,14 @@ back with *this library's* serde stack: rebuilds the `MetricSet` from the present keys. The bit-exactness of float magnitudes is a property of *this library's* -parser configuration, not of JSON text in general — a downstream consumer +parser configuration, not of JSON text in general: a downstream consumer parsing the same document with a stock serde_json (no `float_roundtrip`) may differ by up to 1 ULP. Per the value-stability carve-out above, do not treat float magnitudes as a stable cross-version identity; compare them with a tolerance. Note that `bca --version` prints the CLI -binary's own version, not the library version — to record the +binary's own version, not the library version. To record the library version from a CLI run, capture it from `Cargo.lock` or the resolved `cargo metadata` output for the `big-code-analysis` package. @@ -492,7 +492,7 @@ pin to the last patch of `2.x` before moving to `3.0`. These public items are intentionally lower-level and follow their underlying dependency rather than our own SemVer. They are the narrow seams where the `2.x` shape contract gives way to the -upstream grammar contract — depend on them only when you need to +upstream grammar contract; depend on them only when you need to reach the raw tree-sitter surface. - **`Node` exposes its `tree_sitter::Node` through an accessor.** @@ -505,7 +505,7 @@ reach the raw tree-sitter surface. This mirrors the higher-level `Ast::as_tree_sitter` seam below; reach for `Node::as_tree_sitter` only when you already hold a `Node` from this surface. **(breaking, 2.0)** The pre-2.0 shape - was `pub struct Node<'a>(pub OtherNode<'a>)` — a public tuple + was `pub struct Node<'a>(pub OtherNode<'a>)`, a public tuple field welding the value-not-stable `tree_sitter::Node` into the stable struct's layout. #556 demoted the field to private and added the accessor; #534 had already narrowed the sibling @@ -514,8 +514,8 @@ reach the raw tree-sitter surface. `2.0`. - **`tree_sitter` is re-exported as `big_code_analysis::tree_sitter`.** Consumers who construct trees themselves should depend on the - re-export rather than adding a sibling `tree-sitter` dependency - — that guarantees the `tree_sitter::Tree` they pass into + re-export rather than adding a sibling `tree-sitter` dependency; + that guarantees the `tree_sitter::Tree` they pass into `Ast::from_tree_sitter` agrees with the version the metric walker was compiled against. The re-export follows our pin: bumping `tree-sitter` to a new version is a @@ -539,9 +539,9 @@ reach the raw tree-sitter surface. at `2.0` (`Parser::from_tree` demoted to `pub(crate)` along with `Parser`; the parser-generic `metrics_from_tree` deleted) in favour of this single explicit-name seam. -- **`Ast`** is the parse-once seam — and, as of `2.0`, the single +- **`Ast`** is the parse-once seam and, as of `2.0`, the single public analysis entry point. `Ast::as_tree_sitter` exposes the held - `tree_sitter::Tree` — that single method follows the `tree-sitter` + `tree_sitter::Tree`; that single method follows the `tree-sitter` pin in the same value-not-stable sense as the `tree_sitter` re-export. The rest of `Ast`'s API surface (`parse`, `from_tree_sitter`, `metrics`, `ops`, `strip_comments`, `functions`, @@ -580,7 +580,7 @@ output-specific flag (`--paths`/`-p`, `--include`/`-I`, `--exclude`/`-X`, `--no-skip-generated`, `--paths-from`, `--exclude-from`) must follow the subcommand (`bca metrics --paths src`, never `bca --paths src metrics`). A flag passed to a subcommand that never consumed it is a hard usage -error (exit 1), not a silent no-op — `bca vcs commit --exclude-tests` +error (exit 1), not a silent no-op: `bca vcs commit --exclude-tests` and `bca list-metrics --paths` both error. Input paths are also accepted positionally on the walking subcommands (`bca metrics src/`), unioned with `--paths` (#651); `bca find` / `bca count` take node kinds via a @@ -624,16 +624,16 @@ mis-matching. Recent transitions: root) emits source metrics in two families of format. The split between them is itself part of the contract: -- **Round-trip formats** — JSON, YAML, TOML, CBOR. These project the +- **Round-trip formats**: JSON, YAML, TOML, CBOR. These project the metric tree through the `wire` module and read back into the matching `wire` types (see [Reading serialized output back](#reading-serialized-output-back-the-wire-module)). A document this library writes, parsed with this library's serde stack, reconstructs the tree and re-serializes byte-for-byte. -- **One-way projections** — CSV, SARIF, Checkstyle, code-climate, +- **One-way projections**: CSV, SARIF, Checkstyle, code-climate, the Clang/MSVC warning lines, and the AST / `dump_*` output. These are lossy, write-only views with **no reader** in this crate. They - drop or reshape data the round-trip formats keep — e.g. the CSV row + drop or reshape data the round-trip formats keep: e.g. the CSV row carries no per-`FuncSpace` `suppressed` scope (it is present in `wire::FuncSpace` but absent from the flattened CSV columns). Do not expect to parse a projection back into a `FuncSpace`; pin to a @@ -646,7 +646,7 @@ at the crate root from `src/output/`: | Writer | Signature (eliding ``) | |--------|----------------------------------| | `write_csv` | `(space: &FuncSpace, source_path: &Path, writer: W) -> io::Result<()>` | -| `write_csv_aggregate` | `(spaces: impl IntoIterator, writer: W) -> io::Result<()>` — one shared header, then every tree's rows | +| `write_csv_aggregate` | `(spaces: impl IntoIterator, writer: W) -> io::Result<()>`; one shared header, then every tree's rows | | `write_sarif` | `(offenders: &[OffenderRecord], writer: W) -> io::Result<()>` | | `write_sarif_with_suppressed` | `(active, suppressed: &[OffenderRecord], writer: W) -> io::Result<()>` | | `write_checkstyle` | `(offenders: &[OffenderRecord], writer: W) -> io::Result<()>` | @@ -654,9 +654,9 @@ at the crate root from `src/output/`: | `write_msvc_warning` | `(offenders: &[OffenderRecord], writer: W) -> io::Result<()>` | | `write_code_climate` | `(offenders: &[OffenderRecord], writer: W) -> io::Result<()>` | -The shared types they consume — `OffenderRecord`, `Severity` (see +The shared types they consume (`OffenderRecord`, `Severity` (see *Offender / catalog enums*), `TOOL_ID` (the `"big-code-analysis"` -tool name), `CSV_HEADER`, and `CSV_EXTENSION` (`".csv"`) — are part of +tool name), `CSV_HEADER`, and `CSV_EXTENSION` (`".csv"`)) are part of the same surface. The AST dump entry points (`dump_node`, `dump_root`, `dump_ops`, and the `Dump` / `DumpCfg` config types) are likewise re-exported and shape-stable. Each terminal dump entry point also has a @@ -720,7 +720,7 @@ integer metrics exact `u64`; `CellMetric` renders integer-valued finites without a decimal point.) The integer-*named* columns (every `*.sum`, `*.min`, `*.max`, the counts, `halstead.length`/`vocabulary`, the WMC/NOM/NARGS/ABC/NPM/NPA -count columns) are always populated — they are never non-finite, so the +count columns) are always populated: they are never non-finite, so the empty-cell branch fires only for the float-derived columns (`*_average`, ratios, `abc.magnitude`, the derived Halstead scores, the `mi.*` family). @@ -747,7 +747,7 @@ and `location.positions.begin.{line,column}` (the last two emitted only when present). The spec's `type`, `categories`, `remediation_points`, and `content` are deliberately omitted (additive to re-add later). -The **`fingerprint` is contractually stable** — GitLab persists it +The **`fingerprint` is contractually stable**: GitLab persists it across pipeline runs to deduplicate and to compute the base-vs-head diff. It is the SHA-256 of `path \0 function \0 metric` (NUL-separated; `function` is the empty string when the offender is file-scoped), @@ -759,19 +759,19 @@ violation. When two distinct findings in one file would otherwise share the same `path \0 function \0 metric` triple (same-named functions breaching the -same metric — a collision that previously made one finding vanish under +same metric, a collision that previously made one finding vanish under GitLab's fingerprint dedup), a `\0 ` disambiguator (the occurrence index as a little-endian `u32`) is appended before hashing. This is backward-compatible: the **first** occurrence carries ordinal `0` and so keeps the byte-identical historical fingerprint; only the -second-and-later colliding findings — which previously had no stable -fingerprint of their own — receive a new, distinct value. +second-and-later colliding findings, which previously had no stable +fingerprint of their own, receive a new, distinct value. #### AST / dump JSON `dump_*` and the `AstNode` / `Span` types (re-exported with `AstPayload` / `AstResponse` / `AstCfg`) serialize a -**one-way projection** of the parse tree — `Serialize` only, by +**one-way projection** of the parse tree: `Serialize` only, by design. There is intentionally **no** `Deserialize`: the AST output is a debugging / inspection view like CSV and SARIF, not a wire format you round-trip through. The JSON shape uses `snake_case` keys and is pinned @@ -790,8 +790,8 @@ the #535 snake_case landing change. ### Schema-bump operational contract CLI artifact schemas (the `version` field on `.bca-baseline.toml`, -the shape of the `[thresholds]` config — the `bca.toml` manifest table -or a `--config` file — the on-disk report formats) are +the shape of the `[thresholds]` config (the `bca.toml` manifest table +or a `--config` file), the on-disk report formats) are read by whichever `bca` binary the user has installed. A schema bump in the binary that goes out without a coordinated update to the *install-bca* surfaces leaves downstream users in a state where @@ -805,7 +805,7 @@ constant must also: 1. Update [`.github/workflows/pages.yml`](.github/workflows/pages.yml) if it pins a `bca` release (today it builds from checkout, so - this is automatic — but a future revert to a pinned install + this is automatic, but a future revert to a pinned install would re-introduce the gap). 2. Update every install-bca example in the [CI integration recipe](https://dekobon.github.io/big-code-analysis/recipes/ci.html): @@ -817,7 +817,7 @@ constant must also: merge the schema bump → cut a release on the new commit → bump the documented `BCA_VERSION` to that release. Skipping step 2 ships a documented install path that cannot read the artifact - `main` now writes — exactly the failure mode that motivated + `main` now writes, exactly the failure mode that motivated the pages.yml workflow's switch to build-from-checkout. The schema author owns this checklist. The release-prep section of @@ -830,7 +830,7 @@ library SemVer surface: | Exit | Meaning | |------|---------| -| `0` | clean — no threshold violations | +| `0` | clean: no threshold violations | | `1` | tool error (bad config, unknown metric, unreadable path) | | `2` | one or more threshold violations | @@ -870,19 +870,19 @@ alias for `--exit-codes=tiered` (warns; removed at the next major, `3.0`). value-taking `--tier ` ([#688](https://github.com/dekobon/big-code-analysis/issues/688)): -- `hard` (default) — flag a function only at/over its `[thresholds]` +- `hard` (default): flag a function only at/over its `[thresholds]` limit. -- `soft` — early-warning tier flagging a function at `RATIO` of any +- `soft`: early-warning tier flagging a function at `RATIO` of any limit (default `0.95`), before the hard gate trips. With a `[thresholds.soft]` table present, its per-metric limits take precedence over the blanket ratio. -- `soft=0.90` — soft tier scaling every limit by `0.90`; `soft=1.0` +- `soft=0.90`: soft tier scaling every limit by `0.90`; `soft=1.0` disables the blanket scale. A bare `--tier` means `soft`. The manifest `[check] headroom` key supplies the soft ratio for a bare `--tier=soft`. The retired `--headroom ` flag is a hidden one-cycle alias for `--tier=soft=` -(warns; removed at the next major, `3.0`) — it now promotes a hard run to the +(warns; removed at the next major, `3.0`); it now promotes a hard run to the soft tier rather than being ignored at the hard tier. ### CLI / manifest list-merge semantics @@ -902,7 +902,7 @@ list, how the two are combined depends on the *meaning* of the list only when the CLI passed nothing. - **Negative filter keys (`exclude`, `[check] exclude`) UNION CLI values with the manifest list.** A CLI `--exclude`/`--check-exclude` - is *added to* the manifest's exemptions, never a replacement — so a + is *added to* the manifest's exemptions, never a replacement, so a command-line filter can never silently un-exclude a directory the project config deliberately skipped (e.g. `vendor/`). Duplicates across the two sources collapse; CLI patterns sort first. This @@ -925,7 +925,7 @@ change afterward without breaking every consumer's imports. - **Distribution name** `big-code-analysis` (the `pip install` target / PyPI project name). - **Import name** `big_code_analysis` (the top-level package). -- **Compiled extension** `big_code_analysis._native` — a private, +- **Compiled extension** `big_code_analysis._native`: a private, `#[doc(hidden)]`-equivalent module. It is an implementation detail: import from the package (`big_code_analysis.analyze`), never from `_native` directly. Its layout may change between @@ -938,14 +938,14 @@ The bound surface tracks the library: `analyze`, `analyze_source`, `supported_languages`, `to_sarif`, `flatten_spaces`, the `Ast` parse-once handle (#727) and the lazy `Node` traversal handle it hands out via `Ast.root_node` / `Ast.find` (#728), the `AnalysisFailure` -value type (a per-file batch failure, **returned not raised** — renamed +value type (a per-file batch failure, **returned not raised**; renamed from `AnalysisError` at 2.0, #614), the `ParseError` / `UnsupportedLanguageError` exception types, the change-history exception taxonomy (`VcsError` and its `NotARepositoryError` / `InvalidRevisionError` / `InvalidDiffError` / `VcsEnvironmentError` subclasses), `__version__`, and the `METRIC_NAMES` constant. The change-history surface lives in the `big_code_analysis.vcs` submodule (#612): `vcs.rank` / `vcs.trend` / -`vcs.commit` / `vcs.score_diff` plus the shared `vcs.Options` object — +`vcs.commit` / `vcs.score_diff` plus the shared `vcs.Options` object, names mirroring the `bca vcs` CLI subcommands. ### Language and metric string enums @@ -958,8 +958,8 @@ returns `Lang | None`, and `METRIC_NAMES` is a "halstead"`, and the members work anywhere a string slug does (`metrics=[MetricName.LOC]`, `language_extensions(Lang.RUST)`). The enums are **generated** from the same upstream tables the CLI and -JSON output use — `LANG::name()` (the canonical slugs, see *What is -stable in shape* above) and `Metric::NAMES` — by a checked-in +JSON output use, `LANG::name()` (the canonical slugs, see *What is +stable in shape* above) and `Metric::NAMES`, by a checked-in generator with a drift-gate test, so the Python values can never diverge from the slugs emitted elsewhere. The contract is: **every `Lang` / `MetricName` value equals the corresponding CLI / JSON @@ -970,17 +970,17 @@ the existing values are frozen. Per-file failures from `analyze_batch` / `analyze_paths` are **returned**, not raised, as `AnalysisFailure` values (not `Exception` -subclasses — the class is deliberately not raisable; it was renamed from +subclasses: the class is deliberately not raisable; it was renamed from `AnalysisError` at 2.0 because the `…Error` suffix mislead readers into -`except` clauses, #614). The `error_kind` field is a closed set — -`Literal["UnsupportedLanguage", "ParseError", "IoError"]` — and is +`except` clauses, #614). The `error_kind` field is a closed set +(`Literal["UnsupportedLanguage", "ParseError", "IoError"]`) and is part of the contract: callers may branch on it. The raising entry points (`analyze`, `analyze_source`) map upstream failures to `UnsupportedLanguageError` (a `ValueError` subclass), `ParseError`, or the appropriate `OSError` subclass. The change-history surface (`vcs.rank` / `vcs.trend` / `vcs.commit` / `vcs.score_diff`) maps `vcs::Error` variants to `VcsError` (a `ValueError` subclass) and its -named subclasses — +named subclasses: `NotARepositoryError`, `InvalidRevisionError`, `InvalidDiffError`, `VcsEnvironmentError`. The subclasses are pinned additively: a caller may catch the specific class or the `VcsError` / `ValueError` base. @@ -1005,14 +1005,14 @@ nested metric blocks (`CodeMetricsDict`, `LocDict`, `HalsteadDict`, **generated** from the `big_code_analysis::wire` structs (`src/wire.rs`, the single source of the serialized shape) by a checked-in generator with a drift-gate test, so the Python types cannot diverge from the JSON the -CLI emits. The change is stub-only — the runtime values are plain -dicts, byte-identical to the CLI output — so it only narrows the +CLI emits. The change is stub-only (the runtime values are plain +dicts, byte-identical to the CLI output), so it only narrows the static type. Every metric block is `NotRequired` because a `metrics=` selection can elide blocks; under the default full suite every block is present. The VCS *report* dicts are now single-sourced and typed too (#664): `vcs.rank` returns `VcsReportDict`, `vcs.trend` returns `VcsTrendDict`, `vcs.commit` returns `JitCommitReportDict`, and -`vcs.score_diff` returns `JitDiffReportDict` — the report / trend envelope +`vcs.score_diff` returns `JitDiffReportDict`. The report / trend envelope structs moved into `big_code_analysis::wire` and the jit shapes are mirrored from `src/vcs/jit.rs`, so the former `dict[str, Any]` returns are gone. The same additive / major-only shape contract applies. @@ -1035,7 +1035,7 @@ contract points are: empty when the request carried none), a `language` field, and the endpoint's result. The `language` value is the #540 canonical slug (routed through `guess_language` → `LANG::name()`), the same - identifier used by the CLI JSON output and the Python bindings — the + identifier used by the CLI JSON output and the Python bindings; the comment endpoint reports the *guessed* language, not its internal `ccomment` grammar swap. `/ast` joined the language echo at `2.0` (#654), bringing the published `AstResponse` library type to @@ -1044,8 +1044,8 @@ contract points are: returned under the `root` key (a single object, not the misleading plural `spaces`); its own nested `spaces` list holds the child spaces. The request flag selecting file-level-only output is - `scope` — `"full"` (default, the full nested tree) or `"file"` (the - root only) — both as a JSON-body field and as a query parameter on + `scope`: `"full"` (default, the full nested tree) or `"file"` (the + root only), both as a JSON-body field and as a query parameter on the octet-stream variant. The former plural `spaces` envelope key and the boolean `unit` flag were renamed as a `2.0`-line break (#638); the old `unit` key now fails as an unknown field (see *Unknown @@ -1062,7 +1062,7 @@ contract points are: and whose `error_kind` is `unknown_field` (#633). A typo can no longer silently change analysis semantics; clients probing for feature support use the `GET /v1` route index (#643) instead. This is a - `2.0`-line break — payloads with extra/typo'd fields that `200`'d + `2.0`-line break: payloads with extra/typo'd fields that `200`'d before now `400`. - **`/comment` result shape.** The JSON variant returns `code` as a **string** holding the stripped source; the request `code` arrived @@ -1073,7 +1073,7 @@ contract points are: was changed to `string` as a `2.0`-line break (#629). - **Empty `/comment` result.** When the source has no removable comments, both content-type variants signal it as `200` with an - empty payload — the JSON variant returns `{… "code": ""}` (empty + empty payload: the JSON variant returns `{… "code": ""}` (empty string) and the octet-stream variant returns an empty body. The former `204 No Content` on the octet-stream variant was removed as a `2.0`-line break (#558) so the empty outcome shares one status code @@ -1084,7 +1084,7 @@ contract points are: human-readable cause; `error_kind` carries a stable `snake_case` machine token clients branch on without string-matching the prose (#631); `id` is always present. The token vocabulary is closed and - governed by this contract — adding a token is additive, renaming or + governed by this contract: adding a token is additive, renaming or removing one is a break. The current tokens are: `unsupported_language`, `unknown_field`, `bad_request`, `bad_query`, `invalid_scope_flag`, `vcs_mode_conflict`, `payload_too_large`, @@ -1111,7 +1111,7 @@ contract points are: and `points` to 12 (formerly hard-required). An explicit `top: 0` / `top_deltas: 0` still returns all (the #602 `0 = all` escape). Aligning these from the former unbounded "all" default is a behavioural - `2.0`-line break — payloads omitting these fields get a smaller result. + `2.0`-line break: payloads omitting these fields get a smaller result. - **Introspection.** `GET /v1/version` reports the server and library versions; `GET /v1/languages` reports the supported languages and their extensions, sourced from the `LANG` table (not @@ -1121,15 +1121,15 @@ contract points are: - **Unprefixed aliases removed at 2.0.** The original unprefixed paths (`/metrics`, `/comment`, `/function`, `/ast`, `/ping`, `/version`, `/languages`, the bare `/` index, and the `/vcs*` routes) were - removed at the 2.0 release cut (#517 / #637) — a deliberate, + removed at the 2.0 release cut (#517 / #637), a deliberate, already-planned break. For one cycle each carried `Deprecation: true`, a `Sunset` HTTP-date, and a `Link rel="successor-version"` naming the canonical `/v1` twin; they now `404` like any other unknown URL. Clients must use the `/v1` prefix. The `{id, language}` envelope on `/function` and `/comment` (and -`/ast` at #654), the uniform error body (now `{error, error_kind, id}` -— #631), the `/metrics` `root` key + `scope` flag and the `*_line` +`/ast` at #654), the uniform error body (now `{error, error_kind, +id}`, #631), the `/metrics` `root` key + `scope` flag and the `*_line` span vocabulary (#638), unknown-field rejection (#633), the CLI-aligned `/vcs`-family defaults (#636), and the removal of the unprefixed aliases (#637) all landed as `2.0`-line breaks. The @@ -1164,8 +1164,8 @@ Every release MUST update [`CHANGELOG.md`][changelog]: - **Minor bumps** list every additive shape change (a new public item, a new `LANG` variant, a new `MetricsError` variant, a new language feature) and every *known* value change. A minor bump - may also carry an **(breaking)** entry *only* for an MSRV bump — - see [MSRV policy](#msrv-policy) — which is a toolchain break, not + may also carry an **(breaking)** entry *only* for an MSRV bump + (see [MSRV policy](#msrv-policy)), which is a toolchain break, not a source-level shape break. Treat that as the single permitted exception; any other `(breaking)` entry in a minor section is a bug. @@ -1175,18 +1175,18 @@ Every release MUST update [`CHANGELOG.md`][changelog]: If you find a value change that was not flagged in the changelog, or a shape break that landed in a patch or minor bump, that is a -bug — please open an issue. +bug; please open an issue. ## On the `3.0` horizon The breaking changes that were once on the `2.0` horizon have now shipped in `2.0.0`. See the `[2.0.0]` entry in -[`CHANGELOG.md`][changelog] for the full list — the +[`CHANGELOG.md`][changelog] for the full list (the `#[non_exhaustive]` markers on the open public enums and the per-metric `Stats` structs, the serialized-key normalization, the integer-metric `u64` shift, the language-dispatch and grammar -defaults, the Python and REST surface changes — and for the +defaults, the Python and REST surface changes) and for the consolidated metric-value re-baseline that folds in the drift accumulated since `1.0`. diff --git a/big-code-analysis-book/src/commands/README.md b/big-code-analysis-book/src/commands/README.md index 8166783c..c1d9200d 100644 --- a/big-code-analysis-book/src/commands/README.md +++ b/big-code-analysis-book/src/commands/README.md @@ -48,7 +48,7 @@ scripts can branch on the process status without inspecting output: | `0` | Success. | | `1` | Tool error — a bad flag / threshold / glob spec, unreadable input, or a parse failure. This includes **usage errors** (unknown flag, bad subcommand, a malformed `--threshold` value rejected by clap). **Never** a metric signal. | | `2` | Metric gate: [`check`](check.md) thresholds were exceeded, [`vcs commit --fail-above`](vcs.md) was breached, or [`diff`](../recipes/exporting-data.md) / [`diff-baseline`](../recipes/baselines.md) under `--exit-code` found a non-empty filtered diff. | -| `3`–`5` | [`check --exit-codes=tiered`](check.md#tiered-exit-codes---exit-codestiered) only: tiered violation severity (new-only / regression-only / mixed / hard-breach). | +| `3`–`5` | [`check --exit-codes=tiered`](check.md#tiered-exit-codes---exit-codestiered) only: tiered violation severity (regression-only / mixed / hard-breach; in tiered mode code `2` means new-only). | Codes `2`–`5` are gate signals, emitted only by [`check`](check.md), [`vcs commit --fail-above`](vcs.md), and `diff` / `diff-baseline` under @@ -62,9 +62,11 @@ gate band — CI can always distinguish "the gate found a regression" ## Flag placement and input paths -Every subcommand reads the input it analyzes as a trailing positional +Most subcommands read the input they analyze as a trailing positional path, so the common case reads like every other code tool -(`tokei`, `cloc`, `scc`, `rg`): +(`tokei`, `cloc`, `scc`, `rg`). The exceptions: `report` and `vcs` +select input with `--paths`, `diff` compares two result sets, and +`init` targets a directory via `--dir`. ```bash bca metrics src/ # analyze the src/ tree diff --git a/big-code-analysis-book/src/commands/metrics.md b/big-code-analysis-book/src/commands/metrics.md index 965f4c6d..56b7db7d 100644 --- a/big-code-analysis-book/src/commands/metrics.md +++ b/big-code-analysis-book/src/commands/metrics.md @@ -79,7 +79,7 @@ bca metrics --paths /path/to/your/file/or/directory \ that default explicitly (for example to override a `bca.toml` that set a structured format). The structured per-file serializers are `cbor`, `csv`, `json`, `toml`, and `yaml`. `--output-format` is accepted as a - deprecated alias and will be removed in 2.0. + deprecated alias and is slated for removal in the next major. - `-o, --output`: a single file holding one aggregate document for the whole run — a top-level array of the per-file results (TOML wraps the array under a `files` key; CSV concatenates each file's rows). If diff --git a/big-code-analysis-book/src/commands/nodes.md b/big-code-analysis-book/src/commands/nodes.md index 6af4576a..129baa16 100644 --- a/big-code-analysis-book/src/commands/nodes.md +++ b/big-code-analysis-book/src/commands/nodes.md @@ -63,7 +63,7 @@ bca dump --line-start 5 --line-end 10 /path/to/your/file/or/directory These flags are specific to `dump` and `find`, so they must follow the subcommand. The short `--ls` / `--le` spellings still work as -deprecated aliases but are slated for removal in 2.0. +deprecated aliases but are slated for removal in the next major. ## Listing functions diff --git a/big-code-analysis-book/src/commands/report.md b/big-code-analysis-book/src/commands/report.md index d9b53798..acd5b3ae 100644 --- a/big-code-analysis-book/src/commands/report.md +++ b/big-code-analysis-book/src/commands/report.md @@ -6,8 +6,8 @@ requests, wikis, or issue trackers. Pick the format with `--format` / `-O` (`bca report --format html`). When omitted, the report defaults to `markdown`. The bare positional -form (`bca report markdown`) still works as a deprecated alias and will -be removed in 2.0; prefer `--format`. +form (`bca report markdown`) still works as a deprecated alias and is +slated for removal in the next major; prefer `--format`. > **CI integration.** For runnable GitHub Actions and GitLab CI > recipes that post the Markdown report as a PR/MR comment, see the @@ -125,8 +125,9 @@ alongside `SLOC` so two complementary size proxies are visible per row. clamps to 0 still contributes its true (often negative) maintainability rather than a misleading 0. - *Actionable Summary* — counts of functions exceeding common - thresholds (CC > 10, cognitive > 15, SLOC > 100, args > 3, - Halstead bugs > 1). Emitted **first**, directly after the Summary + thresholds (by default CC > 10, cognitive > 15, SLOC > 100, + args > 3, Halstead bugs > 1; a manifest `[thresholds]` table + overrides each cutoff). Emitted **first**, directly after the Summary and before any hotspot table, so a reader who stops after a table or two still sees the highest-altitude counts. These are **raw** counts that ignore suppression; the section is captioned to say diff --git a/big-code-analysis-book/src/commands/vcs.md b/big-code-analysis-book/src/commands/vcs.md index 945d0db4..40160937 100644 --- a/big-code-analysis-book/src/commands/vcs.md +++ b/big-code-analysis-book/src/commands/vcs.md @@ -26,8 +26,9 @@ Change-history risk (long window 365d, recent 90d, formula v2) With no `--format`, a human-readable ranked table is printed. Pass `--format markdown|html` for a rendered report page, or `--format json|yaml|toml|cbor|csv` for structured output. Unlike -`bca metrics` / `bca ops` (whose `--output` is a *directory* of per-file -emissions), a change-history report is a single whole-repo document, so +`bca metrics` / `bca ops` (whose `--output-dir` is a *directory* of +per-file emissions), a change-history report is a single whole-repo +document, so `bca vcs --output ` writes **one file** (CBOR, being binary, requires `--output`). The global `--paths` / `--include` / `--exclude` / `--no-ignore` filters are reused to pick which tracked files to report. diff --git a/big-code-analysis-book/src/developers/update-grammars.md b/big-code-analysis-book/src/developers/update-grammars.md index 7dc0bf8d..8aac180f 100644 --- a/big-code-analysis-book/src/developers/update-grammars.md +++ b/big-code-analysis-book/src/developers/update-grammars.md @@ -106,6 +106,12 @@ For `tree-sitter-mozjs`, run ./generate-grammars/generate-mozjs.sh ``` +`tree-sitter-tcl`, the fifth vendored grammar, has no regeneration +script: it vendors pre-generated parser sources only (no +`grammar.js`), so updating it means re-vendoring the generated `src/` +from its upstream project rather than running `tree-sitter generate` +locally. + Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems introduced by changes in the grammars. diff --git a/big-code-analysis-book/src/library/cargo-features.md b/big-code-analysis-book/src/library/cargo-features.md index 19c4f084..24c290ba 100644 --- a/big-code-analysis-book/src/library/cargo-features.md +++ b/big-code-analysis-book/src/library/cargo-features.md @@ -6,7 +6,7 @@ the default ```toml [dependencies] -big-code-analysis = "2.0.1" +big-code-analysis = "2.0.0" ``` pulls every grammar in — matching the library's historical @@ -24,7 +24,7 @@ A downstream service that only analyses Rust and TypeScript: ```toml [dependencies] -big-code-analysis = { version = "2.0.1", default-features = false, features = ["rust", "typescript"] } +big-code-analysis = { version = "2.0.0", default-features = false, features = ["rust", "typescript"] } ``` The library still compiles, the `LANG` enum still has every @@ -86,8 +86,8 @@ disabled state as `Err(MetricsError::LanguageDisabled(LANG))`: - `Ast::parse` / `Ast::from_tree_sitter` (and the `metrics` / `ops` methods on the returned `Ast`) - [`LANG::tree_sitter_language`] — this returns - `Result` (changed in 0.0.26) - rather than the previous `Language` + `Result` rather than the bare + `Language` the upstream project returned Callers can query the compiled-in set without going through a dispatcher: diff --git a/big-code-analysis-book/src/library/error-handling.md b/big-code-analysis-book/src/library/error-handling.md index dc005a2b..21f7c071 100644 --- a/big-code-analysis-book/src/library/error-handling.md +++ b/big-code-analysis-book/src/library/error-handling.md @@ -75,8 +75,9 @@ tree. That means: If you need to know whether the input parsed cleanly, count `ERROR` nodes by walking the tree-sitter AST yourself (see the [`Node`][Node] escape hatch in -[`STABILITY.md`](https://github.com/dekobon/big-code-analysis/blob/main/STABILITY.md#escape-hatches)) or use the -[`bca nodes`](../commands/nodes.md) subcommand on the CLI side. +[`STABILITY.md`](https://github.com/dekobon/big-code-analysis/blob/main/STABILITY.md#escape-hatches)) or use +`bca find -t ERROR` on the CLI side (see the +[Nodes](../commands/nodes.md) page). ## Bubbling `MetricsError` through `?` diff --git a/big-code-analysis-book/src/library/in-memory.md b/big-code-analysis-book/src/library/in-memory.md index a0392618..a1eb946b 100644 --- a/big-code-analysis-book/src/library/in-memory.md +++ b/big-code-analysis-book/src/library/in-memory.md @@ -21,7 +21,7 @@ This is useful for: ```rust use big_code_analysis::{analyze, MetricsOptions, Source, LANG}; -fn analyze_buffer(source: &[u8]) -> Option { +fn analyze_buffer(source: &[u8]) -> Option { // `Source::name` is the display identifier baked into the // top-level `FuncSpace`. Pick whatever is meaningful for // downstream consumers (logs, JSON output); pass `None` if diff --git a/big-code-analysis-book/src/library/parse-once.md b/big-code-analysis-book/src/library/parse-once.md index f98571cc..0eb0d94f 100644 --- a/big-code-analysis-book/src/library/parse-once.md +++ b/big-code-analysis-book/src/library/parse-once.md @@ -6,7 +6,7 @@ multiple times — different metric subsets, an interleaved custom tree-sitter walk, or a metric re-run after a configuration change — that re-parse is wasted work. -The [`Ast`][ast] type, added in `0.0.26`, exposes the seam: +The [`Ast`][ast] type, added in `1.1.0`, exposes the seam: parse the source once, then call [`Ast::metrics`][ast_metrics] as many times as you need against the held parse. diff --git a/big-code-analysis-book/src/library/quick-start.md b/big-code-analysis-book/src/library/quick-start.md index 5ca00d05..3006da1b 100644 --- a/big-code-analysis-book/src/library/quick-start.md +++ b/big-code-analysis-book/src/library/quick-start.md @@ -8,7 +8,7 @@ metrics from a string of source code. ```toml # Cargo.toml [dependencies] -big-code-analysis = "2.0.1" +big-code-analysis = "2.0.0" ``` The crate uses Rust edition 2024 and pins `rust-version = "1.94"`. @@ -46,11 +46,13 @@ fn main() { `Source::name` ends up as the top-level [`FuncSpace::name`]; passing `None` leaves the top-level name unset. The return type is -[`Result`][MetricsError]. The `Err` variant -tells parse-failure apart from empty-input apart from disabled- -language; see [Error handling](error-handling.md) for the variant -set and matching patterns. `MetricsError` is `#[non_exhaustive]`, so -always include a `_` arm when matching. +[`Result`][MetricsError]. In practice the +`Err` variant means the requested language's Cargo feature is +disabled in this build; a parse failure does **not** produce `Err` +(tree-sitter recovers with `ERROR` nodes). See +[Error handling](error-handling.md) for the variant set and matching +patterns. `MetricsError` is `#[non_exhaustive]`, so always include a +`_` arm when matching. Tip: `use big_code_analysis::prelude::*;` brings the recommended entry points (`analyze`, `Ast`, `Source`, `MetricsOptions`, diff --git a/big-code-analysis-book/src/library/reuse-tree.md b/big-code-analysis-book/src/library/reuse-tree.md index 7f9ed68b..da7d41c5 100644 --- a/big-code-analysis-book/src/library/reuse-tree.md +++ b/big-code-analysis-book/src/library/reuse-tree.md @@ -59,7 +59,9 @@ let source = source_code.as_bytes().to_vec(); // matches the one the metric walker was compiled against. let mut parser = tree_sitter::Parser::new(); parser - .set_language(&LANG::Rust.tree_sitter_language()) + .set_language( + &LANG::Rust.tree_sitter_language().expect("rust feature enabled"), + ) .expect("rust grammar pinned to a compatible version"); let tree = parser .parse(&source, None) diff --git a/big-code-analysis-book/src/library/stability.md b/big-code-analysis-book/src/library/stability.md index 6ff48961..b52890a4 100644 --- a/big-code-analysis-book/src/library/stability.md +++ b/big-code-analysis-book/src/library/stability.md @@ -1,6 +1,6 @@ # Stability and versioning -`big-code-analysis` is on the `2.x` line (currently `2.0.1`). The +`big-code-analysis` is on the `2.x` line (currently `2.0.0`). The full stability contract lives in [`STABILITY.md`][stability] at the root of the repository — that file is the source of truth and is updated alongside the changelog at every release. @@ -22,7 +22,7 @@ The headlines for library consumers: bump or a bug fix in a metric definition can shift any metric value on any file in any direction, even across a patch bump. Each such drift is flagged in the changelog. Pin to an exact - version (`big-code-analysis = "= 2.0.1"`) if you need bit-for-bit + version (`big-code-analysis = "= 2.0.0"`) if you need bit-for-bit reproducibility across runs. - **MSRV is `1.94`.** Bumping the MSRV is treated as a minor-bump event and is flagged in the changelog under **(breaking)** — diff --git a/big-code-analysis-book/src/library/walking-funcspace.md b/big-code-analysis-book/src/library/walking-funcspace.md index 0ee6044d..3f5ecd55 100644 --- a/big-code-analysis-book/src/library/walking-funcspace.md +++ b/big-code-analysis-book/src/library/walking-funcspace.md @@ -34,16 +34,15 @@ use big_code_analysis::{ analyze, FuncSpace, MetricsOptions, SpaceKind, Source, LANG, }; -fn hotspots(space: &FuncSpace, threshold: f64, out: &mut Vec) { +fn hotspots(space: &FuncSpace, threshold: u64, out: &mut Vec) { if space.kind == SpaceKind::Function && space.metrics.cognitive.cognitive_sum() > threshold + && let Some(name) = &space.name { - if let Some(name) = &space.name { - out.push(format!( - "{name} (lines {}–{})", - space.start_line, space.end_line, - )); - } + out.push(format!( + "{name} (lines {}–{})", + space.start_line, space.end_line, + )); } for child in &space.spaces { hotspots(child, threshold, out); @@ -64,7 +63,7 @@ fn hard(x: i32) -> i32 { .expect("parses"); let mut hits = Vec::new(); - hotspots(&space, 2.0, &mut hits); + hotspots(&space, 2, &mut hits); for hit in hits { println!("{hit}"); } diff --git a/big-code-analysis-book/src/metrics-vcs.md b/big-code-analysis-book/src/metrics-vcs.md index c7a04e6e..474e1524 100644 --- a/big-code-analysis-book/src/metrics-vcs.md +++ b/big-code-analysis-book/src/metrics-vcs.md @@ -417,7 +417,7 @@ The **hotspot score** is the product of a file's complexity and its recent churn: `hotspot_score = cyclomatic_sum × churn_recent`. It is an `Option`, present only when an AST complexity figure is computed alongside the history (for example `bca metrics --metrics -cyclomatic,vcs`), because it needs both halves. +cyclomatic --vcs`), because it needs both halves. The metric is the central idea of Adam Tornhill's 2015 book [*Your Code as a Crime @@ -450,7 +450,8 @@ whose departure would leave more than half of a directory's files without a knowledgeable maintainer. The broader concept has a [Wikipedia summary](https://en.wikipedia.org/wiki/Bus_factor); big-code-analysis emits it as a `vcs_aggregate` object covering the -whole repository and each top-level directory. +whole repository, each top-level directory, and each of its immediate +subdirectories. The estimation method is Guilherme Avelino, Leonardo Passos, Andre Hora, and Marco Tulio Valente's 2016 ICPC paper [*A Novel Approach for diff --git a/big-code-analysis-book/src/metrics.md b/big-code-analysis-book/src/metrics.md index 6b3a059c..8d330c68 100644 --- a/big-code-analysis-book/src/metrics.md +++ b/big-code-analysis-book/src/metrics.md @@ -22,8 +22,7 @@ A few framing notes before we start: method*, per *class or unit-like space*, and per *file*. The underlying tree-sitter parser produces a tree of "spaces" (functions, closures, classes, namespaces, …) and every metric is rolled up - through that tree. See the [Supported Languages](./languages.md) - chapter for which scopes apply to which languages. + through that tree. - **Object-oriented metrics only fire on object-oriented constructs.** WMC, NPA, and NPM report `0` on a Rust file that has no `impl` blocks or on a Python module without classes; that is the correct @@ -931,8 +930,8 @@ serialised output is three fields: `class_wmc_sum` (sum of WMC across all classes in the file), `interface_wmc_sum` (sum across interfaces), and `total` (the two combined). No min/max/average aggregation is emitted at the file scope — to rank individual classes by WMC, use -the report subcommand, which surfaces a *WMC hotspots* section -(see [Commands → Report](./commands/report.md)). +the report subcommand, which surfaces a *Type hotspots (top N by +WMC)* section (see [Commands → Report](./commands/report.md)). ### How to read it @@ -958,10 +957,10 @@ gargantuan methods (split the methods, not the class). A class with ## Where to go next -- The [Supported Languages](./languages.md) chapter lists which - metrics fire for which languages — language coverage varies - because some metric definitions (`NPA`, `NPM`, `WMC`) only make - sense in languages with classes. +- The [Supported Languages](./languages.md) chapter lists every + supported language and grammar. Metric coverage varies by + language because some metric definitions (`NPA`, `NPM`, `WMC`) + only make sense in languages with classes. - The [Supported Change-history (VCS) Metrics](./metrics-vcs.md) chapter covers the complementary family derived from version-control history — commit frequency, churn, ownership, and the composite risk diff --git a/big-code-analysis-book/src/migration.md b/big-code-analysis-book/src/migration.md index 6640f34b..8d2e8f62 100644 --- a/big-code-analysis-book/src/migration.md +++ b/big-code-analysis-book/src/migration.md @@ -56,7 +56,7 @@ big-code-analysis-cli \ bca \ report \ --paths "$PWD" \ - markdown \ + --format markdown \ --top 20 \ --strip-prefix "$PWD/" ``` @@ -152,7 +152,7 @@ For example: $ bca --metrics -O markdown note: the CLI was restructured into subcommands. See migration.md for the full mapping. --metrics -> bca metrics - -O markdown -> bca report markdown [--top N] [--strip-prefix P] + -O markdown -> bca report markdown|html [--top N] [--strip-prefix P] Run `bca --help` for the new command list. error: unexpected argument '--metrics' found diff --git a/big-code-analysis-book/src/python/README.md b/big-code-analysis-book/src/python/README.md index 10b8c026..7e8fffb3 100644 --- a/big-code-analysis-book/src/python/README.md +++ b/big-code-analysis-book/src/python/README.md @@ -49,8 +49,12 @@ option. feeding sqlite / pandas. * [Metric selection](metrics.md) — `metrics=` kwarg, `bca.METRIC_NAMES`, dependency-pull semantics. +* [AST traversal](traversal.md) — `Ast`, `Node`, `walk()`, and + `find()` over the held parse. * [SARIF output](sarif.md) — `to_sarif` + GitHub Code Scanning upload. +* [Change-history (VCS) metrics](vcs.md) — `vcs.rank`, `vcs.trend`, + `vcs.commit`, and `vcs.score_diff` over a git working tree. * [Error handling](errors.md) — the full exception taxonomy and the never-raise batch contract. * [Async patterns](async.md) — `asyncio.to_thread` is the diff --git a/big-code-analysis-book/src/python/errors.md b/big-code-analysis-book/src/python/errors.md index d3430a56..7145a2ff 100644 --- a/big-code-analysis-book/src/python/errors.md +++ b/big-code-analysis-book/src/python/errors.md @@ -171,7 +171,10 @@ import big_code_analysis as bca log = logging.getLogger(__name__) def report(paths: list[str]) -> None: - for path, slot in zip(paths, bca.analyze_batch(paths)): + # skip_generated=False keeps the result list index-aligned with + # `paths`; with the default True, a generated file yields no slot + # and the zip silently misaligns. + for path, slot in zip(paths, bca.analyze_batch(paths, skip_generated=False)): if isinstance(slot, bca.AnalysisFailure): log.warning( "skip %s (%s): %s", path, slot.error_kind, slot.error diff --git a/big-code-analysis-book/src/python/metrics.md b/big-code-analysis-book/src/python/metrics.md index a334d21a..6ec50e51 100644 --- a/big-code-analysis-book/src/python/metrics.md +++ b/big-code-analysis-book/src/python/metrics.md @@ -36,7 +36,7 @@ selection = [MetricName.CYCLOMATIC, "cognitive"] The members are generated from the same `Metric` table the CLI and JSON output use, so the values never drift from the slugs you see in -`bca metrics --output-format json`. +`bca metrics --format json`. Names are case-sensitive lowercase; passing an unknown name raises `ValueError` with the canonical list in the message. The diff --git a/big-code-analysis-book/src/python/traversal.md b/big-code-analysis-book/src/python/traversal.md index 7596fc18..0bb9003f 100644 --- a/big-code-analysis-book/src/python/traversal.md +++ b/big-code-analysis-book/src/python/traversal.md @@ -54,7 +54,7 @@ variants), `child(i)` / `named_child(i)`, `walk()` is a lazy pre-order iterator over a node and its descendants; `descendants_by_kind(kinds)` collects the matches in one pass; and `ast.find(filters)` searches the whole tree, accepting the -same vocabulary as [`bca count`](../commands/count.md) (`function`, +same vocabulary as [`bca count`](../commands/nodes.md#counting-nodes) (`function`, `call`, `comment`, `string`, an exact kind, …): ```python @@ -129,6 +129,5 @@ are also safe to share across threads. * [Metric selection](metrics.md) — compute only the metrics you need from the same parse. -* The CLI's [`dump`](../commands/dump.md) and - [`count`](../commands/count.md) commands are the shell-level - equivalents of `dump()` and `find()`. +* The CLI's [`dump` and `count`](../commands/nodes.md) commands are + the shell-level equivalents of `dump()` and `find()`. diff --git a/big-code-analysis-book/src/recipes/README.md b/big-code-analysis-book/src/recipes/README.md index c475d2f4..ce635b09 100644 --- a/big-code-analysis-book/src/recipes/README.md +++ b/big-code-analysis-book/src/recipes/README.md @@ -12,6 +12,9 @@ The recipes are grouped by goal: - [CI integration](ci.md) — wire `bca check` and `bca report` into GitHub Actions and GitLab CI, including the baseline / ratchet pattern and the Code Quality widget path. +- [Baselines](baselines.md) — record existing offenders in a + `.bca-baseline.toml` so the gate only fires on new or worsened + violations, and audit or diff that baseline during review. - [Local threshold gates](local-gates.md) — mirror the CI threshold gate on a developer machine with a two-tier (hard + headroom) Makefile / `just` / `pre-commit` pattern, so regressions never diff --git a/big-code-analysis-book/src/recipes/agent-feedback.md b/big-code-analysis-book/src/recipes/agent-feedback.md index 014e956c..a70e6c11 100644 --- a/big-code-analysis-book/src/recipes/agent-feedback.md +++ b/big-code-analysis-book/src/recipes/agent-feedback.md @@ -222,7 +222,7 @@ export const BcaCheck = async ({ $ }) => { .quiet() .nothrow() // 0 clean, 1 tool error: not a complexity issue. `< 2` rather than - // `=== 2` so the tiered exit codes (3-5, from `--strict-exit-codes` + // `=== 2` so the tiered exit codes (3-5, from `--exit-codes=tiered` // / `exit_codes = "tiered"`) still report. if (res.exitCode < 2) return diff --git a/big-code-analysis-book/src/recipes/ast-queries.md b/big-code-analysis-book/src/recipes/ast-queries.md index cb8eb4a3..3ed065b8 100644 --- a/big-code-analysis-book/src/recipes/ast-queries.md +++ b/big-code-analysis-book/src/recipes/ast-queries.md @@ -21,10 +21,11 @@ bca find \ -t ERROR ``` -> **Flag ordering.** `--include` and `--exclude` are variadic and -> consume tokens until the next flag begins, so put them **before** -> `--paths` to avoid the subcommand name being eaten as a glob. The -> single-value `=` form (`--include="*.rs"`) also works. +> **One glob per occurrence.** `--include` and `--exclude` take +> exactly one value each time they appear; repeat the flag for +> multiple globs (`--include "*.rs" --include "*.py"`). A positional +> path that follows is never swallowed. The `=` form +> (`--include="*.rs"`) also works. A clean run prints nothing. Wire this into a pre-commit hook to fail fast when a syntactically broken file is staged. @@ -85,8 +86,8 @@ bca find \ --line-start 42 --line-end 88 -t return_expression ``` -The short `--ls` / `--le` spellings remain as deprecated aliases for one -release cycle and are slated for removal in 2.0. +The short `--ls` / `--le` spellings remain as deprecated aliases and +are slated for removal in the next major. ## List every function or method diff --git a/big-code-analysis-book/src/recipes/baselines.md b/big-code-analysis-book/src/recipes/baselines.md index 8abbab17..d2450e6b 100644 --- a/big-code-analysis-book/src/recipes/baselines.md +++ b/big-code-analysis-book/src/recipes/baselines.md @@ -150,8 +150,9 @@ For a PR bot, `bca diff-baseline --format markdown` emits a fenced block ready to drop into a sticky comment, and the `--worsened-only` / `--added-only` filters narrow it to just the regressions reviewers must look at. `--format json` feeds the same -diff to other tooling. The command always exits 0 — it informs review, -it does not gate; the gate is `bca check` itself. +diff to other tooling. The command exits 0 by default — it informs +review, it does not gate (the gate is `bca check` itself) — though +the opt-in `--exit-code` flag exits 2 on a non-empty filtered diff. ### Reading the gate output @@ -353,8 +354,9 @@ bca exemptions --paths src/ ... ``` -The baseline section reads the same `--baseline` / `bca.toml` top-level -`baseline` source `bca check` does (or `.bca-baseline.toml` by default). +The baseline section reads the same `--baseline` / `bca.toml` +`[check] baseline` source `bca check` does (or `.bca-baseline.toml` +by default). Use `--baseline-only` to list just the baselined offenders, `--format markdown` for a PR comment, or `--format json` for dashboards. During PR review, pair it with `bca diff-baseline ` (above): the diff --git a/big-code-analysis-book/src/recipes/ci.md b/big-code-analysis-book/src/recipes/ci.md index 5b994d84..de99c617 100644 --- a/big-code-analysis-book/src/recipes/ci.md +++ b/big-code-analysis-book/src/recipes/ci.md @@ -22,7 +22,7 @@ runnable example. | Code Scanning alerts (GitHub) | `bca check … --report-format sarif --no-fail` + `github/codeql-action/upload-sarif` | | Merge-request widget (GitLab Code Quality) | `bca check … --report-format code-climate --no-fail` | | Jenkins / SonarQube ingestion | `bca check … --report-format checkstyle` | -| Human-readable PR/MR comment or downloadable | `bca report markdown --top 20 --strip-prefix "$PWD/"` | +| Human-readable PR/MR comment or downloadable | `bca report -O markdown --top 20 --strip-prefix "$PWD/"` | | Machine-readable artifact for dashboards | `bca metrics --format json --output-dir ./out` | *(‡) Recommended adoption path when introducing thresholds on a @@ -103,11 +103,11 @@ so a green-path rerun skips the download entirely: ```yaml env: - BCA_VERSION: "2.0.1" + BCA_VERSION: "2.0.0" BCA_TARGET: "x86_64-unknown-linux-gnu" # sha256 of big-code-analysis-${BCA_VERSION}-${BCA_TARGET}.tar.gz from the # release's SHA256SUMS file. Bump together with BCA_VERSION. - BCA_SHA256: "f11c324fd80787e1a9edf99d3c1763980e035e51abb5479527b14b1e2f83e919" + BCA_SHA256: "a205fff13108d0f8c679a062e352ba8468109c4adfdd8c9e3567cf5fcc99c3d5" steps: # Cache key MUST include BCA_SHA256 (and BCA_TARGET). Without the @@ -176,7 +176,7 @@ default for downstream adopters. - name: Install bca uses: taiki-e/install-action@v2 with: - tool: big-code-analysis-cli@2.0.1 + tool: big-code-analysis-cli@2.0.0 ``` ```yaml @@ -184,7 +184,7 @@ default for downstream adopters. - name: Install cargo-binstall uses: cargo-bins/cargo-binstall@main - name: Install bca - run: cargo binstall --no-confirm big-code-analysis-cli --version 2.0.1 + run: cargo binstall --no-confirm big-code-analysis-cli --version 2.0.0 ``` If either action falls back to compilation, cache the cargo registry + @@ -233,11 +233,11 @@ jobs: - name: Install bca uses: taiki-e/install-action@v2 with: - tool: big-code-analysis-cli@2.0.1 + tool: big-code-analysis-cli@2.0.0 - name: Generate report run: | bca \ - report markdown \ + report -O markdown \ --paths "$PWD" \ --top 20 \ --strip-prefix "$PWD/" \ @@ -429,7 +429,7 @@ table, a per-metric count breakdown, and the top-10 offenders by suppresses the digest even when `$GITHUB_STEP_SUMMARY` is set. The digest is bracketed by HTML-comment markers (`` -/ ``) so a retried step replaces (not +/ ``) so a retried step replaces (not stacks) the previous block — three retries converge to exactly one up-to-date digest. Content outside the markers (e.g. summaries written by other tools earlier in the same step) is preserved. @@ -622,11 +622,12 @@ Both tiers consume the same `bca.toml` thresholds and the same with every threshold value multiplied by `BCA_HEADROOM`. Both exit `0` clean, `2` on any threshold violation, `1` on tool error — the soft tier is a real gate, not advisory, so do not -wrap `make self-scan-headroom` in `|| true`. All four targets -are wired into `make pre-commit`, `make ci`, and -`.pre-commit-config.yaml`, with `self-scan-headroom: self-scan` -as a Make prerequisite so the hard tier always reports a true -regression before the soft tier reports near-limit headroom. +wrap `make self-scan-headroom` in `|| true`. The two gate targets +(`self-scan`, `self-scan-headroom`) are wired into `make +pre-commit`, `make ci`, and `.pre-commit-config.yaml`; those chains +run the hard tier before the soft tier, so a true regression always +reports before near-limit headroom. The two `write-baseline` +targets are side-effecting and deliberately not wired in. `BCA_HEADROOM=0.90 make self-scan-headroom` widens the band; `BCA_HEADROOM=0.99` tightens it to the last 1%. When the soft @@ -662,11 +663,11 @@ stages: - quality variables: - BCA_VERSION: "2.0.1" # pin a published big-code-analysis-cli release + BCA_VERSION: "2.0.0" # pin a published big-code-analysis-cli release BCA_TARGET: "x86_64-unknown-linux-gnu" # sha256 of big-code-analysis-${BCA_VERSION}-${BCA_TARGET}.tar.gz from # the release's SHA256SUMS file. Bump together with BCA_VERSION. - BCA_SHA256: "f11c324fd80787e1a9edf99d3c1763980e035e51abb5479527b14b1e2f83e919" + BCA_SHA256: "a205fff13108d0f8c679a062e352ba8468109c4adfdd8c9e3567cf5fcc99c3d5" bca-quality: stage: quality @@ -708,7 +709,7 @@ bca-quality: --output bca-checkstyle.xml --no-fail - bca - report markdown + report -O markdown --paths "$PWD" --top 20 --strip-prefix "$PWD/" diff --git a/big-code-analysis-book/src/recipes/exporting-data.md b/big-code-analysis-book/src/recipes/exporting-data.md index 7a8747f0..ee627eb7 100644 --- a/big-code-analysis-book/src/recipes/exporting-data.md +++ b/big-code-analysis-book/src/recipes/exporting-data.md @@ -114,7 +114,9 @@ explicit request: an unresolvable ref, a missing `git`, or a non-git working directory is a hard error (exit 1). `--since` takes at most one positional (the after side); passing two is an error. -`bca diff` always exits 0 on success — it is informational, not a gate. +`bca diff` exits 0 on success — it is informational, not a gate — +unless the opt-in `--exit-code` flag is passed, which exits 2 when +the filtered diff is non-empty. It replaces the former `json-minimal-tests` + `split-minimal-tests.py` chain used to validate that a grammar bump did not regress metrics; the `check-grammar-crate.py` helper now calls `bca diff` internally. @@ -159,10 +161,11 @@ bca ops \ --output-dir /tmp/ops ``` -> **Flag ordering.** Variadic flags like `--include` and `--exclude` -> consume tokens until the next flag, so put them before `--paths` -> (or use the `--include=GLOB` single-value form) to keep the -> subcommand from being eaten as a glob. +> **One glob per occurrence.** `--include` and `--exclude` take +> exactly one value each time they appear; repeat the flag for +> multiple globs (`--include "*.rs" --include "*.py"`). A positional +> path that follows is never swallowed. The `=` form +> (`--include="*.rs"`) also works. Each output file mirrors the input path under `/tmp/ops/`. diff --git a/big-code-analysis-book/src/recipes/quality-reports.md b/big-code-analysis-book/src/recipes/quality-reports.md index 19c6598a..42b902bf 100644 --- a/big-code-analysis-book/src/recipes/quality-reports.md +++ b/big-code-analysis-book/src/recipes/quality-reports.md @@ -10,8 +10,9 @@ Recipes for producing aggregated, human-readable Markdown reports. ## Live example reports -`big-code-analysis` publishes the output of `bca report markdown --vcs` -and `bca report html --vcs` against its own source tree on every push to +`big-code-analysis` publishes the output of `bca report -O markdown +--vcs` and `bca report -O html --vcs` against its own source tree on +every push to `main`. Open either to see exactly what the recipes on this page produce on a multi-language Rust + Python codebase: @@ -37,7 +38,7 @@ Run from the project root and write the report to a file: ```bash bca report \ --paths "$PWD" \ - markdown \ + -O markdown \ --top 20 \ --strip-prefix "$PWD/" \ --output report.md @@ -62,7 +63,7 @@ include/exclude globs do the filtering: bca report \ --include "*.rs" --include "*.py" \ --paths "$PWD" \ - markdown --output report.md + -O markdown --output report.md ``` To exclude vendored or generated trees, layer in `--exclude`: @@ -72,7 +73,7 @@ bca report \ --include "*.rs" \ --exclude "**/target/**" --exclude "**/vendor/**" \ --paths "$PWD" \ - markdown + -O markdown ``` > **Flag arity.** `--include` and `--exclude` take exactly one glob per @@ -94,7 +95,7 @@ values; blank lines and `#`-prefixed comments are skipped: bca report \ --paths . \ --exclude-from .bcaignore \ - markdown --output report.md + -O markdown --output report.md ``` ## Show only the worst offenders @@ -103,7 +104,7 @@ For a quick triage view that highlights the top three problems per section: ```bash -bca report -p src/ markdown --top 3 +bca report -p src/ -O markdown --top 3 ``` The report still includes every section, but each table is short @@ -116,10 +117,10 @@ on each side and diff the Markdown: ```bash git worktree add /tmp/before main -bca report -p /tmp/before markdown \ +bca report -p /tmp/before -O markdown \ --strip-prefix /tmp/before/ --output /tmp/before.md -bca report -p "$PWD" markdown \ +bca report -p "$PWD" -O markdown \ --strip-prefix "$PWD/" --output /tmp/after.md diff -u /tmp/before.md /tmp/after.md | less @@ -145,12 +146,14 @@ bca preproc \ bca report \ --paths src/ \ --preproc-data /tmp/preproc.json \ - markdown --output report.md + -O markdown --output report.md ``` -`--preproc-data` is a global flag, so it works with `metrics`, `ops`, -`functions`, and the other subcommands as well — anywhere accurate -C/C++ analysis matters. +`--preproc-data` is accepted by every metric-computing walking +subcommand (`metrics`, `ops`, `functions`, `report`, `check`, …) — +anywhere accurate C/C++ analysis matters. Subcommands that do not +consume it (`vcs`, `preproc`, `list-metrics`, `diff-baseline`) +reject it as a usage error. ## Analyze only files changed in a PR @@ -176,7 +179,7 @@ pipeline: ```bash git diff --name-only --diff-filter=AM origin/main...HEAD \ - | bca report --paths-from - markdown \ + | bca report --paths-from - -O markdown \ --top 10 --output pr-report.md ``` diff --git a/big-code-analysis-cli/src/cli_args/analyze.rs b/big-code-analysis-cli/src/cli_args/analyze.rs index 9d196131..7cc01932 100644 --- a/big-code-analysis-cli/src/cli_args/analyze.rs +++ b/big-code-analysis-cli/src/cli_args/analyze.rs @@ -25,7 +25,7 @@ pub(crate) struct StructuredArgs { /// that set a structured format). `json` / `yaml` / `toml` / `cbor` / /// `csv` emit structured per-file data. `--output-format` is accepted /// as a deprecated alias; it is hidden from help and slated for - /// removal in 2.0. + /// removal in the next major. // The `--output-format` alias is the pre-rename spelling from issue #513. #[clap( long = "format", @@ -146,7 +146,8 @@ pub(crate) struct NodeTypesArgs { /// names cluttered all of their help. The descriptive `--line-start` / /// `--line-end` are canonical; `--ls` / `--le` survive as hidden /// deprecated aliases for one release cycle (mirrors the #513 -/// `--output-format` deprecation) and are slated for removal in 2.0. +/// `--output-format` deprecation) and are slated for removal in the +/// next major. #[derive(Args, Debug, Default)] pub(crate) struct LineRange { /// First line of the range to analyze (1-based, inclusive). diff --git a/big-code-analysis-cli/src/cli_args/check.rs b/big-code-analysis-cli/src/cli_args/check.rs index 0625a0c3..41bf1798 100644 --- a/big-code-analysis-cli/src/cli_args/check.rs +++ b/big-code-analysis-cli/src/cli_args/check.rs @@ -79,7 +79,7 @@ CLI `--threshold` flags override values read from this file." /// extension (`.sarif` → sarif, `.xml` → checkstyle); an extension /// with no unique dialect is a usage error. The old `--format` / `-O` /// / `--output-format` spellings stay hidden aliases for one release - /// cycle and are slated for removal in 2.0. + /// cycle and are slated for removal in the next major. // `--report-format` split from the data `--format` in issue #659; the // `--format`/`-O`/`--output-format` aliases trace to issues #513/#659. #[clap( diff --git a/big-code-analysis-cli/src/cli_args/preproc.rs b/big-code-analysis-cli/src/cli_args/preproc.rs index 3f606fce..af501bc5 100644 --- a/big-code-analysis-cli/src/cli_args/preproc.rs +++ b/big-code-analysis-cli/src/cli_args/preproc.rs @@ -23,7 +23,8 @@ pub(crate) struct ReportArgs { pub(crate) format: Option, /// Deprecated positional form of the report format, kept working /// for one release cycle. Hidden from help; use `--format`/`-O` - /// instead. The flag wins when both are given. To be removed in 2.0. + /// instead. The flag wins when both are given. To be removed in + /// the next major. // The `--format`/`-O` flag superseded the positional form in issue #513. #[clap(value_enum, hide = true, value_name = "FORMAT")] pub(crate) format_positional: Option, diff --git a/big-code-analysis-py/README.md b/big-code-analysis-py/README.md index b1c44739..022edf99 100644 --- a/big-code-analysis-py/README.md +++ b/big-code-analysis-py/README.md @@ -40,29 +40,34 @@ docs. | `sarif_output.py` | Minimal SARIF rendering. Embedded by *SARIF output*. | | `errors_taxonomy.py` | The full exception map across the entry points. Embedded by *Error handling*. | | `async_patterns.py` | `asyncio.to_thread` (canonical) vs the in-loop anti-pattern. Embedded by *Async patterns*. | -| `cli_parity.py` | Byte-for-byte parity smoke test vs `bca metrics --output-format json`. Wired into `make py-test`. | +| `cli_parity.py` | Byte-for-byte parity smoke test vs `bca metrics --format json`. Wired into `make py-test`. | | `pipeline_db.py` | Directory walk → `analyze_batch` → `flatten_spaces` → sqlite top-N, with a deliberately broken file to exercise the never-raise contract. | | `sarif_upload.py` | SARIF emission tuned for GitHub Code Scanning (`github/codeql-action/upload-sarif@v3`). | | `jupyter_quickstart.ipynb` | Pandas DataFrame + matplotlib `cyclomatic.sum` per function + top-N. Executed in CI via `python-examples-nbconvert`. | ## Installation -The package is not yet published on PyPI. When it is, the -distribution will be **`big-code-analysis`** and the import name -**`big_code_analysis`** — these names are locked by the stability -contract (see [`STABILITY.md`](../STABILITY.md), *Python bindings*) -and cannot change post-publish without breaking every consumer. For -development, build locally via [maturin](https://www.maturin.rs/). +The package is published on PyPI: the distribution is +**`big-code-analysis`** and the import name **`big_code_analysis`** — +names locked by the stability contract (see +[`STABILITY.md`](../STABILITY.md), *Python bindings*). + +```bash +pip install big-code-analysis +``` + +For development, build locally via [maturin](https://www.maturin.rs/). The recommended bootstrap uses [uv](https://docs.astral.sh/uv/) so the resolved dev set matches the checked-in `uv.lock` — every contributor on this path gets the same ruff/mypy/pyright/maturin/pytest versions. CI -does not yet consume `uv.lock` (its `python-*` jobs still pip-install -the pyproject floors directly); convergence is tracked as a follow-up -in the PR-2 plan committed alongside this work: +consumes `uv.lock` through the hash-pinned exports under +`requirements/` (`pip install --require-hashes -r …` in the +workflows), so a `uv.lock` change and its regenerated exports must +land in the same commit: ```bash -make py-bootstrap # runs `uv sync --extra dev` in big-code-analysis-py/ +make py-bootstrap # runs `uv sync --locked --extra dev` in big-code-analysis-py/ # creates .venv with ruff, mypy, pyright, maturin, pytest cd big-code-analysis-py source .venv/bin/activate @@ -84,7 +89,7 @@ is pinned to `uv.lock`. import big_code_analysis as bca # Analyse a file by path. The returned dict matches the JSON -# emitted by `bca metrics --output-format json` for the same +# emitted by `bca metrics --format json` for the same # file at the `FuncSpace` boundary — same field order, same # numeric formatting, same shape. Language detection mirrors the # CLI (path extension, then shebang, then emacs `-*- mode -*-`). @@ -100,7 +105,7 @@ if result is not None: print(result["metrics"]["cognitive"]["sum"]) # Analyse a Rust file with `#[test]` subtrees pruned out — same -# result as `bca metrics --exclude-tests --output-format json`. +# result as `bca metrics --exclude-tests --format json`. prod_only = bca.analyze("src/main.rs", exclude_tests=True) # Non-UTF-8 paths raise `ValueError` by default so the `name` @@ -237,19 +242,20 @@ a well-formed SARIF document with empty `results` and `rules` arrays. This matches the CLI's posture: there are **no built-in default thresholds**; every check run supplies its own limits. -**Unit-level findings.** `to_sarif` emits file-scope (unit-space) -findings for every metric whose JSON headline at the unit space -matches the CLI's per-space accessor (`loc.*`, `halstead.*`, -`mi.*`, `nom`, `nargs`, `nexits`, `tokens`, `abc`, `wmc`, `npm`, -`npa`). The three exceptions — `cyclomatic`, `cyclomatic.modified`, -`cognitive` — are skipped at the unit level because the JSON only -exposes the aggregate `sum` across children while the CLI's -per-space accessor returns just the unit's own scalar; emitting -findings from the aggregate would diverge from the CLI for parent -spaces. Unit findings carry `logicalLocations: [{"fullyQualifiedName": +**Unit-level findings.** `to_sarif` emits a finding at every space +whose **own** value breaches its limit, exactly matching +`bca check --report-format sarif`. Emission is scope-gated per metric +(`loc.*` at the file unit, `nom` / `wmc` / `npm` / `npa` at +containers, the rest at function spaces), and the four +subtree-aggregate metrics (`cyclomatic`, `cyclomatic.modified`, +`cognitive`, `abc`) read the per-space `value` field rather than the +rolled-up aggregate (#958, #969). Unit findings carry +`logicalLocations: [{"fullyQualifiedName": ""}]`; nameless non-unit spaces (rare parse-failure case) carry `""` — both matching the CLI's `function_token` -placeholders. +placeholders. See the book's +[SARIF output](https://dekobon.github.io/big-code-analysis/python/sarif.html) +page for the full contract. ### Upload to GitHub Code Scanning @@ -432,7 +438,8 @@ Exception types raised by `bca.analyze` / `bca.analyze_source`: with `err.errno` and `err.filename` populated. Exception types raised by the change-history surface -(`bca.vcs_metrics` / `bca.vcs_trend` / `bca.vcs_jit`, and +(`bca.vcs.rank` / `bca.vcs.trend` / `bca.vcs.commit` / +`bca.vcs.score_diff`, and `bca.analyze(..., vcs=True)` for its option parsing) — all subclass `bca.VcsError`, itself a `ValueError`, so existing `except ValueError` handlers keep working (#624): @@ -442,8 +449,9 @@ handlers keep working (#624): directory". (`analyze(..., vcs=True)` never raises it — a non-repository file just yields no `vcs` block.) - `bca.InvalidRevisionError` — a `reference` / `commit` could not be - resolved (`vcs_jit`). -- `bca.InvalidDiffError` — the `diff` passed to `vcs_jit` is malformed. + resolved (`vcs.commit`). +- `bca.InvalidDiffError` — the `diff` passed to `vcs.commit` is + malformed. - `bca.VcsEnvironmentError` — an environment / backend failure (opening the repository, walking history, diffing, `.mailmap`, blame, or cache I/O). Mirrors the `500` (rather than `400`) diff --git a/big-code-analysis-py/python/big_code_analysis/_native.pyi b/big-code-analysis-py/python/big_code_analysis/_native.pyi index 49223e8a..de28a565 100644 --- a/big-code-analysis-py/python/big_code_analysis/_native.pyi +++ b/big-code-analysis-py/python/big_code_analysis/_native.pyi @@ -965,15 +965,14 @@ def to_sarif( tool driver name / version, and rule descriptions match the CLI byte-for-byte. - Unit-level (file-scope) findings are emitted for every metric - whose JSON headline at the file-level ``unit`` space matches - the CLI's per-space accessor (``loc.*``, ``halstead.*``, - ``mi.*``, ``nom``, ``nargs``, ``nexits``, ``tokens``, ``abc``, - ``wmc``, ``npm``, ``npa``). For the three metrics whose CLI - per-space accessor returns just the unit's own scalar while - the JSON exposes the aggregate ``sum`` across children — - ``cyclomatic``, ``cyclomatic.modified``, ``cognitive`` — the - unit space is skipped (those metrics emit per-function only). + A finding is emitted at every space whose *own* value breaches + its limit, exactly matching ``bca check --report-format sarif``. + Emission is scope-gated per metric (``loc.*`` at the file unit, + ``nom`` / ``wmc`` / ``npm`` / ``npa`` at containers, the rest at + function spaces), and the four subtree-aggregate metrics + (``cyclomatic``, ``cyclomatic.modified``, ``cognitive``, + ``abc``) read the per-space ``value`` field rather than the + rolled-up aggregate (#958, #969). Unit-level findings carry ``logicalLocations: [{"fullyQualifiedName": ""}]``; nameless non-unit spaces carry ``""`` — matching the CLI's ``function_token`` placeholder. diff --git a/check-versions-test.py b/check-versions-test.py index 7666057f..1b646b4d 100644 --- a/check-versions-test.py +++ b/check-versions-test.py @@ -168,11 +168,20 @@ def test_runtime_version_output_is_not_matched(self) -> None: text = "$ bca --version\nbig-code-analysis-cli 1.1.0\n" self.assertEqual(self._stale_lines(text, "1.2.0"), []) - def test_real_ci_md_pins_are_at_canonical(self) -> None: - canonical = cv.workspace_version(REPO_ROOT) + def test_real_ci_md_pins_are_at_a_published_release(self) -> None: + # ci.md pins track the published releases (latest or the one + # before), never the workspace version — they can only move + # once the release's SHA256SUMS exists. + ci_allowed = cv.released_versions(REPO_ROOT)[:2] for ci_path in cv.CI_RECIPE_FILES: text = cv.read(REPO_ROOT / ci_path) - self.assertEqual(self._stale_lines(text, canonical), [], ci_path) + stale = [ + cited + for m in cv.CI_PIN_RE.finditer(text) + for cited in [next(g for g in m.groups() if g is not None)] + if not cv.matches_any(cited, ci_allowed) + ] + self.assertEqual(stale, [], ci_path) def test_real_ci_md_would_flag_on_bump(self) -> None: # On a hypothetical bump, the real ci.md pins go stale — proving @@ -183,6 +192,42 @@ def test_real_ci_md_would_flag_on_bump(self) -> None: self.assertTrue(stale, f"{ci_path} should expose stale pins on a bump") +class ChangelogReleaseTest(unittest.TestCase): + """Doc pins are checked against CHANGELOG's released sections.""" + + SYNTHETIC = ( + "# Changelog\n\n" + "## [Unreleased]\n\n### Added\n- something\n\n" + "## [2.1.0] - 2026-08-01\n\n### Added\n- thing\n\n" + "## [2.0.0] - 2026-06-29\n\n### Changed\n- other\n\n" + "## [2.0.0-rc1] - 2026-06-19\n\n- rc\n" + ) + + def test_release_headers_parse_newest_first(self) -> None: + found = cv.CHANGELOG_RELEASE_RE.findall(self.SYNTHETIC) + self.assertEqual(found, ["2.1.0", "2.0.0", "2.0.0-rc1"]) + + def test_unreleased_section_is_not_a_release(self) -> None: + found = cv.CHANGELOG_RELEASE_RE.findall("## [Unreleased]\n") + self.assertEqual(found, []) + + def test_real_changelog_has_a_released_section(self) -> None: + released = cv.released_versions(REPO_ROOT) + self.assertTrue(released, "CHANGELOG.md must have a released section") + self.assertRegex(released[0], r"^\d+\.\d+\.\d+") + + def test_ci_pins_may_lag_one_release_but_not_two(self) -> None: + allowed = ["2.1.0", "2.0.0"] + self.assertTrue(cv.matches_any("2.1.0", allowed)) + self.assertTrue(cv.matches_any("2.0.0", allowed)) # lag of one + self.assertFalse(cv.matches_any("1.1.0", allowed)) # lag of two + + def test_doc_pin_prefix_still_normalizes(self) -> None: + # The README's major-line form ("2") must satisfy the latest + # published release via prefix normalization. + self.assertEqual(cv.normalize("2", "2.1.0"), "2.1.0") + + class SmokeTest(unittest.TestCase): def test_clean_repo_reports_lockstep(self) -> None: result = subprocess.run( diff --git a/check-versions.py b/check-versions.py index 047f09e6..96faaa45 100755 --- a/check-versions.py +++ b/check-versions.py @@ -7,8 +7,18 @@ vendored grammar leaves — must share one version number. Every internal-dep pin must reference that same version. -See `RELEASING.md` "Lockstep version policy" for the policy this -enforces. Wired into `make pre-commit` and the CI lint job. +Documentation pins follow a different clock: readers deploy the +latest *published* release, not the workspace version, which runs +ahead of it between releases. Doc snippets are therefore checked +against the topmost released `## [X.Y.Z] - YYYY-MM-DD` section of +`CHANGELOG.md` (which release-prep moves in the same commit as the +doc pins), and the `recipes/ci.md` install pins may additionally +lag one release because they can only move once the release's +`SHA256SUMS` exists. + +See `RELEASING.md` "Lockstep version policy" and "Version strings +in documentation" for the policies this enforces. Wired into +`make pre-commit` and the CI lint job. Exits 0 on lockstep, non-zero with a per-source listing on drift. """ @@ -44,9 +54,13 @@ "big-code-analysis-web/Cargo.toml", ) -# Doc files that hard-code the workspace version in install snippets -# or stability prose. Every plain `X.Y.Z` or `= X.Y.Z` match in these -# files must equal the canonical version. +# Doc files that hard-code a version in install snippets or +# stability prose. Every plain `X.Y.Z` or `= X.Y.Z` match in these +# files must equal the **latest published release** (the topmost +# released section of CHANGELOG.md), not the workspace version: +# readers copy these lines, so they must resolve against the +# registries today. See RELEASING.md "Version strings in +# documentation". DOC_VERSION_FILES = ( "README.md", "STABILITY.md", @@ -63,9 +77,12 @@ # CI-recipe docs that pin a *published* big-code-analysis-cli release in # install snippets. These use install-action / binstall / env-var forms # rather than the ` = "X.Y.Z"` Cargo-snippet shape DOC_PIN_RE -# matches, so they need their own file list + pattern (#879). The -# line-665 comment ("pin a published big-code-analysis-cli release") -# confirms these are meant to track the current release, not lag it. +# matches, so they need their own file list + pattern (#879). Because +# the paired BCA_SHA256 values come from the release's SHA256SUMS +# asset, these pins move in a post-publish follow-up commit and may +# therefore cite either the latest published release or the one +# immediately before it (RELEASING.md "Version strings in +# documentation"); anything older is stale. # # Deliberately *not* gated here: the `key: bca-…-X.Y.Z` GitHub Actions # cache key (ci.md:205). It embeds the version too, but a stale cache @@ -89,6 +106,15 @@ r'|BCA_VERSION:\s*"(\d+\.\d+(?:\.\d+)?(?:-[0-9A-Za-z.]+)?)"' ) +# A released CHANGELOG section header: `## [X.Y.Z] - YYYY-MM-DD`. +# `## [Unreleased]` has no version-date shape and never matches. The +# file is newest-first, so the first match is the latest published +# release. +CHANGELOG_RELEASE_RE = re.compile( + r"^## \[(\d+\.\d+\.\d+(?:-[0-9A-Za-z.]+)?)\] - \d{4}-\d{2}-\d{2}", + re.MULTILINE, +) + WORKSPACE_VERSION_RE = re.compile( r"^\[workspace\.package\][^\[]*?^version\s*=\s*\"([^\"]+)\"", re.MULTILINE | re.DOTALL, @@ -188,6 +214,16 @@ def workspace_version(root: pathlib.Path) -> str: return m.group(1) +def released_versions(root: pathlib.Path) -> list[str]: + """Released versions from CHANGELOG.md section headers, newest first.""" + return CHANGELOG_RELEASE_RE.findall(read(root / "CHANGELOG.md")) + + +def matches_any(cited: str, allowed: list[str]) -> bool: + """True if `cited` equals (or is a prefix of) any allowed version.""" + return any(normalize(cited, version) == version for version in allowed) + + def package_version(manifest: pathlib.Path) -> str | None: m = PACKAGE_VERSION_RE.search(read(manifest)) return m.group(1) if m else None @@ -228,6 +264,15 @@ def check_external_grammar_lockstep(root: pathlib.Path) -> list[str]: def main() -> int: root = REPO_ROOT canonical = workspace_version(root) + released = released_versions(root) + if not released: + sys.exit( + "error: CHANGELOG.md has no released '## [X.Y.Z] - YYYY-MM-DD' section" + ) + latest_release = released[0] + # ci.md pins move in a post-publish follow-up (they need the + # release's SHA256SUMS), so they may lag one release behind. + ci_allowed = released[:2] failures: list[str] = [] for leaf in EXCLUDED_LEAF_DIRS: @@ -263,22 +308,25 @@ def main() -> int: doc = root / doc_path for m in DOC_PIN_RE.finditer(read(doc)): cited = m.group(1).strip() - if normalize(cited, canonical) != canonical: + if normalize(cited, latest_release) != latest_release: line = read(doc)[: m.start()].count("\n") + 1 failures.append( f"{doc_path}:{line}: snippet cites version " - f"{cited!r}, expected {canonical!r} (or a prefix)" + f"{cited!r}, expected the latest published release " + f"{latest_release!r} (or a prefix)" ) for ci_path in CI_RECIPE_FILES: text = read(root / ci_path) for m in CI_PIN_RE.finditer(text): cited = next(g for g in m.groups() if g is not None) - if normalize(cited, canonical) != canonical: + if not matches_any(cited, ci_allowed): line = text[: m.start()].count("\n") + 1 failures.append( f"{ci_path}:{line}: install snippet pins release " - f"{cited!r}, expected {canonical!r} (or a prefix)" + f"{cited!r}, expected one of {ci_allowed!r} (ci.md " + f"pins move in the post-publish follow-up and may " + f"lag one release)" ) failures.extend(check_external_grammar_lockstep(root)) @@ -289,7 +337,10 @@ def main() -> int: for f in failures: print(f" {f}", file=sys.stderr) return 1 - print(f"versions OK: every owned crate at {canonical}") + print( + f"versions OK: every owned crate at {canonical}, " + f"doc pins at published release {latest_release}" + ) return 0 diff --git a/docs/conventions/documentation.md b/docs/conventions/documentation.md index 5df1a2df..364600c9 100644 --- a/docs/conventions/documentation.md +++ b/docs/conventions/documentation.md @@ -236,7 +236,7 @@ specifications. They go stale immediately. `CHANGELOG.md` follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) format and [Semantic Versioning](https://semver.org/spec/v2.0.0.html). This is a -published `1.x` library with a written stability contract in +published `2.x` library with a written stability contract in [`STABILITY.md`](../../STABILITY.md); the detailed entry, sectioning, and SemVer discipline lives in [`AGENTS.md`](../../AGENTS.md). The rules that matter most for prose: diff --git a/docs/development/mutation_testing.md b/docs/development/mutation_testing.md index 1f287053..a912adc1 100644 --- a/docs/development/mutation_testing.md +++ b/docs/development/mutation_testing.md @@ -52,8 +52,10 @@ cargo install cargo-mutants --locked ``` The repo ships a `cargo mutants` alias (`.cargo/config.toml`) that -matches the CI flags. To exercise a single metric file (the typical -local case): +pins the package and mutation order (`--package big-code-analysis +--no-shuffle --in-place`); CI additionally passes `-j 2 +--minimum-test-timeout 120 --output target/mutants`. To exercise a +single metric file (the typical local case): ```bash cargo mutants -f src/metrics/cognitive.rs @@ -71,7 +73,9 @@ parallelism if the run is starving other work; the CI workflow uses ## Interpreting an escaped mutant -`cargo-mutants` writes its results to `target/mutants/`: +`cargo-mutants` writes its results to `target/mutants/` in CI (via +`--output`); a local run with the alias writes to the default +`mutants.out/` in the repo root: | File | Meaning | |------|---------| diff --git a/docs/development/output_name_normalization_design.md b/docs/development/output_name_normalization_design.md index d8f2cc57..212bc613 100644 --- a/docs/development/output_name_normalization_design.md +++ b/docs/development/output_name_normalization_design.md @@ -1,13 +1,15 @@ # Output-name normalization: postmortem and round-two design input -**Status:** first attempt reverted. The implementation is preserved on -branch `archive/walkfile-name-normalization` (tip `1031be6e`) for -reference. This document records what we were trying to solve, the +**Status:** first attempt reverted. The implementation was preserved +on branch `archive/walkfile-name-normalization` (tip `1031be6e`), but +that branch no longer exists on the remote; recover the code from the +revert commit's parent if needed. This document records what we were +trying to solve, the design we tried, exactly how and why it failed, and the options for a second design round. **Audience:** whoever picks up the next attempt. Read this before -re-deriving the problem — the failure mode is subtle and the obvious +re-deriving the problem: the failure mode is subtle and the obvious fix (the one we tried) is the one that doesn't work. --- @@ -33,46 +35,46 @@ different string depending on how the seed was written: That emitted path is then consumed, unmodified, by several features that each need to compare or key on a file's identity: -- **`[check.exclude]` / `--exclude` globs** — patterns are authored in a +- **`[check.exclude]` / `--exclude` globs**: patterns are authored in a `./`-anchored, walk-root-relative convention (`vendor/**`, `./generated/*`). -- **Baseline keys** (`.bca-baseline.toml`) — must be stable across runs +- **Baseline keys** (`.bca-baseline.toml`): must be stable across runs so a recorded offender is recognized again. -- **`--changed-only` / `--since` diff-scope filtering** — compares a +- **`--changed-only` / `--since` diff-scope filtering**: compares a violation's path against the set of files git reports as changed. -- **`bca diff`** — pairs the "before" and "after" metric sets by file +- **`bca diff`**: pairs the "before" and "after" metric sets by file identity. Because the emitted path form varied with seed spelling, each of these consumers grew its *own* re-anchoring step, and each one was the subject of a separate bug: -- **#488** — absolute seeds defeated the `./`-anchored exclude globset. +- **#488**: absolute seeds defeated the `./`-anchored exclude globset. Fixed by `reanchor_seed` (collapse an at/under-CWD absolute seed to its CWD-relative form before walking). The strip is lexical first (the as-spelled seed, so path-form independence holds even for a symlinked seed), falling back to the *canonicalized* seed only when the lexical - strip fails — the case where the seed reaches the CWD through a + strip fails: the case where the seed reaches the CWD through a symlinked ancestor (`current_dir` is canonical via getcwd; macOS's `/var/folders` and `/tmp` are symlinks into `/private`). Without the fallback, such a seed stayed absolute and nested every emitted file under its full path. -- **#489** — a manifest root *above* the CWD (run from a subdirectory) +- **#489**: a manifest root *above* the CWD (run from a subdirectory) could not be collapsed by `reanchor_seed`, so the walker emitted absolute paths the `./`-anchored deny-set never matched. Fixed by `match_path_for` (anchor the glob match to the walk root, per-seed). -- **#493 / #497 (Bug B)** — `[check.exclude]` re-anchored only `--paths` +- **#493 / #497 (Bug B)**: `[check.exclude]` re-anchored only `--paths` seeds, not `--paths-from` ones, so a `--paths-from` offender escaped the exclude. Fixed by `anchor_against_seeds` (re-anchor a post-walk violation path against the full seed set). -- **#497 (Bug A)** — `bca diff --since ` mis-paired because the +- **#497 (Bug A)**: `bca diff --since ` mis-paired because the before side (a `git archive` extraction) and the after side were rooted differently. Fixed by treating the `--since` positional as a scope and anchoring both sides at the repo root. The pattern was clear: **the same "which file is this, in canonical form?" question was being answered independently, and slightly -differently, at four or five places** — and each new consumer (or each +differently, at four or five places**. Each new consumer (or each new seed source) reopened the bug. ### 1.2 The two goals @@ -88,7 +90,7 @@ new seed source) reopened the bug. `name`** so the same file reports the same name regardless of seed spelling. Concretely, `bca metrics --paths /abs/repo`, `--paths .`, and `--paths "$PWD"` should all emit `name: "./src/foo.rs"` instead - of three different strings — making two `bca metrics` captures + of three different strings, making two `bca metrics` captures directly comparable, and making the JSON output stable. The user explicitly authorized a `2.0` breaking change to achieve goal 2 @@ -143,7 +145,7 @@ root cause. ### 3.1 The three regressions -**R1 — `bca diff --since ` reports the file as added *and* +**R1: `bca diff --since ` reports the file as added *and* removed, with zero deltas.** Reproduced: ```text @@ -154,14 +156,14 @@ removed: ["/tmp//src/work.rs"] ``` A single-file scope has no *directory* seed, so the analysis root is -`None` and the file falls into the verbatim branch — emitting the +`None` and the file falls into the verbatim branch, emitting the absolute path of each side's tree. The before side (a `/tmp` `git archive` extraction) and the after side (the working tree) therefore get non-matching absolute names and never pair. (The directory-scope form, -`--since HEAD src`, worked, which is why every `diff_since` test — all -of which use directory scopes — stayed green.) +`--since HEAD src`, worked, which is why every `diff_since` test, all +of which use directory scopes, stayed green.) -**R2 — subdirectory-scoped baseline keys lose the scope prefix.** +**R2: subdirectory-scoped baseline keys lose the scope prefix.** Reproduced: ```text @@ -176,18 +178,18 @@ seed), so `canonical_name` strips the whole `src/` and emits `./work.rs` `src/work.rs`. A baseline written under one scope no longer suppresses under the other; offenders silently re-fire. -**R3 — `--changed-only` from a subdirectory silently drops violations +**R3: `--changed-only` from a subdirectory silently drops violations (gate bypass).** The canonical `name` is anchored to the analysis root, but `DiffScope::contains` resolves a violation path with `canonicalize_for_match` (`path.canonicalize()`), which is **CWD-based**. When the analysis root ≠ CWD (a subdir scope, or a manifest root above the CWD), the canonical `./`-name resolves to the wrong absolute path, -fails to match the git-changed set, and the violation is dropped — the +fails to match the git-changed set, and the violation is dropped; the gate can exit 0 on a real regression. ### 3.2 The root cause: consumers have *heterogeneous anchors* -The premise — "compute one canonical identity" — is **unsound**, because +The premise ("compute one canonical identity") is **unsound**, because the consumers do not share an anchor: | Consumer | Anchor it needs | Why | @@ -205,7 +207,7 @@ file, so a consumer that resolves it against a *different* anchor (CWD, baseline dir) can no longer recover the file's true location. Crucially, the pre-existing per-consumer design was **not** redundant -boilerplate — each consumer re-anchored *because it genuinely has a +boilerplate: each consumer re-anchored *because it genuinely has a different anchor*. Collapsing them was the wrong altitude: we mistook four correct, anchor-specific transforms for one duplicated transform. @@ -216,7 +218,7 @@ The project's own dogfooding always runs `--paths .` (or manifest repo-root == baseline-anchor**. In that one configuration all anchors coincide, so the lossy name round-trips and nothing breaks. The regressions only surface for subdir scopes, single-file diffs, and -`--changed-only`/manifest runs from a subdirectory — none of which the +`--changed-only`/manifest runs from a subdirectory, none of which the test suite or the self-scan exercised. This is itself a lesson: **the gate's uniformity hid an anchor-coupling bug.** @@ -224,21 +226,21 @@ gate's uniformity hid an anchor-coupling bug.** ## 4. Options for round two -### Option A — Do not unify; keep per-consumer anchoring (what we reverted to) +### Option A: Do not unify; keep per-consumer anchoring (what we reverted to) Accept that identity is anchor-specific and keep each consumer's transform. This is correct and is the current state. The cost is the "five places answer the same-ish question" maintainability smell that -motivated the attempt — but they are *not* actually the same question, +motivated the attempt, but they are *not* actually the same question, so the smell is partly illusory. **Lowest risk; abandons goal 2.** If we stay here, the worthwhile hardening is a **shared, well-tested "resolve this emitted path to an absolute path" primitive** that each -consumer calls before applying its *own* anchor — i.e. unify the +consumer calls before applying its *own* anchor, i.e. unify the *resolution* step (lexical-abs, the part that is genuinely identical) without unifying the *anchoring* step (which is not). -### Option B — Normalize the *display* name only; keep raw paths for keys +### Option B: Normalize the *display* name only; keep raw paths for keys Split the two roles the failed design conflated: @@ -251,28 +253,28 @@ Split the two roles the failed design conflated: `WalkFile` would carry both `io_path` and a `display_name`, but the *keying* consumers would continue to use `io_path` (or a -resolve-to-absolute of it) and their own anchors — never the display +resolve-to-absolute of it) and their own anchors, never the display name. This achieves the user-visible normalization without feeding the lossy form into the keying paths. **Risk:** medium. Requires auditing every consumer to confirm it uses the key path, not the display name. The display name is only safe where it is purely presentational. (Note: `bca diff`'s pairing key is *also* -display-ish — two captures must pair on the normalized name — so diff +display-ish: two captures must pair on the normalized name, so diff specifically *wants* the normalized form, but anchored per-side at the tree root, not the LCA. Diff may need its own normalization parameter.) -### Option C — Fix-forward the unified name with an explicit, non-lossy anchor +### Option C: Fix-forward the unified name with an explicit, non-lossy anchor -Keep one `name`, but (1) make it **non-lossy** — anchor it to a single +Keep one `name`, but (1) make it **non-lossy**: anchor it to a single *fixed* root passed explicitly by the caller (CWD for standalone check/metrics; the manifest dir when a manifest set the paths; the tree -root for each diff side) rather than the LCA of the scope seeds — and +root for each diff side) rather than the LCA of the scope seeds; and (2) ensure every consumer resolves it against *that same* fixed root. The hard part is that the consumers' anchors still differ (baseline-dir vs CWD vs tree-root). To make one name work for all, you would have to -re-express every consumer's anchor in terms of the chosen fixed root — +re-express every consumer's anchor in terms of the chosen fixed root: e.g. make baseline keys relative to the analysis root instead of the baseline-file dir (a baseline-format change), and make `--changed-only` compare in analysis-root space instead of canonicalizing against CWD. @@ -295,23 +297,24 @@ the fallback if normalization is judged not worth the consumer audit. Whatever the choice, the next attempt **must** add regression tests for the three configurations the gate's uniformity hid: -1. a subdirectory scope (`--paths src`) — assert the baseline key and +1. a subdirectory scope (`--paths src`): assert the baseline key and emitted name retain the `src/` segment; -2. a single-file `--since` scope — assert a real paired delta; -3. `--changed-only` (or a manifest run) **from a subdirectory** — assert +2. a single-file `--since` scope: assert a real paired delta; +3. `--changed-only` (or a manifest run) **from a subdirectory**: assert in-scope violations are kept. --- ## 5. Reference -- Reverted implementation: branch `archive/walkfile-name-normalization`, - tip `1031be6e` (`refactor(api)!: normalize output names via dual-path +- Reverted implementation: formerly on branch + `archive/walkfile-name-normalization`, tip `1031be6e`; no longer on + the remote (`refactor(api)!: normalize output names via dual-path WalkFile` + `refactor(cli): share lexical path normalizer; hoist walk CWD read`). - The correct, retained consumer-side fixes for #497 Bug A/B are on the main work branch as `f0b83a7d`. -- `verify-name-only-churn.py` (kept) — proves a bulk snapshot regen is +- `verify-name-only-churn.py` (kept): proves a bulk snapshot regen is name-only/value-preserving; reusable for whichever round-two option touches emitted paths. - Related: STABILITY.md (the `2.0` deferral list), issues @@ -322,7 +325,7 @@ the three configurations the gate's uniformity hid: 1. **Re-anchoring at multiple consumers is not necessarily duplication.** When N consumers transform the same value, check whether they share a *parameter* (here: the anchor). If they don't, "compute it once" is - the wrong altitude — you will produce a value that is correct for one + the wrong altitude: you will produce a value that is correct for one consumer and lossy for the rest. 2. **A uniform gate can hide a coupling bug.** Every self-scan/CI path here ran `--paths .` from the repo root, where four distinct anchors diff --git a/docs/file-detection.md b/docs/file-detection.md index 59d01da2..f6c7307a 100644 --- a/docs/file-detection.md +++ b/docs/file-detection.md @@ -3,7 +3,7 @@ How `big-code-analysis` decides which language a file is written in, and what it reads off disk before parsing. All of the logic lives in [`src/tools.rs`](../src/tools.rs), [`src/langs.rs`](../src/langs.rs), -and the macros in [`src/macros.rs`](../src/macros.rs). +and the macros in [`src/macros/mod.rs`](../src/macros/mod.rs). ## Reading the file @@ -13,10 +13,10 @@ happens: | Function | Behaviour | |----------|-----------| | `read_file(path)` | Reads the whole file. Normalises CRLF and lone CR to LF in place. Buffer ends with exactly one trailing `\n`. | -| `read_file_with_eol(path)` | Same normalisation, plus: returns `None` for files ≤ 3 bytes; strips a leading UTF-8/UTF-16 BOM; sniffs the first ~64 bytes and returns `None` if a U+FFFD replacement char appears (treated as non-UTF-8). | +| `read_file_with_eol(path)` | Same normalisation, plus: returns `None` for files ≤ 3 bytes and for files with a UTF-16 BOM (#803); strips a leading UTF-8 BOM; byte-validates the first ~64 bytes with `std::str::from_utf8` and returns `None` on invalid UTF-8 (#746, #758; a lossy U+FFFD check was rejected because U+FFFD can legitimately appear in source). | Downstream consumers must assume the buffer contains no `\r` bytes. The -metric engine depends on this — passing raw CRLF input to a parser +metric engine depends on this: passing raw CRLF input to a parser would shift line numbers and break LoC counts. ## Detecting the language @@ -24,18 +24,18 @@ would shift line numbers and break LoC counts. There are two public entry points, both returning a [`LANG`](../src/langs.rs) variant: -### `get_language_for_file(path)` — extension only +### `get_language_for_file(path)`: extension only Lowercases the file's extension and looks it up. Returns `None` if the path has no extension, the extension is not valid UTF-8, or no language -claims that extension. This is the cheap path — no buffer required. +claims that extension. This is the cheap path; no buffer required. -### `guess_language(buf, path)` — extension + mode line + shebang +### `guess_language(buf, path)`: extension + mode line + shebang Combines the extension lookup with an Emacs/Vim *mode line* scan of the buffer and a shebang scan of the first line. Returns `(Option, &str)` where the second element is the canonical lowercase language slug -(`"cpp"`, `"csharp"`, `"rust"`, etc.) — the same string `LANG::name` +(`"cpp"`, `"csharp"`, `"rust"`, etc.), the same string `LANG::name` emits and a valid `FromStr` lookup token. Objective-C (`.m`) parses with the dedicated `tree-sitter-objc` grammar and reports `"objc"`; Objective-C++ (`.mm`) stays on the C++ grammar and reports `"cpp"` (see @@ -49,10 +49,10 @@ Mode line scanning runs three regexes (compiled once via `OnceLock`): | `-\*-\s*([^:;\s]+)\s*-\*-` | Bare Emacs mode | `// -*- c++ -*-` | | `(?i)vim\s*:.*[^\w]ft\s*=\s*([^:\s]+)` | Vim `ft=` modeline | `// vim: set ts=4 ft=c++` | -The scan checks the **first 4 lines** for any of the three patterns, -then the **last 4 lines** for the Vim pattern only (Vim modelines are +The scan checks the **first 5 lines** for any of the three patterns, +then the **last 5 lines** for the Vim pattern only (Vim modelines are conventionally at the bottom of the file, Emacs ones at the top). -"Lines" here means LF-delimited segments — `guess_language` relies on +"Lines" here means LF-delimited segments; `guess_language` relies on the CRLF/CR → LF normalisation performed by the readers above. #### Resolution rules @@ -60,19 +60,17 @@ the CRLF/CR → LF normalisation performed by the readers above. Given an extension result and a mode result, `guess_language` resolves as follows: -1. **Both agree** — return that language. Apply the Objective-C - override (see below) before picking the display name. -2. **Both disagree** — extension wins. The mode line is treated as - advisory, and the Objective-C overlay is **not** consulted in this - branch (the display name comes straight from the extension's - `LANG`). -3. **Only extension matches** — return it. -4. **Only mode matches** — return it. -5. **Only shebang matches** — return it. The shebang signal is +1. **Both agree**: return that language. +2. **Both disagree**: extension wins. The mode line is treated as + advisory; the display name comes straight from the extension's + `LANG`. +3. **Only extension matches**: return it. +4. **Only mode matches**: return it. +5. **Only shebang matches**: return it. The shebang signal is consulted **after** the extension and mode-line lookups have both come back empty, so an explicit `.py` extension or `mode: python` line on a script with `#!/bin/sh` still resolves to Python. -6. **Nothing matches** — return `(None, "")`. +6. **Nothing matches**: return `(None, "")`. #### Shebang scan @@ -111,16 +109,16 @@ Since #724, Objective-C has its own dedicated grammar: `LANG::Cpp`, reporting `"cpp"`. Objective-C++ mixes Objective-C and C++; the `tree-sitter-objc` grammar parses the Objective-C and C parts but not C++ constructs (templates, namespaces, classes, `::`). - Because a `.mm` file is `.mm` precisely because it contains C++ — the - larger, more pervasive surface — the C++ grammar degrades more + Because a `.mm` file is `.mm` precisely because it contains C++ (the + larger, more pervasive surface), the C++ grammar degrades more gracefully on `.mm` than the Objective-C grammar would (it only trips on the Objective-C glue, not the whole C++ half). This is the same asymmetric-failure trade-off #721 used to keep `.h` on `LANG::Cpp`. Metrics for the Objective-C portions of a `.mm` file are therefore - approximate — a known limitation tracked in the original #724 decision. + approximate, a known limitation tracked in the original #724 decision. -The earlier `fake::get_true` overlay — which folded both extensions onto -the `"cpp"` slug while they shared one grammar (#540) — was retired in +The earlier `fake::get_true` overlay, which folded both extensions onto +the `"cpp"` slug while they shared one grammar (#540), was retired in the #724 change: `.m` now reports `"objc"` natively and `.mm` reports `"cpp"` natively, so no override is needed. @@ -144,32 +142,32 @@ last two tuple fields of each `mk_langs!` entry in ``` The `mk_extensions!` and `mk_emacs_mode!` macros in -[`src/macros.rs`](../src/macros.rs) expand these into the public +[`src/macros/mod.rs`](../src/macros/mod.rs) expand these into the public `get_from_ext(ext) -> Option` and `get_from_emacs_mode(mode) -> Option` lookup functions. Both are -plain `match` arms — no fuzzy matching, no fallback. +plain `match` arms: no fuzzy matching, no fallback. To add or change an alias, edit the `mk_langs!` invocation; the generated lookups update automatically. ## How callers use detection -- **Library** — `guess_language` is the standard entry point; the CLI +- **Library**: `guess_language` is the standard entry point; the CLI and REST server both go through it. `get_language_for_file` is available for callers that have only a path. -- **CLI (`bca`)** — auto-detects via `guess_language` +- **CLI (`bca`)**: auto-detects via `guess_language` unless the user passes `--language ` (short form `-l`; the old `--language-type` spelling stays as a hidden alias for one release cycle). The flag value is resolved by trying `LANG`'s `FromStr` (canonical name, e.g. `rust`) first, then `get_from_ext` (extension, e.g. `rs`), with an `Action::PreprocProduce` short-circuit. An unrecognised value is a hard error (exit 1) that lists the valid - language names — it no longer silently disables analysis. See + language names; it no longer silently disables analysis. See [`big-code-analysis-cli/src/main.rs`](../big-code-analysis-cli/src/main.rs). -- **REST (`bca-web`)** — every endpoint that takes a path plus +- **REST (`bca-web`)**: every endpoint that takes a path plus buffer calls `guess_language` to resolve the language before dispatching to a parser. -- **Tests** — `tests/common/mod.rs` falls back to `guess_language` when +- **Tests**: `tests/common/mod.rs` falls back to `guess_language` when the test harness cannot infer the language another way. ## Detection failures @@ -178,7 +176,7 @@ If `guess_language` returns `(None, _)`: - The CLI logs and skips the file. - The REST server returns an error to the caller. -- The library leaves it to the caller — there is no default parser. +- The library leaves it to the caller; there is no default parser. Beyond the shebang scan described above, there is no content-based heuristic and no MIME sniffing. Add a missing extension or Emacs mode diff --git a/enums/src/rust.rs b/enums/src/rust.rs index 074eafd0..22217f17 100644 --- a/enums/src/rust.rs +++ b/enums/src/rust.rs @@ -70,7 +70,7 @@ pub fn generate_macros(output: &Path) -> std::io::Result<()> { fn create_macros_file(output: &Path, filename: &str, u_name: &str) -> std::io::Result<()> { let mut macro_file = File::open(Path::new(&format!( "{}/{}/{}.txt", - &env::var("CARGO_MANIFEST_DIR").unwrap(), + env::var("CARGO_MANIFEST_DIR").unwrap(), MACROS_DEFINITION_DIR, filename )))?; diff --git a/man/bca-check.1 b/man/bca-check.1 index 05605749..73671ed7 100644 --- a/man/bca-check.1 +++ b/man/bca-check.1 @@ -31,7 +31,7 @@ Ignore in\-source suppression markers (`bca: suppress`, `#lizard forgives`, etc. Surface suppressed debt in the offender document instead of dropping it. Offenders silenced by an in\-source `bca: suppress` marker or covered by the baseline are still kept out of the gate (exit code and human stream are unaffected), but are emitted into the `\-\-format sarif` document carrying a SARIF `suppressions` entry — GitHub Code Scanning renders them as suppressed (closed) alerts so the debt stays visible. Only the SARIF format represents suppression; other formats ignore the flag. Mutually exclusive with `\-\-no\-suppress` (which un\-silences markers) and `\-\-write\-baseline` .TP \fB\-\-report\-format\fR \fI\fR -CI/IDE *report dialect* for offender records (Checkstyle 4.3 XML, SARIF 2.1.0 JSON, GitLab Code Climate JSON, clang/GCC warning lines, MSVC warning lines). Named `\-\-report\-format` to separate "which CI report dialect" from the data\-serialization `\-\-format` the structured subcommands use. When omitted *and* `\-\-output` is also omitted, only the human\-readable stderr stream is emitted; the exit\-code contract is unaffected. When omitted but `\-\-output` is given, the dialect is inferred from the output extension (`.sarif` → sarif, `.xml` → checkstyle); an extension with no unique dialect is a usage error. The old `\-\-format` / `\-O` / `\-\-output\-format` spellings stay hidden aliases for one release cycle and are slated for removal in 2.0 +CI/IDE *report dialect* for offender records (Checkstyle 4.3 XML, SARIF 2.1.0 JSON, GitLab Code Climate JSON, clang/GCC warning lines, MSVC warning lines). Named `\-\-report\-format` to separate "which CI report dialect" from the data\-serialization `\-\-format` the structured subcommands use. When omitted *and* `\-\-output` is also omitted, only the human\-readable stderr stream is emitted; the exit\-code contract is unaffected. When omitted but `\-\-output` is given, the dialect is inferred from the output extension (`.sarif` → sarif, `.xml` → checkstyle); an extension with no unique dialect is a usage error. The old `\-\-format` / `\-O` / `\-\-output\-format` spellings stay hidden aliases for one release cycle and are slated for removal in the next major .br .br diff --git a/man/bca-metrics.1 b/man/bca-metrics.1 index 0eb992d0..797e6ee1 100644 --- a/man/bca-metrics.1 +++ b/man/bca-metrics.1 @@ -10,7 +10,7 @@ Compute per\-file metrics and emit them in a structured format .SH OPTIONS .TP \fB\-O\fR, \fB\-\-format\fR \fI\fR -Output format. When omitted, the default `text` format prints a human\-readable colored tree to stdout (`metrics` shows the metric tree, `ops` the operator/operand tree); pass `\-\-format text` to request that default explicitly (e.g. to override a `bca.toml` that set a structured format). `json` / `yaml` / `toml` / `cbor` / `csv` emit structured per\-file data. `\-\-output\-format` is accepted as a deprecated alias; it is hidden from help and slated for removal in 2.0 +Output format. When omitted, the default `text` format prints a human\-readable colored tree to stdout (`metrics` shows the metric tree, `ops` the operator/operand tree); pass `\-\-format text` to request that default explicitly (e.g. to override a `bca.toml` that set a structured format). `json` / `yaml` / `toml` / `cbor` / `csv` emit structured per\-file data. `\-\-output\-format` is accepted as a deprecated alias; it is hidden from help and slated for removal in the next major .br .br diff --git a/man/bca-ops.1 b/man/bca-ops.1 index f373a170..875d97fd 100644 --- a/man/bca-ops.1 +++ b/man/bca-ops.1 @@ -10,7 +10,7 @@ Extract per\-file operands and operators .SH OPTIONS .TP \fB\-O\fR, \fB\-\-format\fR \fI\fR -Output format. When omitted, the default `text` format prints a human\-readable colored tree to stdout (`metrics` shows the metric tree, `ops` the operator/operand tree); pass `\-\-format text` to request that default explicitly (e.g. to override a `bca.toml` that set a structured format). `json` / `yaml` / `toml` / `cbor` / `csv` emit structured per\-file data. `\-\-output\-format` is accepted as a deprecated alias; it is hidden from help and slated for removal in 2.0 +Output format. When omitted, the default `text` format prints a human\-readable colored tree to stdout (`metrics` shows the metric tree, `ops` the operator/operand tree); pass `\-\-format text` to request that default explicitly (e.g. to override a `bca.toml` that set a structured format). `json` / `yaml` / `toml` / `cbor` / `csv` emit structured per\-file data. `\-\-output\-format` is accepted as a deprecated alias; it is hidden from help and slated for removal in the next major .br .br diff --git a/src/node.rs b/src/node.rs index 076df7b3..6e3a0535 100644 --- a/src/node.rs +++ b/src/node.rs @@ -270,11 +270,7 @@ impl<'a> Node<'a> { let mut level = level; let mut node = *self; while level != 0 { - if let Some(parent) = node.parent() { - node = parent; - } else { - return None; - } + node = node.parent()?; level -= 1; } diff --git a/tests/book_library_examples.rs b/tests/book_library_examples.rs new file mode 100644 index 00000000..b179e9f9 --- /dev/null +++ b/tests/book_library_examples.rs @@ -0,0 +1,106 @@ +//! Lifts the runnable examples from the book's *Using as a Library* +//! chapter (`big-code-analysis-book/src/library/in-memory.md`, +//! `walking-funcspace.md`, and `reuse-tree.md`) into a cargo-tested +//! module, so doc rot is caught by `cargo test` instead of by readers +//! trying to copy-paste broken snippets. If you change the book, +//! mirror the change here; if a refactor breaks an example here, fix +//! both. (`ast-traversal.md` is pinned separately by +//! `book_ast_traversal_examples.rs`.) + +#![cfg(feature = "rust")] + +use big_code_analysis::{FuncSpace, LANG, MetricsOptions, Source, SpaceKind, analyze}; + +/// `in-memory.md` — "Reading from a buffer". +#[cfg(feature = "python")] +fn analyze_buffer(source: &[u8]) -> Option { + let space = analyze( + Source::new(LANG::Python, source).with_name(Some("".to_owned())), + MetricsOptions::default(), + ) + .ok()?; + + Some(space.metrics.cognitive.cognitive_sum()) +} + +#[cfg(feature = "python")] +#[test] +fn in_memory_analyze_buffer() { + let source = b"def f(x):\n if x:\n return 1\n return 0\n"; + assert_eq!(analyze_buffer(source), Some(1)); +} + +/// `walking-funcspace.md` — "Recursive walk". +fn hotspots(space: &FuncSpace, threshold: u64, out: &mut Vec) { + if space.kind == SpaceKind::Function + && space.metrics.cognitive.cognitive_sum() > threshold + && let Some(name) = &space.name + { + out.push(format!( + "{name} (lines {}\u{2013}{})", + space.start_line, space.end_line, + )); + } + for child in &space.spaces { + hotspots(child, threshold, out); + } +} + +#[test] +fn walking_funcspace_hotspots() { + let source = b"\ +fn easy() { let _ = 1; } +fn hard(x: i32) -> i32 { + if x > 0 { if x > 10 { 1 } else { 2 } } else { 3 } +} +"; + let space = analyze( + Source::new(LANG::Rust, source).with_name(Some("snippet.rs".to_owned())), + MetricsOptions::default(), + ) + .expect("parses"); + + let mut hits = Vec::new(); + hotspots(&space, 2, &mut hits); + assert_eq!(hits, ["hard (lines 2\u{2013}4)"]); +} + +/// `reuse-tree.md` — "Working example". +#[test] +fn reuse_tree_working_example() { + use big_code_analysis::{Ast, tree_sitter}; + + let source_code = "fn main() { if true { 1 } else { 2 }; }"; + let source = source_code.as_bytes().to_vec(); + + let mut parser = tree_sitter::Parser::new(); + parser + .set_language( + &LANG::Rust + .tree_sitter_language() + .expect("rust feature enabled"), + ) + .expect("rust grammar pinned to a compatible version"); + let tree = parser + .parse(&source, None) + .expect("parser has a language set"); + + let from_tree = + Ast::from_tree_sitter(LANG::Rust, tree, source.clone(), Some("foo.rs".to_owned())) + .expect("rust feature enabled") + .metrics(MetricsOptions::default()) + .expect("non-empty input"); + + let from_bytes = analyze( + Source::new(LANG::Rust, &source).with_name(Some("foo.rs".to_owned())), + MetricsOptions::default(), + ) + .expect("non-empty input"); + + assert_eq!( + from_tree.metrics.cyclomatic.cyclomatic_sum(), + from_bytes.metrics.cyclomatic.cyclomatic_sum(), + ); + // expected: unit base (+1), fn main (+1), the if/else (+1) = 3. + assert_eq!(from_tree.metrics.cyclomatic.cyclomatic_sum(), 3); +}