feat: add CLI startup and command latency budget enforcement (issue #698)

.github/workflows/benchmark-latency.yml

@@ -0,0 +1,125 @@
+# Latency Budget Enforcement CI
+#
+# Runs the CLI cold-start and command-latency Criterion benchmarks on every
+# push / pull request and checks the results against the project's latency
+# budgets.  Fails the workflow when a statistically meaningful regression is
+# detected, helping to keep the CLI snappy for all users.
+#
+# See CLI_LATENCY_BUDGETS.md for budget definitions and environment overrides.
+
+name: Latency Budget
+
+on:
+  push:
+    branches: [master, main]
+    paths:
+      - '**.rs'
+      - 'Cargo.toml'
+      - 'Cargo.lock'
+      - 'benches/**'
+      - '.github/workflows/benchmark-latency.yml'
+  pull_request:
+    paths:
+      - '**.rs'
+      - 'Cargo.toml'
+      - 'Cargo.lock'
+      - 'benches/**'
+      - '.github/workflows/benchmark-latency.yml'
+
+  # Allow manual trigger for ad-hoc budget checks.
+  workflow_dispatch:
+
+jobs:
+  latency-budget:
+    name: Latency Budget Check
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+
+      - name: Install system dependencies
+        run: sudo apt-get update && sudo apt-get install -y libudev-dev
+
+      # Build the project first so benchmark compilation is faster.
+      - name: Build project
+        run: cargo build --locked
+
+      # Run only the latency-relevant benchmark groups.  This keeps CI fast
+      # while still exercising cold-start and command-dispatch paths.
+      - name: Run latency benchmarks
+        run: |
+          cargo bench --locked -- \
+            cli_cold_start \
+            cli_command_latency \
+            latency_budget \
+            2>&1 | tee target/criterion/latency-bench-output.txt
+
+      # Parse Criterion output and check against latency budgets.
+      # The check-latency-budgets.sh script extracts median values from the
+      # default Criterion stdout format and compares them against the budgets
+      # defined in the bash script (which mirror src/utils/latency_budget.rs).
+      - name: Parse and check latency budget report
+        id: budget-check
+        run: |
+          chmod +x scripts/check-latency-budgets.sh
+          REPORT=$(bash scripts/check-latency-budgets.sh \
+            --input target/criterion/latency-bench-output.txt \
+            --report-path target/criterion/latency-budget-report.json \
+            2>&1) || EXIT_CODE=$?
+          echo "::group::Latency Budget Report"
+          echo "$REPORT"
+          echo "::endgroup::"
+          ALL_PASS=$(echo "$REPORT" | jq -r '.all_pass')
+          echo "all_pass=$ALL_PASS" >> "$GITHUB_OUTPUT"
+          if [ "$ALL_PASS" = "false" ]; then
+            echo "❌ Latency budget violations detected!"
+            echo "failures=true" >> "$GITHUB_OUTPUT"
+            exit 1
+          fi
+          echo "✅ All latency budgets met."
+
+      - name: Upload Criterion report (artefact)
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: criterion-latency-report
+          path: |
+            target/criterion/cli_cold_start/
+            target/criterion/cli_command_latency/
+            target/criterion/latency_budget/
+            target/criterion/latency-bench-output.txt
+            target/criterion/latency-budget-report.json
+
+      # Post a PR comment with the budget check summary when run on a PR.
+      - name: Comment PR with budget summary
+        if: github.event_name == 'pull_request' && always()
+        uses: actions/github-script@v7
+        with:
+          script: |
+            const fs = require('fs');
+            const reportPath = 'target/criterion/latency-budget-report.json';
+            let summary = '## ⏱ CLI Latency Budget Check\n\n';
+            if (fs.existsSync(reportPath)) {
+              const report = JSON.parse(fs.readFileSync(reportPath, 'utf8'));
+              let details = '';
+              for (const check of report.checks) {
+                const icon = check.status === 'PASS' ? '✅' :
+                             check.status === 'FAIL' ? '❌' :
+                             check.status === 'NOISY' ? '⚠️' :
+                             check.status === 'SKIPPED' ? '⏭️' : '❗';
+                summary += `| ${icon} ${check.budget} | ${check.budget_max_ms} ms | ${check.actual_median_ms} ms | ${check.status} |\n`;
+              }
+              if (report.any_fail) summary += '\n❌ **Some budgets were violated.**';
+              else summary += '\n✅ **All budgets met.**';
+            } else {
+              summary += '_No latency budget report was generated._';
+            }
+            github.rest.issues.createComment({
+              issue_number: context.issue.number,
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              body: summary
+            });

BENCHMARKS.md

@@ -70,6 +70,9 @@ the next baseline so pull requests can compare performance consistently.
 |---|---|
 | `simulate_ops_10k` | Baseline wrapping-add loop — regression guard |
 | `cli_arg_parsing` | Argument tokenisation cost for common subcommands |
+| `cli_cold_start` | Cold-start latency: `info`, `--help`, `--version` |
+| `cli_command_latency` | Command dispatch latency for wallet, network, config, template, deploy |
+| `latency_budget` | Latency budget check engine overhead |
 | `template_registry_deserialise` | JSON deserialisation at 10 / 50 / 200 / 1000 entries |
 | `template_registry_search` | Linear search over registry at 10 / 100 / 500 entries |
 | `wallet_entry_format` | KV formatting and JSON serialisation for wallet lists |
@@ -125,3 +128,17 @@ To catch performance regressions in pull requests, compare the Criterion
 
 Criterion's `--output-format bencher` emits a machine-readable format compatible
 with [github-action-benchmark](https://github.com/benchmark-action/github-action-benchmark).
+
+### Latency budget enforcement
+
+A dedicated [latency budget](CLI_LATENCY_BUDGETS.md) CI workflow
+(`benchmark-latency.yml`) runs cold-start and command-latency benchmarks on
+every push / PR and **fails the pipeline** if any budget is exceeded:
+
+```yaml
+- name: Run latency benchmarks
+  run: cargo bench --locked -- cli_cold_start cli_command_latency latency_budget
+```
+
+See [`CLI_LATENCY_BUDGETS.md`](./CLI_LATENCY_BUDGETS.md) for the full list of
+budgets, environment variable overrides, and how to add new budgets.

CLI_LATENCY_BUDGETS.md

@@ -0,0 +1,160 @@
+# CLI Latency Budgets
+
+StarForge enforces **latency budgets** for its CLI's cold-start and command
+dispatch paths.  This ensures that every `starforge` invocation feels snappy
+and that performance regressions are caught in CI before they reach users.
+
+---
+
+## What is a latency budget?
+
+A **latency budget** is a per-command upper bound on wall-clock execution time
+(e.g. `cli_info` must complete within **300 ms** median).  Budgets are defined
+in [`src/utils/latency_budget.rs`](./src/utils/latency_budget.rs) and checked
+against Criterion benchmark results.
+
+| Budget label                | Default max (ms) | CV threshold | Description                                |
+|-----------------------------|------------------|--------------|--------------------------------------------|
+| `cli_cold_start_info`       | 500              | 0.15         | Cold start: `starforge -q info`            |
+| `cli_cold_start_help`       | 350              | 0.15         | Cold start: `starforge --help`             |
+| `cli_cold_start_version`    | 300              | 0.15         | Cold start: `starforge --version`          |
+| `cli_wallet_list`           | 300              | 0.20         | `starforge wallet list`                    |
+| `cli_wallet_show`           | 350              | 0.20         | `starforge wallet show <name>`             |
+| `cli_network_show`          | 250              | 0.20         | `starforge network show`                   |
+| `cli_network_switch`        | 300              | 0.20         | `starforge network switch <net>`           |
+| `cli_config_show`           | 300              | 0.20         | `starforge config show`                    |
+| `cli_info`                  | 300              | 0.20         | `starforge info`                           |
+| `cli_template_list`         | 350              | 0.20         | `starforge template list`                  |
+| `cli_template_search`       | 400              | 0.20         | `starforge template search <query>`        |
+| `cli_deploy_help`           | 350              | 0.20         | `starforge deploy --help`                  |
+| `cli_benchmark_wasm`        | 500              | 0.20         | `starforge benchmark wasm`                 |
+
+> **CV threshold**: the maximum **coefficient of variation** (std_dev / mean)
+> before a measurement is flagged as _noisy_ (yellow / warning).  High noise
+> means results are unreliable — the run should be retried with more samples.
+
+---
+
+## How budgets are checked
+
+1. **Criterion benchmarks** measure median latency for each path
+   ([`benches/benchmarks.rs`](./benches/benchmarks.rs), groups
+   `cli_cold_start`, `cli_command_latency`, `latency_budget`).
+
+2. The **latency budget engine** ([`src/utils/latency_budget.rs`](./src/utils/latency_budget.rs))
+   compares the measured median against the budget:
+
+   - **PASS**  — median ≤ budget, CV ≤ threshold
+   - **FAIL**  — median > budget
+   - **NOISY** — median OK, but CV > threshold (retry recommended)
+   - **SKIPPED** — budget inactive for this environment
+   - **ERROR** — missing measurement or invalid input
+
+3. A **JSON report** is written to `target/criterion/latency-budget-report.json`
+   and consumed by CI to gate the pipeline.
+
+---
+
+## Running budget checks locally
+
+```bash
+# Run only the latency benchmarks
+cargo bench -- locked -- cli_cold_start cli_command_latency latency_budget
+
+# Run the full suite (includes gas analyzer benchmarks)
+cargo bench
+
+# Override a budget via environment variable (no code change needed)
+STARFORGE_LATENCY_BUDGET_CLI_INFO=800 cargo bench -- cli_cold_start
+```
+
+---
+
+## Environment variable overrides
+
+Each budget can be overridden at runtime with the environment variable
+`STARFORGE_LATENCY_BUDGET_<UPPER_LABEL>`.
+
+| Value                  | Effect                                  |
+|------------------------|-----------------------------------------|
+| A positive integer (ms)| Sets the budget's `max_median`          |
+| `off` or `0`           | Deactivates the budget (always skipped) |
+| Any other string       | Silently ignored (fallback to default)  |
+
+Examples:
+
+```bash
+# Double the info budget to 600 ms (useful on slower CI runners)
+STARFORGE_LATENCY_BUDGET_CLI_INFO=600 cargo bench -- cli_cold_start
+
+# Deactivate the wallet list budget entirely
+STARFORGE_LATENCY_BUDGET_CLI_WALLET_LIST=off cargo bench -- cli_cold_start
+```
+
+---
+
+## CI integration
+
+The [`benchmark-latency.yml`](.github/workflows/benchmark-latency.yml) workflow
+runs on every push / PR that touches `.rs` files, `Cargo.toml`, or the benches
+directory.  It:
+
+1. Builds the project
+2. Runs the latency benchmark groups
+3. Generates and inspects the JSON budget report
+4. **Fails the workflow** if any budget is violated
+5. Uploads the Criterion report as a build artefact
+6. Comments on the PR with a budget summary table
+
+---
+
+## Adding a new budget
+
+1. Add a new entry to the `default()` impl of `LatencyBudgets` in
+   [`src/utils/latency_budget.rs`](./src/utils/latency_budget.rs):
+
+   ```rust
+   LatencyBudget::new_static("cli_my_new_command", 400, Some(0.20), true),
+   ```
+
+2. Add a corresponding Criterion benchmark function in
+   [`benches/benchmarks.rs`](./benches/benchmarks.rs) with a matching label.
+
+3. Register the function in the `criterion_group!` macro at the bottom of
+   `benches/benchmarks.rs`.
+
+4. Run `cargo test --test latency_budget_test` to verify the new budget is
+   picked up by the tests.
+
+5. Run `cargo bench -- cli_cold_start cli_command_latency` to establish a
+   baseline for the new path.
+
+---
+
+## Security notes
+
+- Latency budgets are **purely local measurements**.  No measurement data is
+  transmitted over the network.
+- Environment variable overrides are scoped to the current process; they are
+  never persisted or saved to config files.
+- Budget definitions are compiled into the binary.  There is no external
+  configuration file that could be tampered with to alter performance guarantees.
+
+---
+
+## Migration notes
+
+If you are upgrading from a version without latency budgets (before this
+feature was introduced), the new CI workflow will start checking budgets
+automatically.  To avoid breakage on a slow CI runner:
+
+1. Set the env var overrides listed in the workflow file to generous values.
+2. Run one CI cycle, inspect the generated report, then tighten budgets.
+
+---
+
+## Compatibility
+
+- **Platforms**: Linux (CI), macOS, and Windows are all supported.
+- **Rust version**: Rust 1.70+ (stable).
+- **No external services** required.  Budget checks work fully offline.