Skip to content

feat: add CLI startup and command latency budget enforcement (issue #698) - #702

Merged
Nanle-code merged 2 commits into
Nanle-code:masterfrom
meem08:feat/migration-ai-assistance
Jul 29, 2026
Merged

feat: add CLI startup and command latency budget enforcement (issue #698)#702
Nanle-code merged 2 commits into
Nanle-code:masterfrom
meem08:feat/migration-ai-assistance

Conversation

@meem08

@meem08 meem08 commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Enforce CLI Startup and Command Latency Budgets (Issue #698)

Summary

Adds a comprehensive latency budget enforcement system for the StarForge CLI. Tracks representative cold-start and command paths via Criterion benchmarks and flags statistically meaningful regressions in CI.

Changes

Core Enginesrc/utils/latency_budget.rs (new)

  • LatencyBudget / LatencyBudgets — per-command threshold definitions with defaults for 13 CLI paths
  • check_measurement() — checks a single benchmark measurement against its budget (Pass/Fail/Noisy/Skipped/Error)
  • check_latency_budget() — runs a full budget check across all active budgets
  • is_significant_regression() — CV-based z-score statistical regression detection
  • budget_report_to_json() / print_budget_summary() — JSON and human-readable output
  • Environment variable overrides via STARFORGE_LATENCY_BUDGET_<LABEL>
  • 18 unit tests covering primary flow, boundaries, and failures

Criterion Benchmarksbenches/benchmarks.rs (modified)

  • bench_cli_cold_start — measures info, --help, --version cold-start latency
  • bench_cli_command_latency — measures 10 representative command dispatch paths (wallet, network, config, template, deploy, benchmark)
  • bench_latency_budget_check — measures budget check engine overhead

Integration Teststests/latency_budget_test.rs (new, 22 tests)

  • Primary flow: fast/slow measurements, single-budget pass/fail
  • Boundary: zero budget, exact boundary, high CV, CV disabled
  • Failure: zero sample size, missing measurement, inactive budget
  • Environment: env var override, disable, invalid value (with proper save/restore)
  • Statistical regression: detection, no-regression, zero baseline, zero CV
  • JSON report: round-trip validation, print summary

CI Workflow.github/workflows/benchmark-latency.yml (new)

  • Runs latency benchmarks on every push/PR touching relevant files
  • Parses Criterion output via scripts/check-latency-budgets.sh and checks budgets
  • Fails the pipeline on budget violations
  • Uploads Criterion artefact and posts PR comment with budget summary table

Budget Check Scriptscripts/check-latency-budgets.sh (new)

  • Parses Criterion's default stdout format to extract median latencies
  • Mirrors budgets from the Rust code with env var override support
  • Generates JSON report consumed by CI

Documentation

  • CLI_LATENCY_BUDGETS.md — full user-facing docs with budgets table, env var overrides, CI integration, adding budgets, security/compatibility/migration notes
  • BENCHMARKS.md — updated benchmark groups table and CI integration section

Acceptance Criteria Met

  • ✅ Handles invalid input (zero sample size, missing measurements, unparseable env vars)
  • ✅ Handles unsupported environments (inactive budgets, env var "off" toggle)
  • ✅ Handles failure paths (Error status propagation)
  • ✅ Automated tests cover primary flow, boundary cases, and failure cases (22 integration + 18 unit = 40 tests)
  • ✅ User-facing documentation includes compatibility, security, and migration notes

Related Issue

Closes #698

bakinsuy added 2 commits July 29, 2026 14:49
…anle-code#698)

Add latency budget engine, Criterion benchmarks, CI workflow, tests, and documentation.
@drips-wave

drips-wave Bot commented Jul 29, 2026

Copy link
Copy Markdown

@meem08 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Nanle-code
Nanle-code merged commit 771c76d into Nanle-code:master Jul 29, 2026
5 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[2026 Performance] Enforce CLI startup and command latency budgets

2 participants