perf: isolate benchmark and e2e dependencies - #39
Open
NickNYU wants to merge 1 commit into
Open
Conversation
Move Criterion benchmarks and Docker-backed OSS e2e coverage into dedicated workspace packages so root library tests no longer activate AWS SDK, testcontainers, or Criterion. Constraint: Preserve existing Makefile entry points and ignored e2e behavior Rejected: Feature-gate root dev-dependencies | Cargo still activates package dev-dependencies for library test builds Confidence: high Scope-risk: moderate Reversibility: clean Directive: Keep benchmark and Docker-backed e2e dependencies out of the root agentenv dev-dependencies Tested: cargo fmt --all -- --check; cargo test -p agentenv --lib --locked (633 passed); e2e and four benchmark targets --no-run; targeted clippy --all-targets -D warnings; dependency gate forbidden=0 Not-tested: Docker-backed ignored e2e runtime and KVM/capability benchmark execution Related: kvcache-ai#8
|
🔍 OpenCodeReview found 1 issue(s) in this PR.
|
Comment on lines
+7
to
+10
| [dependencies] | ||
| agentenv = { path = "../.." } | ||
| anyhow = "1.0.100" | ||
| criterion = { version = "0.8", features = ["async_tokio"] } |
There was a problem hiding this comment.
This package defines only benchmark targets, so these crates are benchmark-only dependencies. Keeping them under [dependencies] exposes them as normal package dependencies and broadens normal dependency/feature resolution unnecessarily. Move the benchmark dependency list to [dev-dependencies], which Cargo makes available to bench targets.
Suggestion:
Suggested change
| [dependencies] | |
| agentenv = { path = "../.." } | |
| anyhow = "1.0.100" | |
| criterion = { version = "0.8", features = ["async_tokio"] } | |
| [dev-dependencies] | |
| agentenv = { path = "../.." } | |
| anyhow = "1.0.100" | |
| criterion = { version = "0.8", features = ["async_tokio"] } |
LSX-s-Software
left a comment
Member
There was a problem hiding this comment.
Thanks for contributing. Could you please fix the comment posted by the AI reviewer?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
agentenv-benchmarksworkspace packageagentenv-e2e-testsPart of #8 (P1).
Why
Cargo activates a package's dev-dependencies for library test builds even when the test target does not use them. Keeping Criterion and
agentenv-test-supporton the root package causedcargo test -p agentenv --lib --no-runto compile benchmark tooling, AWS SDK crates, and testcontainers.Dedicated workspace packages preserve the benchmark/e2e targets while moving those costs behind explicit package selection.
Measured result
Same isolated VM for both runs: Ubuntu 22.04 ARM64, 8 vCPU, 16 GiB RAM, rustc 1.97.1. Commands used a fresh target directory and a warm Cargo source cache.
cargo test -p agentenv --lib --no-run --lockedThe compiler-artifact gate found all six P1 dependencies before the change and none afterward:
criterionagentenv-test-supportaws-sdk-s3aws-configtestcontainerstestcontainers-modulesThe ARM64 timings are screening data rather than a replacement for the issue's x86_64 baseline; the dependency isolation result itself comes from Cargo's compiled artifact stream and dependency tree.
Verification
cargo fmt --all -- --checkcargo test -p agentenv --lib --locked— 633 passed, 0 failed, 2 ignoredcargo test -p agentenv-e2e-tests --test snapshot_oss_e2e_test --no-run --lockedcargo test -p agentenv-benchmarks --benches --no-run --locked— 4 executablescargo clippy -p agentenv -p agentenv-benchmarks -p agentenv-e2e-tests --all-targets --locked -- -D warningsNot run