Skip to content

perf: isolate benchmark and e2e dependencies - #39

Open
NickNYU wants to merge 1 commit into
kvcache-ai:mainfrom
NickNYU:autoresearch/issue-8-p1
Open

perf: isolate benchmark and e2e dependencies#39
NickNYU wants to merge 1 commit into
kvcache-ai:mainfrom
NickNYU:autoresearch/issue-8-p1

Conversation

@NickNYU

@NickNYU NickNYU commented Jul 28, 2026

Copy link
Copy Markdown

Summary

  • move the four Criterion benchmarks into a dedicated agentenv-benchmarks workspace package
  • move the Docker/MinIO OSS snapshot test into agentenv-e2e-tests
  • keep the existing Makefile entry points while removing Criterion and test-support from the root package dev-dependencies
  • update internal documentation for the relocated OSS test

Part of #8 (P1).

Why

Cargo activates a package's dev-dependencies for library test builds even when the test target does not use them. Keeping Criterion and agentenv-test-support on the root package caused cargo test -p agentenv --lib --no-run to compile benchmark tooling, AWS SDK crates, and testcontainers.

Dedicated workspace packages preserve the benchmark/e2e targets while moving those costs behind explicit package selection.

Measured result

Same isolated VM for both runs: Ubuntu 22.04 ARM64, 8 vCPU, 16 GiB RAM, rustc 1.97.1. Commands used a fresh target directory and a warm Cargo source cache.

cargo test -p agentenv --lib --no-run --locked Before After Change
wall time 243.41s 221.34s -22.07s (-9.1%)
target size 7.4 GiB 6.6 GiB about -0.8 GiB
compiler artifacts 749 649 -100 (-13.4%)

The compiler-artifact gate found all six P1 dependencies before the change and none afterward:

  • criterion
  • agentenv-test-support
  • aws-sdk-s3
  • aws-config
  • testcontainers
  • testcontainers-modules

The ARM64 timings are screening data rather than a replacement for the issue's x86_64 baseline; the dependency isolation result itself comes from Cargo's compiled artifact stream and dependency tree.

Verification

  • cargo fmt --all -- --check
  • cargo test -p agentenv --lib --locked — 633 passed, 0 failed, 2 ignored
  • cargo test -p agentenv-e2e-tests --test snapshot_oss_e2e_test --no-run --locked
  • cargo test -p agentenv-benchmarks --benches --no-run --locked — 4 executables
  • cargo clippy -p agentenv -p agentenv-benchmarks -p agentenv-e2e-tests --all-targets --locked -- -D warnings
  • root dependency-tree gate — 0 forbidden dependencies
  • Makefile dry-run for benchmark and OSS e2e entry points

Not run

  • Docker/MinIO-backed ignored e2e tests at runtime
  • KVM/capability-dependent benchmark execution

Move Criterion benchmarks and Docker-backed OSS e2e coverage into dedicated workspace packages so root library tests no longer activate AWS SDK, testcontainers, or Criterion.

Constraint: Preserve existing Makefile entry points and ignored e2e behavior
Rejected: Feature-gate root dev-dependencies | Cargo still activates package dev-dependencies for library test builds
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep benchmark and Docker-backed e2e dependencies out of the root agentenv dev-dependencies
Tested: cargo fmt --all -- --check; cargo test -p agentenv --lib --locked (633 passed); e2e and four benchmark targets --no-run; targeted clippy --all-targets -D warnings; dependency gate forbidden=0
Not-tested: Docker-backed ignored e2e runtime and KVM/capability benchmark execution
Related: kvcache-ai#8
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

🔍 OpenCodeReview found 1 issue(s) in this PR.

  • ✅ Successfully posted inline: 1 comment(s)

Comment on lines +7 to +10
[dependencies]
agentenv = { path = "../.." }
anyhow = "1.0.100"
criterion = { version = "0.8", features = ["async_tokio"] }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This package defines only benchmark targets, so these crates are benchmark-only dependencies. Keeping them under [dependencies] exposes them as normal package dependencies and broadens normal dependency/feature resolution unnecessarily. Move the benchmark dependency list to [dev-dependencies], which Cargo makes available to bench targets.

Suggestion:

Suggested change
[dependencies]
agentenv = { path = "../.." }
anyhow = "1.0.100"
criterion = { version = "0.8", features = ["async_tokio"] }
[dev-dependencies]
agentenv = { path = "../.." }
anyhow = "1.0.100"
criterion = { version = "0.8", features = ["async_tokio"] }

@LSX-s-Software LSX-s-Software left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing. Could you please fix the comment posted by the AI reviewer?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants