chore: DO NOT MERGE - benchmark tsam develop-v4 vs v3.4.x#110
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
BenchmarksΔ% vs base, one table sorted by biggest change first Full tablebenchmarks/test_bench_aggregate.py
|
The head run leaves an untracked uv.lock behind, and uv keeps a satisfying lockfile as-is — so a head that changes dependencies leaked its resolution into the base run (both sides benchmarked the head's deps). Remove it before the base run and log each side's resolved tsam version for transparency. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DO NOT MERGE — experiment to measure tsam v4 against v3.4.x with the head-vs-base benchmark workflow. Head resolves tsam from the develop-v4 git branch; base (main) resolves the released v3 line. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2da38ec to
c8d10a3
Compare
DO NOT MERGE — measurement experiment, not a dependency change proposal.
Pins tsam to the
develop-v4git branch via[tool.uv.sources]so the Benchmarks workflow's head-vs-base comparison measures tsam v4 against released v3.4.x on the identical suite (no lockfile is committed, so base resolves the v3 release line while head resolves develop-v4). Includes the #109 benchmark suite commits since the workflow and cases must exist on the head.Early local signal (90-day config cases, min of 5):
dist_global: ~87 ms → ~53 ms (~1.6× faster — consistent with the v4 rework arounddurationRepresentation)medoid/mean/dist_cluster: ~35 ms → ~19-21 ms (~1.7× faster)_accuracy_metrics/_reconstructed_datainternals, so the eager accuracy cost (62% of a v3 run) may have moved or vanished — the CI table will show where.All 542 wrapper tests pass against develop-v4 locally (v4-only features like
preserve_n_clustersfrom FZJ-IEK3-VSA/tsam#410 are not yet on that branch — the wrapper's feature detection correctly reports them absent).🤖 Generated with Claude Code