docs: add benchmark results matrix to README#769
Conversation
Adds a Benchmarks section to the front-page README with a 10-row matrix covering (backend, OS, accelerator) combinations against three grid sizes (64³, 128³, 256³). All cells ship empty (em-dash) as a scaffold — contributors fill rows via follow-up commits with their hardware, k-wave-python version, and BINARY_VERSION. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
it would be great if you two could add entries here! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #769 +/- ##
=======================================
Coverage 75.83% 75.83%
=======================================
Files 58 58
Lines 8274 8274
Branches 1614 1614
=======================================
Hits 6275 6275
Misses 1373 1373
Partials 626 626
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Measured on Apple M1 MacBook Air (8 GB) against k-wave-python 0.6.3rc1: 64³ = 81 s (python backend, CPU, 1000 timesteps, 3 averages). 128³/256³ left empty — 256³ exceeds 8 GB RAM on this machine. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
|
||
| ## Benchmarks | ||
|
|
||
| Reference runtimes for the 3D scaling benchmark in [`benchmarks/benchmark.py`](../benchmarks/README.md). Values are total elapsed seconds for a single default run (3D initial-value problem, heterogeneous absorbing medium, 1000 timesteps, averaged over 3 repeats). |
There was a problem hiding this comment.
Benchmark Link Opens Wrong File
The link is labeled benchmarks/benchmark.py, but it opens benchmarks/README.md. Readers clicking it to inspect the benchmark implementation are sent to the instructions instead; the instructions already have a separate link below.
| Reference runtimes for the 3D scaling benchmark in [`benchmarks/benchmark.py`](../benchmarks/README.md). Values are total elapsed seconds for a single default run (3D initial-value problem, heterogeneous absorbing medium, 1000 timesteps, averaged over 3 repeats). | |
| Reference runtimes for the 3D scaling benchmark in [`benchmarks/benchmark.py`](../benchmarks/benchmark.py). Values are total elapsed seconds for a single default run (3D initial-value problem, heterogeneous absorbing medium, 1000 timesteps, averaged over 3 repeats). |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| | cpp | Windows | CPU (OMP) | — | — | — | — | | ||
| | cpp | Windows | NVIDIA GPU (CUDA) | — | — | — | — | | ||
|
|
||
| Contributions welcome — open a PR filling a row with your k-wave-python version, and (for `cpp` backend) `BINARY_VERSION`. See [`benchmarks/README.md`](../benchmarks/README.md) for the reproducer command. |
There was a problem hiding this comment.
Reproducer Runs Excluded Grid Sizes
The linked default command runs all 12 benchmark cases, including the large cases excluded from this matrix because workstations may run out of memory. A contributor following this instruction to fill a 64³, 128³, or 256³ cell can therefore spend time on unrelated cases or encounter an out-of-memory failure before the command completes; provide a bounded command that reproduces the listed sizes.
Summary
Adds a
## Benchmarkssection to the front-page README (docs/README.md) with a 10-row matrix covering(backend, OS, accelerator)combinations against three grid sizes: 64³, 128³, 256³.All cells ship empty (em-dash) as a scaffold. The plan is for contributors to fill rows via follow-up commits on this branch:
Grid-size columns are drawn from the isotropic sizes in the default
x/y/z_scale_arrayinbenchmarks/helpers.py(indices 0/3/6/9 = 32³/64³/128³/256³, dropped 32³ as too small to be interesting). 512³ was excluded because many workstations OOM.Rows omitted intentionally:
python/cpp+ macOS + GPU — no CUDA on macOS.cpp+ Intel Mac + CPU — rare; can be added when someone runs it.Reproducer command lives in
benchmarks/README.md; the section links to it.Test plan
docs/README.mdis thereadme =inpyproject.toml)🤖 Generated with Claude Code
Greptile Summary
This PR adds benchmark results to the public README. The main changes are:
Confidence Score: 5/5
The documentation is mergeable after small fixes to the benchmark links and reproduction guidance.
docs/README.md
Important Files Changed
Reviews (1): Last reviewed commit: "docs: fill python/macOS/CPU 64³ benchmar..." | Re-trigger Greptile