Use sorted const table + binary search for embedded get() - #24
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #24 +/- ##
==========================================
- Coverage 79.46% 75.97% -3.50%
==========================================
Files 9 9
Lines 448 566 +118
==========================================
+ Hits 356 430 +74
- Misses 92 136 +44 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Release/embedded codegen previously emitted a giant `match file_path` arm per file (O(n) per lookup, large LLVM IR). Replace it with a sorted `const ENTRIES: &[(&str, EmbeddedFile)]` table looked up at runtime with `slice::binary_search_by_key`. Entries are sorted by path at macro-expansion time so the runtime slice is guaranteed sorted. `EmbeddedFile::__internal_make` is now a `const fn` so the table can live in a `const`. Public API (`Asset::get`) is unchanged, and debug/dynamic mode is untouched. Ported from upstream rust-embed commit ce93f7e. Claude-Session: https://claude.ai/code/session_01MixCkrYqoVEZZbPRRNtL3M
Adds a Criterion benchmark (benches/embed_lookup.rs) that measures Asset::get() at the first, middle, last, and a missing key over a large embedded folder, plus a bench_size example for measuring the embedded binary size. The fixture folder is generated at build time by build.rs behind the non-default bench-fixtures feature (N defaults to 2000, overridable via BENCH_FIXTURE_COUNT), written into OUT_DIR, and exposed via cargo:rustc-env=BENCH_FIXTURE_DIR. When the feature is off, build.rs is a no-op, so normal and published builds are unaffected. Claude-Session: https://claude.ai/code/session_01MixCkrYqoVEZZbPRRNtL3M
- Reformat examples/bench_size.rs per rustfmt. - Add codecov.yml ignoring benches/build.rs/examples and allowing a 1% project threshold, so the bench scaffolding (not exercised by tests) doesn't fail the coverage check. Claude-Session: https://claude.ai/code/session_01MixCkrYqoVEZZbPRRNtL3M
SeriousBug
force-pushed
the
binary-search-embed-lookup
branch
from
June 30, 2026 03:04
3bbc5f0 to
36ce6f6
Compare
The embedded get() table is now a const whose entries call the const fn EmbeddedFile::__internal_make, evaluated at compile time. llvm-cov can't see const-evaluated code as executed, so it reports __internal_make as uncovered and project coverage drops ~4.5%. The drop is a measurement artifact, not a real regression, so project coverage is informational. Claude-Session: https://claude.ai/code/session_01MixCkrYqoVEZZbPRRNtL3M
Merged
SeriousBug
added a commit
that referenced
this pull request
Jun 30, 2026
* Bump version to 11.4.0 Releases changes since 11.3.0: - Add allow_missing derive attribute (#22) - Convert macro panics to compile errors (#21) - Use sorted const table + binary search for embedded get() (#24) - Update sha2 dependency to 0.11 (#25) * Pin time below 0.3.52 in dev-dependencies time 0.3.52 changed the Parsable::parse signature, which breaks cookie 0.16.2 (pulled in transitively via the actix-web dev-dependency), failing the CI doc test job. Constrain to the 0.3 series below 0.3.52 so it unifies with cookie's requirement.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the "store file contents statically + binary-search lookup" optimization from upstream rust-embed (commit ce93f7e).
The release/embedded codegen previously emitted a giant
match file_path { "a" => ..., "b" => ... }with one arm per embedded file. That is O(n) per lookup and generates a lot of LLVM IR for large asset folders.This replaces it with a sorted const table looked up via binary search:
generate_embed_implnow collects(path, embed-tokens)pairs, sorts them by path at macro-expansion time, and emitsconst ENTRIES: &[(&str, EmbeddedFile)].get()doesENTRIES.binary_search_by_key(&path, |e| e.0).ok().map(...)(O(log n)).EmbeddedFile::__internal_makeis now aconst fnso the table can be aconst.The closure/index bindings use unlikely identifiers (
__rust_embed_*) becausequote!-emitted identifiers in derive macros resolve at the call site and would otherwise collide with user items (the actix example defines a unit struct namedindex).Compatibility
Asset::get(path)) is unchanged.Testing
cargo build/cargo testpass.cargo test --all-featurespasses (exercises the embed path viaalways-embedand compiles the actix example).cargo test --releasefailure intests/dynamic.rsis unrelated (those tests require debug mode) and reproduces on master.Benchmarks
Setup: Criterion 0.7 benchmark (
benches/embed_lookup.rs) plus abench_sizeexample. Both embed a generated fixture folder of N small (~84 byte) files
produced at build time by
build.rsbehind the non-defaultbench-fixturesfeature. Runtime numbers use N=2000; binary sizes are reported for N=2000 and
N=5000. Measured locally on an Apple Silicon Mac (release build,
debug_assertionsoff, so the embeddedget()path is exercised). Masternumbers come from a temporary branch combining master's source with the
identical bench code.
Runtime
get()(Criterion mean, N=2000)Binary search is position-independent (~42-49 ns regardless of key), while the
matchlookup is O(n)-ish and worst on a middle or missing key. The largest winis the missing-key case (13.5x).
Stripped binary size (
bench_sizeexample)Compile time (recompile of the example crate, N=2000)
Takeaway
The win here is runtime and compile time, not binary size.
get()lookup is3x-13.5x faster and, more importantly, flat with respect to key position and
folder size, removing the linear-scan worst case for misses and middle keys.
The codegen/IR reduction shows up as a ~31% faster recompile of the embedding
crate at N=2000. The final stripped binary is slightly larger (+3.83% at N=2000,
+2.34% at N=5000): the sorted
(&str, EmbeddedFile)const table plus the keyarray adds a bit of
.rodatathat the oldmatchdid not materialize asexplicitly. If binary size is the priority this is a small regression; if lookup
latency or compile time matters, the trade is clearly favorable.
https://claude.ai/code/session_01MixCkrYqoVEZZbPRRNtL3M