Skip to content

blasgraph: route SIMD through ndarray::simd, gate hand-rolled fallback#666

Closed
AdaWorldAPI wants to merge 1 commit into
mainfrom
claude/simd-bitwise-hygiene
Closed

blasgraph: route SIMD through ndarray::simd, gate hand-rolled fallback#666
AdaWorldAPI wants to merge 1 commit into
mainfrom
claude/simd-bitwise-hygiene

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Closed per operator directive — do not touch lance-graph. No changes to lance-graph in this session. (The SIMD-hygiene idea can be revisited separately if wanted.)

Summary by CodeRabbit

  • Bug Fixes
    • Improved distance and popcount handling when enhanced SIMD support is enabled, ensuring the faster path is used consistently.
    • Tightened fallback compilation so scalar and platform-specific alternatives are only included when needed, reducing the risk of conflicting implementations.

Two SIMD-hygiene fixes in ndarray_bridge:

1. Consume ndarray::simd::{hamming_distance_raw,popcount_raw} (the W1a
   polyfill facade) instead of reaching into ndarray::hpc::bitwise::*
   directly. 'All SIMD from ndarray::simd' — hpc is the impl home, simd
   is the contract surface consumers use.

2. The 8 hand-rolled unsafe intrinsics (hamming/popcount scalar+avx2+
   avx512bw+avx512vpopcntdq) were compiled unconditionally but only
   CALLED under #[cfg(not(feature = ndarray-hpc))], so with ndarray-hpc
   on (production) they were dead code — hand-rolled unsafe SIMD sitting
   in a build where ndarray's SIMD is the live path. Gate each definition
   behind not(ndarray-hpc) so they vanish when ndarray is present and the
   only SIMD is ndarray's; they still serve the genuine no-ndarray
   (minimal/wasm) fallback.

Verified: default build (ndarray-hpc) finishes clean with no
ndarray_bridge dead-code warnings; the no-default-features fallback path
still compiles the intrinsics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RbXniqYUYgwUsRrEnRRiid
@cursor

cursor Bot commented Jul 8, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_de8278a8-7425-46fc-b774-47d84410b7fe)

@AdaWorldAPI AdaWorldAPI closed this Jul 8, 2026
@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 32aafbbf-698a-4001-850e-0e6798ce7c84

📥 Commits

Reviewing files that changed from the base of the PR and between 26d2cf5 and 55269e1.

📒 Files selected for processing (1)
  • crates/lance-graph/src/graph/blasgraph/ndarray_bridge.rs

📝 Walkthrough

Walkthrough

The ndarray-lance-graph SIMD dispatch bridge now calls ndarray::simd::*_raw functions for hamming distance and popcount when the ndarray-hpc feature is enabled. Scalar and x86_64 intrinsic fallback implementations are now compiled only when ndarray-hpc is disabled.

Changes

SIMD Dispatch Bridging Update

Layer / File(s) Summary
Dispatch to ndarray::simd raw functions
crates/lance-graph/src/graph/blasgraph/ndarray_bridge.rs
dispatch_hamming and dispatch_popcount now call ndarray::simd::hamming_distance_raw and ndarray::simd::popcount_raw respectively when ndarray-hpc is enabled, with updated documentation.
Fallback implementations gated behind not(ndarray-hpc)
crates/lance-graph/src/graph/blasgraph/ndarray_bridge.rs
Scalar helpers (hamming_scalar, popcount_scalar) and x86_64 AVX2/AVX-512BW/AVX-512 VPOPCNTDQ intrinsic implementations are now compiled only under #[cfg(not(feature = "ndarray-hpc"))].

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

A rabbit hops through feature flags so neat,
Swapping raw calls with a nimble feat.
Scalar burrows sealed when SIMD's in play,
Intrinsics tucked snug for another day.
Hop, cfg, hop — the compile paths align! 🐇✨

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@AdaWorldAPI AdaWorldAPI deleted the claude/simd-bitwise-hygiene branch July 8, 2026 09:18

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55269e195b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// ---------------------------------------------------------------------------

/// Scalar Hamming distance: XOR each u64 pair and sum `count_ones()`.
#[cfg(not(feature = "ndarray-hpc"))]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep scalar helpers available to default tests

With the default feature set, ndarray-hpc is enabled in crates/lance-graph/Cargo.toml, so this cfg removes hamming_scalar from the test build; the same newly added cfg on popcount_scalar has the same effect. The tests in this module still call those helpers directly in test_scalar_hamming_small and test_scalar_popcount_small, so cargo test --manifest-path crates/lance-graph/Cargo.toml will fail name resolution before running the CI-gated default tests. Please either keep these helpers compiled under cfg(test) or gate the scalar-only tests the same way.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants