fix(UniTensor): eliminate overlapping memcpy in combineBonds#1054
Conversation
BlockUniTensorTest.CombineBondForceDoesNotRewriteUserHeldBond aborts under ASan with memcpy-param-overlap at BlockUniTensor.cpp:1996 (verified pre-fix: rank-4 tensor, combining 2 bonds at idor=0, the _inner_to_outer_idx tail-shift memcpy source and destination ranges overlap). combineBonds() has two overlapping-memcpy sites doing a left-shift-by-N into itself, which is undefined behavior whenever the shift distance is smaller than the region being moved: - The _inner_to_outer_idx tail shift (line ~1996) drops indicators.size()-1 elements starting at idor+1 and shifts the tail left; it overlaps whenever the remaining tail is at least twice the shift distance, which the existing 2-bond regression test already hits. Replaced with vector::erase, which performs the equivalent left-shift through a well-defined operation and leaves the vector at the correct final length, so the trailing resize() is no longer needed. - The cb_stride shift-by-one (line ~1964) is latent today (it only overlaps once 3+ bonds are combined at once, which no prior test exercised) but is the same undefined-behavior class in the same function; a commented-out backward loop sat next to it as an abandoned alternative. Replaced with std::copy, which is well-defined for a left-shift because it iterates forward writing to already-read positions. Added CombineBondsThreeBondsPreservesDataAndDimension, a 3-bond combine that exercises the cb_stride site (verified pre-fix: aborts under ASan at BlockUniTensor.cpp:1964 with the same memcpy-param-overlap). It checks the combined bond's dimension against the independently computed product of the three input dimensions, and checks a block's values against the pre-combine block reshaped by hand (row-major merge of its first three axes, matching how Bond_impl::force_combineBond_ orders the combined qnum index) -- independent of the bookkeeping this commit touches. This is a memory-safety-only fix: both replacements perform the exact same left-shift the memcpy calls intended, so combineBonds' numerical output is unchanged. Co-Authored-By: Claude <noreply@anthropic.com>
BlockFermionicUniTensor::combineBonds(vector<cytnx_int64>, force) contains the same two overlapping-memcpy sites as BlockUniTensor::combineBonds (its own inline comment marks it "a copy from BlockUniTensor"): the cb_stride shift-by-one and the _inner_to_outer_idx tail shift. Apply the identical fix here -- std::copy for the former, vector::erase for the latter -- for source hygiene and consistency between the two duplicated implementations, and so the same undefined behavior does not resurface if this function is completed later. This code path is unreachable today: the function starts with an unconditional cytnx_error_msg(true, "...not implemented yet.") behind a "TODOfermion: signflips need to be included!!!" comment, so every public entry point that reaches it (combineBond(labels), combineBonds(labels), combineBonds(indices, force, by_label)) always throws before running any bond-combining logic, including the two lines this commit touches. Lifting that guard to exercise the memcpy fix through the public API would be a functional change to fermionic sign-flip handling, which is out of scope for this memory-safety-only PR. Added CombineBondsStillUnimplemented to pin the current throw behavior, documenting why no live regression test covers the fixed lines here (unlike BlockUniTensor's 3-bond test in the previous commit, which does exercise the fixed code through combineBond_()). Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request replaces unsafe memcpy operations with standard C++ library functions (std::copy and std::vector::erase) in BlockFermionicUniTensor.cpp and BlockUniTensor.cpp to avoid potential undefined behavior during overlapping memory copies. It also adds corresponding unit tests to verify that combining three bonds preserves data and dimensions correctly in BlockUniTensor, and confirms that the unimplemented combineBond_ method in BlockFermionicUniTensor still throws an error as expected. There are no review comments to address.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1054 +/- ##
==========================================
+ Coverage 60.38% 60.52% +0.14%
==========================================
Files 229 229
Lines 31866 31866
Branches 71 71
==========================================
+ Hits 19241 19286 +45
+ Misses 12604 12559 -45
Partials 21 21
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 5 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Same UB class as the BlockUniTensor/BlockFermionicUniTensor fix in this PR: combineBonds' new_shape tail shift used memcpy to left-shift a std::vector into itself. It overlaps whenever the remaining tail is at least twice the shift distance -- combining the first two of four adjacent bonds (idor=0, indicators.size()=2) reproduces the identical overlap geometry. Added CombineBondsFourBondsPreservesDataAndShape, verified failing pre-fix (ASan memcpy-param-overlap abort at DenseUniTensor.cpp:855, same failure signature as the BlockUniTensor case) and passing post-fix. Checks combined shape and values against an independently computed reshape of the original block, not just absence of a crash. Replaced with vector::erase, the same well-defined left-shift used for the BlockUniTensor fix -- memory-safety-only, no numerical change. The erase range is always within bounds: idor + indicators.size() <= new_shape.size() holds because permute_() places the indicators.size() combined bonds contiguously starting at idor within the still-full-rank shape, so the upper erase iterator never passes new_shape.end() even when combining the last bonds in the tensor. Local results, both required CPU debug presets: - debug-openblas-cpu: ctest 1771/1771 passed - debug-mkl-cpu: ctest 1805/1805 passed Co-Authored-By: Claude <noreply@anthropic.com>
852fb09 to
ebffa0a
Compare
Problem
Fixes #1052.
BlockUniTensorTest.CombineBondForceDoesNotRewriteUserHeldBondaborts underASan with
memcpy-param-overlap. The issue's own stack trace points atvec_map(src/utils/vec_map.cpp) as the allocation site of the corruptedbuffer, but that is only where ASan's shadow memory was allocated —
vec_mapitself contains no
memcpyat all and is not the bug.Running the test directly (rather than relying on the issue's captured trace)
gives the real fault stack:
combineBondshas sites in three of the fourUniTensorbackends thatleft-shift a
std::vectorinto itself viamemcpy, which is undefinedbehavior whenever the shift distance is smaller than the region being moved
(source and destination byte ranges overlap):
BlockUniTensor::combineBonds(src/BlockUniTensor.cpp) — two sites:_inner_to_outer_idxtail shift — dropsindicators.size()-1elements starting at
idor+1and shifts the tail left. It overlapswhenever the remaining tail is at least twice the shift distance; the
existing 2-bond regression test already hits this exactly.
cb_strideshift-by-one — latent (only overlaps once 3+ bonds arecombined at once, which no existing test exercised), but the same UB
class in the same function, with a commented-out backward loop sitting
next to it as an abandoned alternative.
BlockFermionicUniTensor::combineBonds(src/BlockFermionicUniTensor.cpp)has the identical two sites (its own comment marks it "a copy from
BlockUniTensor").
DenseUniTensor::combineBonds(src/DenseUniTensor.cpp) has astructurally identical
new_shapetail shift — same overlap condition,same variable names (
idor,indicators). Combining the first two offour adjacent bonds (
idor=0,indicators.size()=2) reproduces the exactsame overlap geometry as the
BlockUniTensorcase.I grepped all four
combineBondsimplementations formemcpyand confirmedthese are the only overlapping sites;
Tensor::combineBonds(dense arrayreshape only, no bond bookkeeping) uses no such shift.
Fix
Every site is replaced with well-defined standard-library equivalents that
perform the exact same left-shift, across
BlockUniTensor.cpp,BlockFermionicUniTensor.cpp, andDenseUniTensor.cpp:cb_strideshift →std::copy(cb_stride.begin() + 1, cb_stride.end(), cb_stride.begin()), well-defined for a left-shift sincestd::copyiterates forward, writing to already-read positions.
_inner_to_outer_idx/new_shapetail shift →vector::erase(begin+idor+1, begin+idor+indicators.size()), which removes exactly the same elementsand leaves the vector at the correct final length, so the trailing
resize(this->rank())is no longer needed.This is a memory-safety-only fix. Every replacement computes the exact
same result the
memcpycalls intended;combineBonds' numerical output isunchanged across all three backends, only the undefined behavior is removed.
BlockFermionicUniTensor::combineBonds(vector, force)is fixedfor source hygiene/consistency with
BlockUniTensor, but is currentlyunreachable through any public API path — it starts with an
unconditional
cytnx_error_msg(true, "...not implemented yet.")behind aTODOfermion: signflips need to be included!!!comment. Lifting that guardto exercise the fix live would be a functional change to fermionic
sign-flip handling, out of scope here; a new test
(
CombineBondsStillUnimplemented) pins the current throw so this scopestatement stays true.
Testing
New tests, each verified to fail pre-fix and pass post-fix:
BlockUniTensorTest.CombineBondsThreeBondsPreservesDataAndDimension— a3-bond combine that exercises the
cb_stridesite. Verified pre-fix: abortsunder ASan at
BlockUniTensor.cpp:1964withmemcpy-param-overlap. Checksthe combined bond's dimension against the independently computed product of
the three input dimensions, and checks block values against the pre-combine
block reshaped by hand (row-major merge of its first three axes, matching
how
Bond_impl::force_combineBond_orders the combined qnum index) —independent of the bookkeeping this PR touches.
BlockFermionicUniTensorTest.CombineBondsStillUnimplemented— documentsthe current unreachability (see above).
DenseUniTensorTest.CombineBondsFourBondsPreservesDataAndShape— a 4-bonddense tensor combining the first two adjacent bonds. Verified pre-fix:
aborts under ASan at
DenseUniTensor.cpp:855withmemcpy-param-overlap(same failure signature as the
BlockUniTensorcase). Checks the combinedshape and values against an independently computed reshape of the original
block, not just absence of a crash.
BlockUniTensorTest.CombineBondForceDoesNotRewriteUserHeldBondre-confirmed pre-fix: aborts under ASan at
BlockUniTensor.cpp:1996withmemcpy-param-overlap; passes post-fix.Local results, both required CPU debug presets:
debug-openblas-cpuctest(test_main)debug-mkl-cpuctest(test_main)No GPU/CUDA code touched (no
#ifdef UNI_GPU, nosrc/backend/linalg_internal_gpu/),so the CUDA compile-check gate does not apply. No
cytnx/,pybind/, orpytests/changes, sopytest pytests/does not apply.Opened as a draft while the fix scope (
BlockUniTensor+BlockFermionicUniTensor+DenseUniTensor) gets a look.