Skip to content

fix(UniTensor): eliminate overlapping memcpy in combineBonds#1054

Merged
IvanaGyro merged 3 commits into
masterfrom
claude/issue-1052-fix-combinebonds-overlap
Jul 17, 2026
Merged

fix(UniTensor): eliminate overlapping memcpy in combineBonds#1054
IvanaGyro merged 3 commits into
masterfrom
claude/issue-1052-fix-combinebonds-overlap

Conversation

@IvanaGyro

@IvanaGyro IvanaGyro commented Jul 15, 2026

Copy link
Copy Markdown
Member

Problem

Fixes #1052.

BlockUniTensorTest.CombineBondForceDoesNotRewriteUserHeldBond aborts under
ASan with memcpy-param-overlap. The issue's own stack trace points at
vec_map (src/utils/vec_map.cpp) as the allocation site of the corrupted
buffer, but that is only where ASan's shadow memory was allocated — vec_map
itself contains no memcpy at all and is not the bug.

Running the test directly (rather than relying on the issue's captured trace)
gives the real fault stack:

==ERROR: AddressSanitizer: memcpy-param-overlap
    #1 cytnx::BlockUniTensor::combineBonds(...)  src/BlockUniTensor.cpp:1996
    #2 cytnx::BlockUniTensor::combineBond(...)   src/BlockUniTensor.cpp:2027
    #3 cytnx::UniTensor::combineBond_(...)       include/UniTensor.hpp:4902

combineBonds has sites in three of the four UniTensor backends that
left-shift a std::vector into itself via memcpy, which is undefined
behavior whenever the shift distance is smaller than the region being moved
(source and destination byte ranges overlap):

  • BlockUniTensor::combineBonds (src/BlockUniTensor.cpp) — two sites:
    1. _inner_to_outer_idx tail shift — drops indicators.size()-1
      elements starting at idor+1 and shifts the tail left. It overlaps
      whenever the remaining tail is at least twice the shift distance; the
      existing 2-bond regression test already hits this exactly.
    2. cb_stride shift-by-one — latent (only overlaps once 3+ bonds are
      combined at once, which no existing test exercised), but the same UB
      class in the same function, with a commented-out backward loop sitting
      next to it as an abandoned alternative.
  • BlockFermionicUniTensor::combineBonds (src/BlockFermionicUniTensor.cpp)
    has the identical two sites (its own comment marks it "a copy from
    BlockUniTensor").
  • DenseUniTensor::combineBonds (src/DenseUniTensor.cpp) has a
    structurally identical new_shape tail shift — same overlap condition,
    same variable names (idor, indicators). Combining the first two of
    four adjacent bonds (idor=0, indicators.size()=2) reproduces the exact
    same overlap geometry as the BlockUniTensor case.

I grepped all four combineBonds implementations for memcpy and confirmed
these are the only overlapping sites; Tensor::combineBonds (dense array
reshape only, no bond bookkeeping) uses no such shift.

Fix

Every site is replaced with well-defined standard-library equivalents that
perform the exact same left-shift, across BlockUniTensor.cpp,
BlockFermionicUniTensor.cpp, and DenseUniTensor.cpp:

  • cb_stride shift → std::copy(cb_stride.begin() + 1, cb_stride.end(), cb_stride.begin()), well-defined for a left-shift since std::copy
    iterates forward, writing to already-read positions.
  • _inner_to_outer_idx / new_shape tail shift → vector::erase(begin+idor+1, begin+idor+indicators.size()), which removes exactly the same elements
    and leaves the vector at the correct final length, so the trailing
    resize(this->rank()) is no longer needed.

This is a memory-safety-only fix. Every replacement computes the exact
same result the memcpy calls intended; combineBonds' numerical output is
unchanged across all three backends, only the undefined behavior is removed.

BlockFermionicUniTensor::combineBonds(vector, force) is fixed
for source hygiene/consistency with BlockUniTensor, but is currently
unreachable through any public API path — it starts with an
unconditional cytnx_error_msg(true, "...not implemented yet.") behind a
TODOfermion: signflips need to be included!!! comment. Lifting that guard
to exercise the fix live would be a functional change to fermionic
sign-flip handling, out of scope here; a new test
(CombineBondsStillUnimplemented) pins the current throw so this scope
statement stays true.

Testing

New tests, each verified to fail pre-fix and pass post-fix:

  • BlockUniTensorTest.CombineBondsThreeBondsPreservesDataAndDimension — a
    3-bond combine that exercises the cb_stride site. Verified pre-fix: aborts
    under ASan at BlockUniTensor.cpp:1964 with memcpy-param-overlap. Checks
    the combined bond's dimension against the independently computed product of
    the three input dimensions, and checks block values against the pre-combine
    block reshaped by hand (row-major merge of its first three axes, matching
    how Bond_impl::force_combineBond_ orders the combined qnum index) —
    independent of the bookkeeping this PR touches.
  • BlockFermionicUniTensorTest.CombineBondsStillUnimplemented — documents
    the current unreachability (see above).
  • DenseUniTensorTest.CombineBondsFourBondsPreservesDataAndShape — a 4-bond
    dense tensor combining the first two adjacent bonds. Verified pre-fix:
    aborts under ASan at DenseUniTensor.cpp:855 with memcpy-param-overlap
    (same failure signature as the BlockUniTensor case). Checks the combined
    shape and values against an independently computed reshape of the original
    block, not just absence of a crash.
  • The pre-existing BlockUniTensorTest.CombineBondForceDoesNotRewriteUserHeldBond
    re-confirmed pre-fix: aborts under ASan at BlockUniTensor.cpp:1996 with
    memcpy-param-overlap; passes post-fix.

Local results, both required CPU debug presets:

Preset Suite Result
debug-openblas-cpu ctest (test_main) 1771/1771 passed
debug-mkl-cpu ctest (test_main) 1805/1805 passed

No GPU/CUDA code touched (no #ifdef UNI_GPU, no src/backend/linalg_internal_gpu/),
so the CUDA compile-check gate does not apply. No cytnx/, pybind/, or
pytests/ changes, so pytest pytests/ does not apply.

Opened as a draft while the fix scope (BlockUniTensor +
BlockFermionicUniTensor + DenseUniTensor) gets a look.

IvanaGyro and others added 2 commits July 15, 2026 03:13
BlockUniTensorTest.CombineBondForceDoesNotRewriteUserHeldBond aborts
under ASan with memcpy-param-overlap at BlockUniTensor.cpp:1996
(verified pre-fix: rank-4 tensor, combining 2 bonds at idor=0, the
_inner_to_outer_idx tail-shift memcpy source and destination ranges
overlap). combineBonds() has two overlapping-memcpy sites doing a
left-shift-by-N into itself, which is undefined behavior whenever the
shift distance is smaller than the region being moved:

- The _inner_to_outer_idx tail shift (line ~1996) drops
  indicators.size()-1 elements starting at idor+1 and shifts the tail
  left; it overlaps whenever the remaining tail is at least twice the
  shift distance, which the existing 2-bond regression test already
  hits. Replaced with vector::erase, which performs the equivalent
  left-shift through a well-defined operation and leaves the vector at
  the correct final length, so the trailing resize() is no longer
  needed.

- The cb_stride shift-by-one (line ~1964) is latent today (it only
  overlaps once 3+ bonds are combined at once, which no prior test
  exercised) but is the same undefined-behavior class in the same
  function; a commented-out backward loop sat next to it as an
  abandoned alternative. Replaced with std::copy, which is
  well-defined for a left-shift because it iterates forward writing to
  already-read positions.

Added CombineBondsThreeBondsPreservesDataAndDimension, a 3-bond combine
that exercises the cb_stride site (verified pre-fix: aborts under ASan
at BlockUniTensor.cpp:1964 with the same memcpy-param-overlap). It
checks the combined bond's dimension against the independently
computed product of the three input dimensions, and checks a block's
values against the pre-combine block reshaped by hand (row-major merge
of its first three axes, matching how Bond_impl::force_combineBond_
orders the combined qnum index) -- independent of the bookkeeping this
commit touches.

This is a memory-safety-only fix: both replacements perform the exact
same left-shift the memcpy calls intended, so combineBonds' numerical
output is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
BlockFermionicUniTensor::combineBonds(vector<cytnx_int64>, force)
contains the same two overlapping-memcpy sites as
BlockUniTensor::combineBonds (its own inline comment marks it "a copy
from BlockUniTensor"): the cb_stride shift-by-one and the
_inner_to_outer_idx tail shift. Apply the identical fix here --
std::copy for the former, vector::erase for the latter -- for source
hygiene and consistency between the two duplicated implementations,
and so the same undefined behavior does not resurface if this function
is completed later.

This code path is unreachable today: the function starts with an
unconditional cytnx_error_msg(true, "...not implemented yet.") behind
a "TODOfermion: signflips need to be included!!!" comment, so every
public entry point that reaches it (combineBond(labels),
combineBonds(labels), combineBonds(indices, force, by_label)) always
throws before running any bond-combining logic, including the two
lines this commit touches. Lifting that guard to exercise the memcpy
fix through the public API would be a functional change to fermionic
sign-flip handling, which is out of scope for this memory-safety-only
PR. Added CombineBondsStillUnimplemented to pin the current throw
behavior, documenting why no live regression test covers the fixed
lines here (unlike BlockUniTensor's 3-bond test in the previous
commit, which does exercise the fixed code through combineBond_()).

Co-Authored-By: Claude <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces unsafe memcpy operations with standard C++ library functions (std::copy and std::vector::erase) in BlockFermionicUniTensor.cpp and BlockUniTensor.cpp to avoid potential undefined behavior during overlapping memory copies. It also adds corresponding unit tests to verify that combining three bonds preserves data and dimensions correctly in BlockUniTensor, and confirms that the unimplemented combineBond_ method in BlockFermionicUniTensor still throws an error as expected. There are no review comments to address.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.52%. Comparing base (6600059) to head (ebffa0a).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1054      +/-   ##
==========================================
+ Coverage   60.38%   60.52%   +0.14%     
==========================================
  Files         229      229              
  Lines       31866    31866              
  Branches       71       71              
==========================================
+ Hits        19241    19286      +45     
+ Misses      12604    12559      -45     
  Partials       21       21              
Flag Coverage Δ
cpp 60.47% <100.00%> (+0.14%) ⬆️
python 64.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
C++ backend 58.07% <100.00%> (+0.03%) ⬆️
Python bindings 75.01% <ø> (+0.75%) ⬆️
Python package 64.13% <ø> (ø)
Files with missing lines Coverage Δ
src/BlockFermionicUniTensor.cpp 65.67% <ø> (+0.84%) ⬆️
src/BlockUniTensor.cpp 75.26% <100.00%> (-0.10%) ⬇️
src/DenseUniTensor.cpp 86.77% <100.00%> (-0.05%) ⬇️

... and 5 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6600059...ebffa0a. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@IvanaGyro IvanaGyro changed the title fix(BlockUniTensor): eliminate overlapping memcpy in combineBonds fix(UniTensor): eliminate overlapping memcpy in combineBonds Jul 15, 2026
@IvanaGyro
IvanaGyro marked this pull request as ready for review July 15, 2026 04:30
Same UB class as the BlockUniTensor/BlockFermionicUniTensor fix in this
PR: combineBonds' new_shape tail shift used memcpy to left-shift a
std::vector into itself. It overlaps whenever the remaining tail is at
least twice the shift distance -- combining the first two of four
adjacent bonds (idor=0, indicators.size()=2) reproduces the identical
overlap geometry.

Added CombineBondsFourBondsPreservesDataAndShape, verified failing
pre-fix (ASan memcpy-param-overlap abort at DenseUniTensor.cpp:855,
same failure signature as the BlockUniTensor case) and passing
post-fix. Checks combined shape and values against an independently
computed reshape of the original block, not just absence of a crash.

Replaced with vector::erase, the same well-defined left-shift used for
the BlockUniTensor fix -- memory-safety-only, no numerical change. The
erase range is always within bounds: idor + indicators.size() <=
new_shape.size() holds because permute_() places the indicators.size()
combined bonds contiguously starting at idor within the still-full-rank
shape, so the upper erase iterator never passes new_shape.end() even
when combining the last bonds in the tensor.

Local results, both required CPU debug presets:
- debug-openblas-cpu: ctest 1771/1771 passed
- debug-mkl-cpu: ctest 1805/1805 passed

Co-Authored-By: Claude <noreply@anthropic.com>
@IvanaGyro
IvanaGyro force-pushed the claude/issue-1052-fix-combinebonds-overlap branch from 852fb09 to ebffa0a Compare July 16, 2026 03:04
@IvanaGyro
IvanaGyro requested review from pcchen and yingjerkao July 16, 2026 08:40

@yingjerkao yingjerkao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@IvanaGyro
IvanaGyro merged commit 89325cd into master Jul 17, 2026
19 checks passed
@IvanaGyro
IvanaGyro deleted the claude/issue-1052-fix-combinebonds-overlap branch July 17, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ASan memcpy-param-overlap in BlockUniTensor::permute_ (via combineBond), CombineBondForceDoesNotRewriteUserHeldBond

2 participants