Conversation
Stop actions/checkout from leaving its short-lived ghs_* installation token in .git/config as http.<host>.extraheader after the workflow. On self-hosted runners (build-and-test, tag_release build, tutorial build) the work tree persists between jobs, so an interrupted or killed job can leave the (later-expired) token behind. None of the existing checkouts run authenticated git operations afterwards, so disabling persistence is safe. Public submodules are unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop-in reference for Claude Code (and similar AI tools) covering the three-simulator pipeline (Gem5 -> Spike -> TOGSim), directory map, test entry points, key env vars and YAML knobs, multi-tenant API contract, build steps, and known gotchas. Complements README.md with a denser cheat-sheet aimed at code navigation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/build_from_source.sh now reads release_tag from the manifest that the CI docker image is built against, and clones gem5, llvm-project, and riscv-isa-sim at those tags. Untagged HEAD clones had caused mlir-opt option-name drift (Pass-Options-Parser error followed by an assert(0) in extension_codecache.py). Documents the pin in README.md and CLAUDE.md, and adds a plain-text / ASCII rule for commit messages in CLAUDE.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…overage [Build] Pin third-party deps to thirdparty/github-releases.json tags
scripts/setup_worktree.sh creates a sibling worktree under $PARENT_DIR/<repo>-<purpose>/, wires per-worktree env vars via a generated .envrc (TORCHSIM_DIR, TORCHSIM_DUMP_PATH, TORCHSIM_LOG_PATH, TOGSIM_CONFIG, PYTHONPATH), unsets the upstream so the first `git push -u origin <branch>` creates the right remote branch, and symlinks the TOGSim binary from the worktree the script was run from to skip a ~10-minute TOGSim rebuild per fresh worktree (readlink -f resolves chains so the link points at the real binary). scripts/clear_codegen_cache.sh wipes Inductor's compile cache (.torchinductor/, set via TORCHINDUCTOR_CACHE_DIR in extension_config.py:139) and the per-source-hash dirs identified by the 11-char prefix from extension_codecache.hash_prefix. Run it between iterations on PyTorchSimFrontend/mlir/* or anything else that affects emitted MLIR -- otherwise the next torch.compile silently replays the previous compile. togsim_results/ and unrelated files under outputs/ are preserved. docs/worktrees.md documents the flow: activate via `source .envrc`, build the .so once per worktree, TOGSim binary sharing, codegen iteration loop, and the diagnostic for the common "forgot to source .envrc" case (tracebacks pointing at the canonical worktree path while editing elsewhere). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Running tests section now states that .github/workflows/pytorchsim_test.yml runs an explicit allowlist of tests/*.py (one Docker job per test, ~40 total), not a glob-discovered set, and calls out that test_gqa.py, test_gqa_decode.py, and test_eager.py exist in the repo but are not in CI. The Gotchas section adds a bullet explaining that codegen iteration requires wiping $TORCHSIM_DUMP_PATH between runs -- Inductor's compile cache and the per-source-hash MLIR/wrapper dirs both live under it and will silently replay the previous (possibly broken) compile otherwise. This prevents two common mistakes: assuming new tests/*.py files are automatically gated on PR, and assuming a fresh re-run will pick up a codegen fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…228) The second clause of the index-cast guard at mlir_ops.py:268 compared the whole [tile_size, dtype] list against the string "index", which is never true. With i64 on the lhs and index on the rhs the code fell through to the same-bit-width branch (MLIR_TO_BIT["i64"] == MLIR_TO_BIT["index"] == 64) and emitted arith.cmpi between vector<Nxi64> and vector<Nxindex>, which mlir-opt rejects. This blocked any MoE model whose (i64_buf == arange) expert-mask pattern landed with that operand orientation -- first observed on deepseek_v3. Fix replaces the dead clause with op_type2[1] == "index" so the operand2-side index_cast at lines 285-288 is reachable, normalizing the rhs to i64 before the cmpi. Add tests/test_expert_mask.py as a focused regression covering the (expert_idx_i64.unsqueeze(-1) == arange(N)) -> torch.where pattern.
Adds three top-level jobs (test_eager, test_exponent, test_sort) and two steps inside the test_fusion job (test_attention_fusion, test_matmul_vector). All five were verified locally before registration. Files in tests/ that remain intentionally out of CI: - test_gqa.py, test_gqa_decode.py: WIP GQA SDPA path, tracked by issue #198 - test_sdpa.py: overlaps with in-flight SDPA template work; ambiguous about which backend it actually exercises - test_topk.py: sort-family coverage now provided by test_sort (stable + unstable + duplicate-key); revisit if topk-specific shapes need gating - test_group_conv.py: not run locally yet (stress config); follow-up after runtime cost is understood - test_vectorops.py: imports from other tests (test_add, test_activation, test_reduce, test_layernorm, test_softmax) which is fragile; needs an independent helper extraction before going into CI Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ovided (issue #236) decompose_native_multi_head_attention referenced attn_bias before it was defined, so any masked-attention call raised NameError. The same block also computed attn_bias but never folded it into scores before softmax, so the mask would have had no effect even if attn_bias existed. Initialize attn_bias as zeros_like(scores), build the bias from the boolean/additive mask, and add it to scores ahead of softmax. Resolves issue #236.
…sue #237) 1. mlir_codegen_backend.py:909 - NotImplementedError was constructed but not raised, so kernels with multiple reduction axes silently fell through to incorrect codegen. Add the missing raise. 2. mlir_codegen_backend.py:171-179 - codegen_sram_plan_prefix called buf.get_size() before checking buf is None, so a None entry in graph_inputs would AttributeError. Reorder the None check first. 3. mlir_common.py - CSEProxy had two identical static check_bounds declarations; only the latter survived class-body evaluation. Drop the duplicate so future drift cannot hide which one is authoritative. 4. extension_codecache.py:274 - CustomAsyncCompile.mlir passed valdiation_wrapper_name (typo) carrying self.validation_binary_name to MLIRCodeCache.load. The misspelled kwarg was absorbed by **kwargs and validation_wrapper_name silently fell back to its default. Use the correct keyword and value. Resolves issue #237.
CONFIG_TORCHSIM_TOG_HOST_{CC,CFLAGS,LDFLAGS} were added in 54ccd4c but
no caller ever consumed them; MLIRCodeCache._load_library is still
`pass`. Issue #239 flagged the `if True:` shortcut in
_default_tog_host_cflags, but the helper's return value is never read,
so no .so is actually compiled with `-Og` as the issue assumed —
removing the whole block is the honest fix instead of restoring the
env-var gate around code nothing calls. If a host-side TOG .so compile
path lands later, the gate can be reintroduced at the real call site.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
) `extension_config.CONFIG_TORCHSIM_DUMP_PATH` was resolved through the module-level `__getattr__`, which read `TORCHSIM_DUMP_PATH` and *mutated* `os.environ["TORCHINDUCTOR_CACHE_DIR"]` as a side effect on every attribute access. Generic attribute reads should not change process state, and the four codegen call sites trigger this on every compile. The dynamic re-derivation is load-bearing for the Jupyter tutorials (tutorial/session1/CompilerOptimization.ipynb, Mapping.ipynb) which flip `TORCHSIM_DUMP_PATH` between cells to compare fusion ON/OFF in separate output dirs, so the issue's "move to module top-level" fix would silently break them. Instead expose the side effect as an explicit `get_dump_path()` function — same behavior at the codegen call sites, but readers can no longer trigger an env mutation by accident, and the function name signals "this writes to os.environ". Also fixes the unrelated bug noted at the end of #240: `__getattr__` fell through to an implicit `None` return for unknown names. Now it raises `AttributeError` per PEP 562, so typos like `extension_config.CONFIG_NONEXISTENT` surface immediately instead of becoming downstream `NoneType` errors. Verified locally: `get_dump_path()` syncs `TORCHINDUCTOR_CACHE_DIR`, follows dynamic env changes between calls, leaves env untouched on unrelated attribute reads, and unknown attrs raise `AttributeError`. Closes #240. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sion
The guard at mlir_codegen_backend.py:909-910
if len(reductions.loops) > 1:
raise NotImplementedError("Not support multiple reduction axis..")
was a leftover from when the codegen was deliberately limited to a
single reduction axis, but multi-axis support was restored months ago
and the guard was never removed.
Timeline:
- 8eacf27 ("Optimize reduce/elementwise code", 2025): replaced
`for reduction in reductions.loops` with `reductions.loops[0]`,
added this guard (no `raise` — silently dead).
- c61f67d ("Support multi reduction dim + Add Diffusion model test",
2025-08-14): restored the `for reduction_loop in reductions.loops`
iteration explicitly to support multi-axis reduction. test_diffusion
was added in this same commit and passed. The guard was left in
place — harmless because still `raise`-less.
- 5045837 ("Fix four codegen correctness issues surfaced by review",
2026-05-26, issue #237): added the missing `raise`, framing it as a
"silently fell through to incorrect codegen" fix. In fact the
codegen below the guard (which iterates all reduction loops) was
correct — c61f67d had made it so — and adding `raise` broke the
very test (test_diffusion) that c61f67d added to validate multi-axis
support.
Right fix is to delete the guard, not add `raise` to it. The other
three items in #237 are unrelated and stand.
Verified: file py_compiles. CI test_diffusion is the empirical proof
that the post-guard codegen handles multi-axis correctly within the
allclose tolerance the test uses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…s.py
Replaces 48 near-duplicate ~12-line `test_result(name, out, ...)` defs
(two signature variants, three slightly divergent bodies — one had a
'pass message only' bug) with a single canonical helper that prints
framed pass/fail messages and exits 1 on mismatch. Positional-argument
compatible, so caller sites are unchanged.
Each migrated file replaces its local def with:
import os, sys
sys.path.insert(0, os.path.join(
os.environ.get("TORCHSIM_DIR", default="/workspace/PyTorchSim"), "tests"))
from _pytorchsim_utils import test_result
The module name is deliberately unique rather than `tests._utils`:
ultralytics ships its own top-level `tests` package in site-packages,
which shadows any generic `tests` import. `insert(0, .../tests)` puts
the repo's tests dir ahead of site-packages so the helper resolves
regardless of installed packages.
Net diff: 50 files changed, 220 insertions(+), 729 deletions(-).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
tests/ was a flat mix of op-level files, single-file model tests, and
inconsistently-cased model directories. Adds a hierarchy:
tests/
_pytorchsim_utils.py
ops/
elementwise/ reduce/ gemm/ conv/ attention/
view/ sort/ sparsity/ misc/ fusion/
models/
DeepSeek/ Diffusion/ Llama/ MLP/ MoE/
MobileNet/ Mixtral8x7B/ Yolov5/
test_mlp.py test_resnet.py test_single_perceptron.py
test_transformer.py test_vit.py
system/
test_eager.py test_hetro.py test_scheduler.py
test_stonne.py test_vectorops.py
Mixtral_8x7B → Mixtral8x7B for consistency with the other PascalCase
model dirs. Existing single-file model dirs are kept as dirs (they
may grow companion files like the Mixtral model.py).
All file moves use `git mv` to preserve history. External path
references rewritten across .github/workflows/pytorchsim_test.yml,
README.md, CLAUDE.md, .github/ISSUE_TEMPLATE/bug_report.md, and
scripts/{sparsity,stonne}_experiment/.
Cross-test imports drop the `tests.` prefix (the prior commit puts
`<repo>/tests` on sys.path[0]). `__init__.py` added to each new subdir
so e.g. `from ops.elementwise.test_add import test_vectoradd` resolves.
Sample-verified locally on tests/ops/elementwise/test_add.py and
tests/ops/fusion/test_matmul_vector.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PyTorchSim never executes CUDA: the npu PrivateUse1 backend (third_party/openreg) simulates CUDA on the CPU and forces math-only SDPA, and flash-attn is never imported. The only thing that required CUDA was find_package(Torch) inheriting it from the CUDA-built torch wheel; installing the CPU wheels removes it, so the whole CUDA base image can go. Dockerfile.base: - Base image ubuntu:22.04 instead of the cuda-cudnn-devel image, kept on 22.04 to match the prebuilt gem5/llvm/spike ABIs (libpython3.11, libprotobuf.so.23). - Python 3.11 via deadsnakes and CPU torch/torchvision wheels. - Remove flash-attn (unused, needed CUDA/nvcc to build). - Install psutil and the other yolov5 hub runtime deps (pyyaml, requests, tqdm, py-cpuinfo) that the old fat base provided implicitly; ultralytics is installed with --no-deps. - Remove duplicate onnx/matplotlib/conan/ninja installs. - Fix the RISC-V toolchain step to download and extract the elf toolchain once (the glibc tarball was downloaded but unused, the elf tarball was extracted twice). - Drop conda/nvidia paths from LD_LIBRARY_PATH. thirdparty/github-releases.json: pytorch_image -> ubuntu:22.04. DeepSeek: its remote modeling file (trust_remote_code) imports flash_attn, which transformers check_imports requires installed even though flash attention is never executed on the npu. Register an import shim in tests/models/DeepSeek/test_deepseek_v3_base.py that satisfies the static import check while leaving is_flash_attn_2_available() False. CI runners: the CPU-only image (~3.7 GB compressed) is small enough to pull on GitHub-hosted runners, so the base build, app build, and the op/model test jobs run on ubuntu-latest. Only test_deepseek (largest model), test_diffusion (UNet2D simulation OOMs the hosted runner), and test_accuracy (accuracy + speedup) stay on self-hosted.
The tests/ reorg (tests -> tests/ops, tests/models) left experiments/BERT.py importing EncoderBlock from the old paths (tests.Fusion.test_transformer_fusion, tests.test_transformer), so every BERT size raised ModuleNotFoundError. Put tests/ on sys.path and import from the new locations (ops.fusion.test_transformer_fusion, models.test_transformer), matching how the test files import their siblings. run_cycle.sh ran each model as 'python3 ... | tee log' under 'set -e' only, so a model crash was masked by tee's exit code and the job stayed green with empty logs (this is why BERT failures went unnoticed). Add 'set -o pipefail' so a failing model aborts the run. Verified: BERT --size base now runs end to end (TOGSim, 329573 cycles). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Run the MLIR -> LLVM pipeline in-process via the bindings PassManager. Add Python out-of-line passes: lower_to_llvm, lower_dma_to_gemmini, lower_vlane_idx. Auto-resolve the bindings path from TORCHSIM_LLVM_PATH and ship the bindings in the LLVM artifact. Add op_coverage tooling and the bindings smoke test. Bump the LLVM pin and rebuild the thirdparty base image.
Emit togsim.transfer for >4D DMA and decompose it to a <=4D customized memref.dma_start: unit-collapse fast path, unrolled-subview peel for >4 effective dims. Fix #258 by emitting affine.apply (not arith.addi) for the peeled DRAM offset so the TOG pass can walk the loop index through it.
Split a loop axis so aligned FloorDiv/ModularIndexing collapse to per-axis affine indices. Mixed-radix split over a divisibility chain; integer-typed split symbols; r-prefix innermost reduce dims. Reindex the collapsed LoopBody instead of re-tracing; fold residual floor/mod via tensor range info. Shared boundary helpers, rank guard, and an uncovered floor/mod ledger. Enabled by default with the recompile fallback instrumented.
… floor/mod Insert a copy to relayout an operand whose floor/mod cannot be removed by axis-split: incompatible-radix shared-axis access and cross-axis multi-variable arguments. Enabled by default alongside axis-split.
Port the analysis and IR-mutation halves of the C++ test-tile-operation-graph pass to Python, wire build_tog into the gem5 path, and drop the C++ pass. Node-id counter is thread-local for concurrent compilation.
…seek seed Add tests/ops/view/test_floormod_axis_split.py covering axis-split and graph-copy patterns. Seed the global RNG in the deepseek base test so config-random weights are deterministic.
…cal SRAM offset Rewrite the >4D peel to mirror the C++ -dma-fine-grained subtile loop: wrap the outer dims in an affine.for nest (marked inner_loop so build_tog/TOG registers the induction var) and emit one <=4D memref.dma_start per iteration. The slice SRAM offset is the lane-banked physical offset -- split-outer dims rescaled by the lane coeff (stride/old_size*new_size, the MVIN block_stride / buildSramAffineMap rule) -- delivered as the last SRAM index operand. The previous unrolled subview carried the offset in the subview, which extract_aligned_pointer_as_index strips in the gemmini lowering, so every slice aliased the same spad location (pixel_shuffle MISMATCH). The DRAM offset folds with the original index into one affine.apply so processDramIndices can walk the loop index (#258). Thread vectorlane (systolic-array size) through run_python_passes into the pass for the rescale's nr_outerloop. Drop the axis-split rank guard now that >4D is peeled correctly, and register tests/ops/view/test_floormod_axis_split.py in the CI allowlist. Validated end-to-end (Gem5+Spike+TOGSim): pixel_shuffle (>4D peel) and the full floor/mod suite pass; elementwise/gemm/conv2d/reduce/softmax/MLP regress clean.
Route every MVIN/MVOUT -- both the MLIRKernel load/store backend path and the template path (gemm/conv/bmm/maxpool/cat) -- through emit_transfer, so a single decompose-transfer pass lowers all DMAs to memref.dma_start. This drops the get_dma_code emitter, the _dma_needs_transfer instance flag, and format_dma_op_attributes. togsim.transfer now also carries subtile_size and async, which decompose propagates onto the lowered dma_start (subtile filtered to the kept axes when unit dims collapse). For <=4D tiles decompose emits the descriptor directly on the original SRAM buffer (no collapse_shape) so the C++ -dma-fine-grained subtile split, which walks the SRAM operand, sees a direct buffer as before. Validated end-to-end (Spike + TOGSim) on elementwise, gemm (matmul/addmm), bmm, conv2d, group_conv, pool, cat, reduce, softmax, layernorm, batchnorm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reflect a6b7ebb: the find_split_plan rank guard is gone (>4D index now lowers through the decompose-transfer affine.for peel, pixel_shuffle end-to-end), and the decompose-transfer peel <-> TOG incompatibility is resolved. Move it from Known-issues to Done; drop the >4D rank-guard caveat and the high-rank next-step. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port mlir/test/lib/Analysis/TestDmaFineGrained.cpp to a Python out-of-line pass (passes/dma_fine_grained.py): split the matmul MVIN DMAs (input/weight/bias) into subtile affine.for nests and fuse the input/weight nests, replacing the C++ -dma-fine-grained pass. The MLIR Python bindings expose no IRMapping, so the fused nest is built directly (each DMA emitted with the fused induction vars) instead of cloning bodies -- structurally equivalent, not byte-exact SSA text. Pipeline: the single mlir-opt invocation is split around the Python pass (loop-padding -> run_fine_grained in place -> pytorchsim-to-vcix) in both the functional and gem5 paths (extension_codecache); vectorlane (systolic-array size) is threaded in for the lane-banked SRAM offset rescale. Validated against mlir-opt -dma-fine-grained on rank 2/3/4 fixtures (matmul / bmm / conv: same vcix dma_start and line counts) and end-to-end (Gem5+Spike+TOGSim): gemm/bmm/conv2d plus the resnet/transformer/vit/mlp models pass. Docs: dma-transfer-lowering.md -- >4D peel is affine.for + lane-banked physical SRAM offset via the last index operand; dma_fine_grained / build_tog are now Python passes; the #258 appendix is marked resolved.
…ings)
Port mlir/test/lib/Conversion/PyTorchSimToVCIX/TestPyTorchSimToVCIXConversion.cpp to
a Python out-of-line pass (passes/lower_to_vcix.py): lower linalg.matmul (gemm and
conv2d) and the transcendental math ops (exp/erf/tanh/sin/cos) to VCIX dialect ops
(RISC-V vector custom instructions), replacing the C++ -test-pytorchsim-to-vcix.
The C++ pass is a dialect conversion (applyPartialConversion); the bindings expose no
conversion framework, so each matchAndRewrite is reimplemented as imperative IR
rewriting. The VCIX dialect is not in the Python bindings, so vcix ops are created as
unregistered generic ops -- mlir-opt / mlir-translate (vcix registered) re-parse the
{}-attr generic form fine, and run_standard_lowering already consumes vcix output via
allow_unregistered_dialects, so this matches the existing pipeline.
Pipeline: the vcix mlir-opt invocation is dropped; run_to_vcix runs in-process after
the Python fine-grained pass and before the standard lowering (both functional and
gem5 paths in extension_codecache). mlir-opt now runs only -test-loop-padding.
Validated structurally against mlir-opt -test-pytorchsim-to-vcix (non-constant ops
byte-identical including the dma_wait tag maps, on gemm and conv2d fixtures) and
numerically end-to-end (Gem5+Spike+TOGSim allclose): gemm/bmm/conv2d (incl. large
N/K), softmax, exp/erf/sin/cos, and the resnet18/vit/transformer/mlp models.
dma-fine-grained and pytorchsim-to-vcix are now Python passes (dma_fine_grained, lower_to_vcix); update the docstring listing -- only test-loop-padding still runs in mlir-opt.
axis-split + graph-copy (on by default) linearize aligned floor/mod at the scheduling layer, so the index reaching get_dma_info is affine and the FloorDiv/ModularIndexing tile-divisibility branches there are never entered (measured: 0 entries across elementwise, gemm, bmm, conv, cat, floor/mod, reduce, attention). Remove those dead branches and their orphans: - the FloorDiv and ModularIndexing tile-forcing + RecompileSignal blocks - the implicit-ModularIndexing index rewrite and implicit_local_dims - the dead ModularIndexing branch in the dram_stride computation - is_modular_indexing, the write-only implicit_dim_size, unused import sys Kept: the non-floor/mod recompile paths (index-divisibility, indirect access, non-power-of-2 vec size), RecompileSignal, and the retry loop. The upstream implicit_dim_ops tile-forcing is left untouched (separate change). Validated end-to-end (Spike + TOGSim): elementwise, gemm, bmm, conv2d, group_conv, pool, cat, floor/mod suite, reduce, softmax, layernorm, batchnorm, gqa -- all pass, 0 recompiles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
togsim_dispatch with TILE_BEGIN/TILE_END; outline each work-item into togsim_kernel_tile.
DMA-capacity throttle and frozen-state guard, per-core VMEM in the configs, and the SA weight-buffer throttle.
trace_timeline.py with per-work-item grouping and resource-centric DMA lanes; the trace logs the first DRAM response and the assigned systolic array, and scopes the compute barrier to its dispatch.
The trace .so is now picked up automatically next to the kernel's tile_graph.onnx; TORCHSIM_LEGACY_TOG=1 falls back to the deprecated ONNX producer. Also make the codegen cache replay safe: FxGraphCache can skip PyTorchSim codegen entirely, so the wrapper must carry what the .so needs, and concurrent compiles of one source must not observe a half-written .mlir.
… runtime model Replace the trace bridge's accumulated special cases with one dataflow rule and clean up the runtime that consumes it. Dependency rule: per SRAM buffer keep a writers SET; a reader depends on all current writers (occupancy=ISSUE when both are systolic-array ops, else latency=DONE); a writer REPLACEs the set. The only exception is is_mm_accum (a matmul that reads and writes the same buffer = a commutative accumulator): skip its read edge and UNION its write, waiting only the non-matmul init seed and not ordering co-matmuls. This drops the matmul-accumulator chain that deadlocked the SA weight-slot pipeline while keeping the init->matmul edge, and lets a vector epilogue or the store wait every K matmul (fixes the pure-vector store that an empty COMPUTE_BAR let slip). Remove COMPUTE_BAR entirely: a matmul is its own DONE-handle (finish == SA drain), so the store JOINs the matmul writers directly. The whole emit/loader chain is gone -- build_skeleton, lower_to_emitc, togsim.compute_barrier, the runtime symbol, the Opcode/case/_fence_finish, and TraceRec::COMPUTE_BAR -- so a stale producer fails loudly instead of emitting records the bridge would drop. Only MEMORY_BAR remains (an async load's DONE is its data arrival, not issue). Model compute-output spad footprint in the SRAM version/capacity machinery so buffer reuse (WAR) is capacity-modeled, not a hard edge. The output size comes from the DMA records that touch the same buffer (a buf_bytes pre-pass); an in-place buffer (accumulator, relu) is version-transparent so footprint is not double-counted. The occupy gate and version release sit in the MOVIN/MOVOUT/COMP issue points (release before the COMP skip path so a skipped matmul still frees). Runtime: collapse child_inst / _pipeline_children into one event-indexed _deps[ISSUE|DONE] with add_dep(c, on) and fire(e); collapse the weight-slot release queue and the async-load wakeup into one _due_events timed-effect table drained by process_due_events. Both are behavior-preserving (byte-identical). Require the weight-slot model: sa_weight_buffer_depth must be > 0 (errors at init), and the round-robin disable mode is removed. Degenerate traces (a consumer-less preload, an unpinned matmul) hit explicit error+exit guards rather than asserts that vanish under NDEBUG. Mark the legacy ONNX TOG path deprecated: it is superseded by the trace path, so TileGraphParser logs a deprecation warning and the TORCHSIM_LEGACY_TOG=1 opt-in warns at command build.
… spad/2
The validation-binary spad-overflow check sat inside `if functional_mode:`, so in
timing-only / autotune (non-functional) runs an over-spad tiling was never
rejected and reached TOGSim, which wedged ("spad too small") and crashed the
compile via assert 0. Move the compile + check out of the functional gate (the
Spike execution itself stays gated, run_spike below) and budget per dispatch at
spad/2 -- two work-items run concurrently (double buffer), so each must fit half
the spad or they deadlock competing for it. This matches the GEMM tiling gate
(max_spad_size = spad/2), which pointwise ops lacked. Fixes the resnet /
test_scheduler wedge where a fused BatchNorm+ReLU tile exceeded spad/2.
Every generated trace.cpp now opens with a "GENERATED ... DO NOT EDIT" banner carrying the TOGSim ABI version and the togsim_* call formats, so a dumped trace is self-documenting. Both are read from togsim_runtime.h at codegen time -- the version from the TOGSIM_ABI_VERSION define, the call-format text from a marked block next to the declarations -- so they never drift when the ABI changes.
…rint The bridge sums each work-item's distinct-buffer footprint (each buffer once, so a reduction's reloads of the same section do not inflate it) onto its Tile. can_issue then admits two concurrent dispatches only when each fits half the spad; a dispatch larger than spad/2 runs alone with the whole spad, so two work-items never compete for the shared spad and deadlock -- a runtime safety net beneath the codegen spad/2 gate. The footprint and resulting max_dispatch are logged on the TILE_SCHEDULED trace line for debugging.
Count the fused-epilogue buffers a tiling actually needs, and stop charging the kernel's stack frame against the spad budget -- it lives in DRAM, not the scratchpad, so including it rejected tilings that fit.
Spike v1.0.3 zero-inits the MVIN/MVOUT DMA address buffer so ROUNDUP padding entries are skipped instead of dereferencing uninitialized garbage, fixing the host store segfault on 8-lane (wrapper3) configs where a tile's split axis is not a multiple of vlane_stride*n_vu.
pytorchsim_functional_verify_per_kernel compares every realized Spike buffer against a CPU golden and stops at the first divergent kernel, naming the op and the offending indices.
The GEMM path already budgeted its fused epilogue; BMM's fused prologue was unaccounted for, so a tiling that overflowed the scratchpad could be chosen.
Skip the ready-queue scan when nothing can newly issue, skip building the per-instruction trace string when trace logging is off, and fix a use-after-erase in the zero-cycle COMP skip path.
…X TOG The main compile/sim path no longer generates or selects the legacy ONNX Tile-Operation-Graph. extension_codecache emits only trace.so + trace_cycles.tsv (the build-skip now keys on trace.so), and TOGSimulator.run_standalone always drives TOGSim with --trace_so. The TORCHSIM_LEGACY_TOG opt-in is removed from the frontend. The ONNX --models_list branch is kept solely for the STONNE sparse path (extension_op.py); TOGSim's C++ ONNX parser is untouched (separate PR). origins (which FX nodes a kernel came from) is preserved: logged per kernel run and recorded as a trailing "# origins:" line in trace_cycles.tsv -- the legacy ONNX TOG carried this as node metadata, and the C++ cycle-table loader stops at the comment so the current parser is unaffected. Also drop the dead tog_file param from mlir_gem5_compile_command, migrate scripts/chiplet.sh to --trace_so/--cycle_table (the trace path stubs per-tensor addresses and --attributes_list is no longer a Simulator option), and refresh the CLAUDE.md TOG-generation notes.
…iptor Represent a kernel body as ordered steps so a multi-step indirect access can be emitted, and hand the gather/scatter offset to the DMA as an explicit operand instead of inferring it from the operand count.
…emmini togsim.transfer is unregistered, so it can carry every runtime descriptor as an operand -- including the masked-clamp low/high vectors -- which a registered memref.dma_start cannot. The trace path (build_skeleton) reads togsim.transfer/togsim.wait, and the indirect offset operand is detected by its `indirect` attribute rather than by counting operands.
…o CONFIG/2/3/4) Instead of packing the transfer params into four CONFIG asm instructions, build a 144-byte DMA descriptor as an 18xi64 memref.global (.data, non-constant so runtime fields can be written) and emit a single CONFIG_DESC that hands its address to the MVIN/MVOUT (which read the struct, matching the Spike-side torchsim_config_desc). This is the vehicle for the masked-DMA low/high clamp: dim_low/dim_high/fill are descriptor fields (currently defaulted, so behavior is unchanged), not new special registers. - _pack_desc encodes the fields into the struct's i64 slots (dim_size/low/high/mm_stride/ spad_stride/element_size/vlane/config_type/flags/indirect stride+esize). - descriptor globals are deduped by content; the indirect offset spad address is a runtime value, stored into slot 15 before the transfer (flags.bit0 = indirect). Requires the Spike torchsim_config_desc instruction (riscv-isa-sim). Validated functional (Spike) + trace (TOGSim): add, matmul (subtile+async), softmax, gather/scatter (indirect runtime store), constant_pad all pass.
Clamp each tile axis to its real DRAM extent in the DMA descriptor, so a tile that runs past the extent is filled with the consuming reduction's identity instead of MAC-ing garbage, and a store skips the ragged tail instead of writing the systolic pad region. Covers pointwise, pad, reduction, gather/scatter, matmul and conv, plus the fused-epilogue output store. Adds a golden unit test for the lower_transfer_to_gemmini lowering and a non-dividing regression suite to CI.
…ixel_shuffle trace) Both the Gemmini descriptor and the TOGSim DMA model cap at 4 tile dims, but a logical tile can exceed 4D (e.g. pixel_shuffle splits two spatial axes -> a 5D tile [1,1,2,4,2]). lower_transfer_to_gemmini reduces >4D as part of emitting Gemmini asm, but that runs only on the Spike/gem5 lowering path. The trace producer (build_tog) reads togsim.transfer BEFORE that, so it emitted a 5D DMA and TOGSim rejected it: "issued tile is not supported format.. tile.size: 5". Exposed by the pixel_shuffle case newly added to the wrapper1 CI suite. Add a peel_transfer POST_OPT pass that reduces every >4D togsim.transfer up front (before build_tog) while KEEPING it a togsim.transfer, so both consumers see <=4D. It mirrors lower_transfer_to_gemmini's two reductions: collapse unit dims when the effective rank is <=4 (memref.collapse_shape), else peel the outer effective dims into an affine.for nest with the lane-banked physical SRAM offset. The gemmini lowering keeps its own reduction as a now-defensive no-op. Validated on 128x128: test_floormod_axis_split (pixel_shuffle + 9 others), test_masked_nondividing, test_matmul, test_conv2d all pass.
Three MVIN over-reads on shapes whose tiles do not divide the extent: a cat concat-dim split, a vlane index_expr row stride on multi-row splits, and a 1-D reduction tile that exceeded its extent.
torch.sort's frontend lowering exposes the sorted VALUES as the sole scheduler-visible template output; the INDICES buffer is written in place by the same kernel (make_inplace) but is not registered as an output. For argsort the values output is dead, so Inductor DCEs the whole sort kernel and the indices buffer is returned uninitialized (all-zeros), silently corrupting any argsort result (e.g. DeepSeek MoE expert routing). Register the in-place indices write as a MutationOutput owned by the sort operation so the scheduler keeps the kernel alive whenever indices is consumed. OperationBuffer.get_outputs() hardcodes [self], so add a small MLIRTemplateBuffer subclass that can own extra MutationOutputs and have MLIRTemplateCaller.output_node build it; the sort lowering attaches the indices mutation via add_mutation_output. For templates with no mutation this is behaviorally identical to the base (get_outputs == [self]). Then gate the values MVOUT on the values output being live: in argsort it is pruned from the kernel args, so emitting its MVOUT referenced an undeclared %YV. The values scratchpad is still produced internally for the bitonic compare; only the dead DRAM store is skipped. Verified: t.argsort() now returns correct indices (was all-zeros); torch.sort with both outputs live is unchanged; test_sort, test_matmul, test_cat, test_indirect_access unaffected by the output_node change. Note: x[argsort] end to end additionally needs the indirect-gather codegen fix (separate).
…oad clamp A bare indirect index (x[idx]) left the loop-local symbol in the DMA base offset. Separately, reverse-engineering per-dim padding from a single flat offset is ill-posed -- under channels_last the stride-1 axis absorbs the remainder and yields an empty [low, high) box that zeroed the whole load.
codegen_epilogue_body() decided whether to emit the template buffer's MVOUT
by asking "did a fused epilogue already store something?":
if len(self.stores._lines) == 0:
template_store()
That is a proxy for the real question, "does anything outside this kernel still
read the template buffer?", and it is wrong in two of the four cases:
- epilogue + live template buffer -> store skipped, buffer never written
- no epilogue + dead template buffer -> store emitted for a pruned DRAM arg
The first case corrupts DeepSeek-V3's MoE gate. The gate is
sigmoid(mm(hidden, gate_w) + bias), so add+sigmoid fuse onto the GEMM template,
but the raw mm result is also read by a later gather kernel. Its buffer was
declared as a kernel output and never stored, so the gather read an
uninitialized buffer (all zeros), scrambling expert routing. Every config was
affected; only 8x8 crossed the test's loose atol=2e-1 (final Max abs diff
0.2546) while 32x32 landed just under it and passed.
Ask the real question instead. Inductor already answers it:
Kernel.remove_kernel_local_buffers() drops a buffer only when
Scheduler.can_buffer_be_removed_through_fusion() finds every user inside this
fused kernel. A buffer that survived removal therefore has an outside user and
must be stored. Record the template buffer's name in def_kernel() and
def_conv_kernel() (conv has its own implementation) and gate template_store()
on it, in both the main and the reduction_fusion branches. The proxy is gone.
Use the kernel-local self.removed_buffers, not V.graph.removed_buffers: the
former is what remove_buffer() populates, and the latter is still empty when
the store hook runs.
Note this makes a previously missing MVOUT issue, so DRAM traffic and cycle
counts rise for any model with a live template buffer under a fused epilogue.
That is the correct cost, not a regression, but reported cycles will shift.
Verified on the 8x8 config: test_deepseek_v3_base goes from Max abs diff 0.2546
to passing; a dead template buffer (relu(mm)) still prunes its DRAM arg and
emits a single MVOUT; the trace/TOGSim timing path runs with the extra store.
Regressions pass: test_bmm_reduction, test_matmul, test_bmm, test_conv2d,
test_cnn, test_group_conv, test_pool, test_reduce, test_softmax, test_layernorm,
test_mlp.
Lower the remaining pointwise ops, route math.log/atan to VCIX in the Python pass, and add tests/ops/elementwise/test_pointwise.py.
Bump the third-party pins to the releases shipping the new pointwise instructions (spike torchsim_vlog/vatan, gem5 CustomVlog/Vatan + decode).
Make int max/min reductions dtype-aware, lower torch.roll to narrow+cat and realize it so the consumer reads a plain affine index, flatten nested single-variable floor/mod in axis-split, define empty_strided_cpu in the generated wrapper header, and place the reduction axis outermost in the per-lane tile layout.
SwinV2 batch>1 used to fail codegen with "Unlinearized floor/mod in DMA index" because the shifted-window path (torch.roll composed with window partition/reverse) produced views axis-split could not linearize. With the roll->slice+cat lowering, the nested/shifted mod handling, and the reduction-axis layout fix, the whole backbone now compiles and matches CPU end to end (max diff ~5e-6 for image 64 / window 8 / batch 2). Add tests/models/test_swinv2.py, wire it as a self-hosted CI job next to the other model tests, and list SwinV2 in the README model coverage.
conv_multi_tile_mapping (used for batch > 1 convs) collects every tile that fits SPAD into tile_candidates and returns them sorted, but it guarded the "no mapping" error on max_used_spad_size instead of on tile_candidates. max_used_spad_size is only bumped when a candidate also satisfies max_k_h_w <= k_h, and max_k_h_w is initialized to K_W, so it only ever fires for the full-kernel (k_h == K_H) tile. When that tile overflows SPAD -- e.g. a CLIP ViT-B/32 patch conv (3->768, 32x32 stride 32) at batch > 1 -- the guard raised "Cannot find a valid mapping" even though smaller-k_h tiles fit and were already in tile_candidates. The mapping variable it tracks is never returned. Guard on `not tile_candidates` instead, so the conv is rejected only when no tile fits at all. The returned candidate list is unchanged, so convs that already worked are unaffected. Adds a batched patch-conv case to test_conv2d.py. Fixes issue #252.
With the conv-mapping fix, CLIP's vision transformer runs with batch > 1 and matches CPU end to end (max diff ~1e-5). Add tests/models/test_clip.py (CLIPVisionModel, batch 2, patch 32), wire it as a self-hosted CI job, and list CLIP in the README model coverage.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.