Skip to content

WGMMA+SP Kernel + AGU Serialization Fix#383

Draft
NikhilRout wants to merge 2 commits into
masterfrom
feature_wg2k
Draft

WGMMA+SP Kernel + AGU Serialization Fix#383
NikhilRout wants to merge 2 commits into
masterfrom
feature_wg2k

Conversation

@NikhilRout

Copy link
Copy Markdown
Collaborator

1. Sparse WGMMA kernel

  • old kernel redundantly loaded every warp’s compressed A and metadata through SMEM causing a lot of GMEM traffic
  • now each warp only loads its own compressed A tile instead of cooperatively serializing loads across warps
  • sp metadata is now directly loaded from GMEM -> TCU SRAM instead of old GMEM -> SMEM -> TCU SRAM
  • now uses tilek/2 (A) tilek (B) strided descriptor layouts matching the TCU buffer

2. TCU metadata AGU

  • old AGU serialized all TCU blocks execution behind all warps' metadata memory round-trip latency. this prevented any memory-latency overlap
  • changed to one in-flight TCU_LD transaction per TCU block with round-robin issue/commit and tagged response routing (just pipelining metadata load/per TCU)

@NikhilRout
NikhilRout marked this pull request as draft July 25, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant