MTP + NVFP4 blocked on sm120: unquantized MTP head forces flashinfer_cutlass/trtllm → Sm120_SafeFP4 MoE kernel crash (marlin rejects unquantized head)

[mabry1985]

Summary

MTP speculative decoding cannot be enabled for Nemotron-3-Puzzle-75B-A9B-NVFP4 (and, by
extension, any NVFP4 model whose MTP draft head is shipped unquantized) on our sm120 Blackwell
cards, because the only MoE backend that can serve both the NVFP4 main model and the unquantized
MTP head routes the NVFP4 MoE through the Sm120_SafeFP4 cutlass FP4-MoE kernel, which crashes on
sm120. Tracking for a later fix; not blocking eval (we run NVFP4 + CUDA graphs at ~116 tok/s).

The wall (verified 2026-07-08, vLLM 0.22.1 / flashinfer 0.6.11 / CUDA 13)

--moe-backend is a single global flag, but the model has mixed quantization: the main MoE is
NVFP4, the MTP draft head's MoE is unquantized (bf16) — NVIDIA ships it that way.

--moe-backend marlin              NVFP4 MoE ✓   MTP head ✗
   → ValueError: moe_backend='marlin' is not supported for unquantized MoE
     (nemotron_h_mtp.py:140, NemotronHMTPMoEDecoderLayer)

--moe-backend flashinfer_cutlass  both accepted ✓   but NVFP4 MoE crashes at runtime ✗
--moe-backend flashinfer_trtllm   both accepted ✓   same crash ✗
   → tensorrt_llm::kernels::cutlass_kernels_oss::...kernelLauncher_Sm120_SafeFP4_SafeFP4...
     MoeGemmRunner<__nv_fp4_e2m1, __nv_fp4_e2m1, __nv_bfloat16...>::runMoe(...)

Sm120_SafeFP4 is the known-bad sm120 FP4-MoE kernel (already documented in CLAUDE.md as why the
daily driver uses --moe-backend marlin). So the two valid-for-both backends both hit it.

Red herring ruled out along the way: the flashinfer_cutlass first-compile OOM-kills the engine
on our 61GB host (ninja -j defaults high; each cutlass cicc spikes ~12GB). Fixed with MAX_JOBS=2
(ninja respects it; FLASHINFER_NVCC_THREADS defaults to 1). After the compile succeeded, it crashed
at runtime on the Sm120_SafeFP4 kernel — so the block is the kernel, not the build.

What works today (no MTP)

--moe-backend marlin + CUDA graphs (drop --enforce-eager) + the [sm120 flashinfer recipe] env
= ~116 tok/s single-stream, coherent, stable. Graph capture works fine on the Nemotron-H Mamba
hybrid (the earlier hang was cutlass-JIT vs graph-capture RAM contention, not Mamba). This is our
eval + serving config.

Unlock paths (later)

1. Re-quantize the MTP head to NVFP4 so both MoEs are quantized → marlin serves everything
   incl. MTP. Uses our quant pipeline; uncertain draft-quality payoff; not yet attempted. (most
   promising, in our control)
2. Upstream vLLM/flashinfer Sm120_SafeFP4 kernel fix so flashinfer_cutlass/trtllm NVFP4 MoE
   runs on sm120. Out of our hands.
3. A vLLM feature to set per-quant / per-submodel MoE backends (marlin for the NVFP4 main model,
   triton for the unquantized MTP head) instead of one global --moe-backend.

Refs

• Serving finding + config: memory reference_nemotron3_puzzle_75b
• sm120 flashinfer recipe: experiments/quantize/FLASHINFER-SM120-RECIPE.md
• Model: nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4

[mabry1985]

Upstream status refresh + a third backend we hadn't tried (2026-07-08)

Is this being resolved upstream? Yes — actively — though not on this (our) tracker. The sm120
NVFP4-MoE problem is one of the hotter areas in vLLM/flashinfer right now, and it has moved past the
Sm120_SafeFP4 cutlass path we hit here:

• vllm#31085 — [Feature] Add SM120 (RTX 6000/5000 Blackwell) native NVFP4 MoE kernels — OPEN, active
  (18 comments), our exact card.
• vllm#47577 — [MoE/b12x] Auto-select FLASHINFER_B12X NVFP4 MoE backend on SM120 — OPEN (draft,
  updated today). Wires the native SM12x CuteDSL fused MoE (FLASHINFER_B12X, integrated #40082 + W4A16
  #43332) into auto-selection ahead of the marlin fallback, on RTX PRO 6000 specifically.
• vllm#37113 — [SM120][GLM-5.1] NVFP4 DCP/MTP stack tracker — OPEN but stale (Apr); shows NVFP4+MTP
  running on sm120 in a custom GLM-5.1 build, so the combo is not fundamentally impossible.
• flashinfer#3119 / #2992 — OPEN, sm120 FP8×FP4 MoE tile-shape constraint + decode perf.
• flashinfer#3294 — CLOSED: flashinfer 0.6.11 (ours) ships no sm_120/121 cubins → we JIT-compile;
  a newer flashinfer ships precompiled sm12x FP4 kernels.

The third backend: flashinfer_b12x (already in our vLLM 0.22.1)

There's an NVFP4 MoE backend distinct from both marlin and the crashing cutlass/trtllm path:
--moe-backend flashinfer_b12x (config/kernel.py, NvFp4MoeBackend.FLASHINFER_B12X). It serves our
Ornith-1.0-35B-NVFP4 cleanly on sm120 — no Sm120_SafeFP4 crash. A/B vs marlin (Ornith-35B-NVFP4,
util 0.5, ctx 32K, speed-test-v2 quick):

regime      marlin   b12x    Δ
chat C1     203.0    198.9   -2.0%
chat C8     933.4    938.4   +0.5%
ctx  C1     192.7    191.1   -0.8%
ctx  C8     737.5    756.4   +2.6%
TTFT C8 p50 297/1447 248/1210 -16%   (b12x better)

Throughput parity (the upstream +10%@bs8 didn't replicate — that was on pure-MoE Qwen3.6-35B-A3B;
Ornith is a Mamba-hybrid so the MoE GEMM is a smaller compute share). Only edge = −16% TTFT @ C8.

Correction to the unlock paths above

flashinfer_b12x does NOT unblock MTP. FlashInferB12xExperts "only supports nvfp4 quantization" —
so like marlin it rejects the unquantized (bf16) MTP head. b12x is a valid (parity) alternative NVFP4
MoE serving path + a TTFT lever, but it does not dodge the mixed-quant problem.

→ Unlock path #1 (re-quantize the MTP head to NVFP4, in our control) remains the way forward. Path #2
(upstream Sm120_SafeFP4 fix) is being worked via #31085/#47577 but targets the serving path, not the
mixed-quant-MTP case. Keeping this open, tracking #47577.