Skip to content

Add Apple Silicon (MPS) support with bf16 autocast#88

Draft
andeplane wants to merge 1 commit into
Robbyant:mainfrom
andeplane:mps-support
Draft

Add Apple Silicon (MPS) support with bf16 autocast#88
andeplane wants to merge 1 commit into
Robbyant:mainfrom
andeplane:mps-support

Conversation

@andeplane

@andeplane andeplane commented Jul 17, 2026

Copy link
Copy Markdown

Summary

Enables demo.py on Apple Silicon GPUs (MPS backend), tested on an M4 / 48 GB with lingbot-map-long.pt on the bundled example/courthouse scene (--use_sdpa, since FlashInfer is CUDA-only).

  • Select mps when CUDA is unavailable; run bf16 under a device-aware autocast context
  • Make the fp32 guard around the camera/depth/point heads device-aware (autocast('cuda', enabled=False) does not disable autocast on other device types, so heads would silently run bf16 on MPS)
  • Downcast precomputed RoPE frequencies from complex128 to complex64 when moving to MPS (MPS has no float64); the fp64 precompute on CPU is unchanged
  • Fix matplotlib>=3.9 compat in the viewer (cm.get_cmap was removed in 3.9)

CUDA behavior is unchanged in all paths.

Related PRs

  • Add Mac MPS accelerated inference support #17 covers similar ground (MPS device selection with a --device flag, auto-SDPA, the same RoPE complex64 downcast) but runs MPS in fp32 without autocast. This PR runs bf16 on MPS — matching the CUDA inference dtype at roughly half the activation/KV memory — which is what required the device-aware autocast and fp32-head-guard changes above. Happy to rebase onto Add Mac MPS accelerated inference support #17's --device UX if that PR is preferred as the base.
  • Add CPU-first inference with automatic SDPA fallback #85 adds CPU-first inference with a utils/device.py device/dtype resolution layer and soft FlashInfer→SDPA fallback; if it lands first, the MPS branch here would slot naturally into those helpers.
  • Add Mac metal MLX inference #50 is a native MLX reimplementation — likely the performance ceiling on Apple Silicon, but a separate stack; this PR is the minimal torch-native path using the existing demo.py flags.

Testing

  • example/courthouse (286 frames, streaming, --mask_sky --use_sdpa) on M4 (macOS 26, torch 2.13): full run in progress at time of writing (~1.5 it/s early, slowing as the KV cache grows); will confirm completion + viewer output in a comment
  • 12-frame smoke test end-to-end through inference, post-processing, and the viser viewer

🤖 Generated with Claude Code

- Select mps device when CUDA is unavailable, with bf16 autocast
- Make autocast contexts device-aware (heads keep fp32 on all backends)
- Downcast precomputed RoPE freqs to complex64 on MPS (no float64 support)
- Fix matplotlib>=3.9 compat: cm.get_cmap removed, use matplotlib.colormaps

Runs with --use_sdpa (FlashInfer is CUDA-only).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant