Implementation of PSGD optimizer in JAX
-
Updated
Dec 31, 2024 - Python
#
psgd
Here are 2 public repositories matching this topic...
Preconditioned Optimizers for MoE Training at scale, with out-of-the-box support for MuP and FSDP support for Muon, built on top of Megatron-LM and TransformerEngine.
amd optimization cuda transformers pytorch parallelism transformer muon gpt adam preconditioning kfac megatron adamw optimizers foof large-language-models psgd newton-muon locoprop
-
Updated
Jun 17, 2026
Improve this page
Add a description, image, and links to the psgd topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the psgd topic, visit your repo's landing page and select "manage topics."