Memory-efficient PyTorch optimizers with Adaptive Log-Space quantization, fused CUDA kernels, and Adafactor, CAME, APOLLO support.
-
Updated
Aug 25, 2026 - Python
Memory-efficient PyTorch optimizers with Adaptive Log-Space quantization, fused CUDA kernels, and Adafactor, CAME, APOLLO support.
reproduce Adam, AdamW, Adafactor optimizors with pytorch, and introduce popular optimizers in the training of the LLMs.
A from-scratch 336M-parameter language model built with JAX and Equinox
Add a description, image, and links to the adafactor topic page so that developers can more easily learn about it.
To associate your repository with the adafactor topic, visit your repo's landing page and select "manage topics."