skewadam

Tiered optimizer state allocation for memory-efficient MoE training. Cuts optimizer memory by 97.4%, outperforming AdamW/Muon/Lion while fitting a 6.78B MoE on a single 40GB GPU.