Last released Dec 24, 2024
"Grams: Gradient Descent with Adaptive Momentum Scaling" implementation.
Last released Oct 5, 2024
"SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models" implementation intergrated with Hugging Face transformers
Supported by
