FMS Acceleration for Accelerated PeFT Techniques

Currently only supports LoRA-related techniques, but more are in the pipeline to be added:

Plugins

Plugin	Description	Depends	Loading	Augmentation	Callbacks
autogptq	Loads 4bit GPTQ-LoRA with quantized GPTQ as base	AutoGPTQ	✅	✅	✅
bnb	Loads 4bit QLoRA with quantized bitsandbytes Linear4	Huggingface bitsandbytes	✅	✅	✅

Key Points

fix upcasting (resulting in slowdown) issue for bnb plugin, originally discovered by inventors of Unsloth. NOTE: we recommend using mixed precision when using 4bit quant for better performance, as per our benchmarks.
bnb properly configured to work with FSDP following this guide.
triton_v2 kernels are not yet properly integrated into huggingface optimum.
triton_v2 kernels are the only 4bit kernels that work for training.

GPTQ-LORA's AutoGPTQ - Current Implementation vs Legacy Implementation

GPTQ-LORA depends on an AutoGPTQ backend to run. There are 2 backend options

Current Implementation
- This is an extracted local subset from ModelCloud's refactored fork.
- It removes redundant code to simplify build and installation of the plugin
Legacy Implementation
- This requires building the package from the official AutoGPTQ repository
- To replicate this implementation, follow the installation below
  - The legacy implementation of GPTQ-LORA uses an external AutoGPTQ package, you must ensure the specific commit is installed
```
pip install git+https://github.com/AutoGPTQ/AutoGPTQ.git@ea829c7bbe83561c2b1de26795b6592992373ef7
```
  - To construct the plugin, in the configuration object that is passed to the plugin - set use_external_lib: True (otherwise defaults to use the local AutoGPTQ package)
```
    peft:
    quantization: 
        auto_gptq:
        kernel: triton_v2
        from_quantized: True
        use_external_lib: True
```

Known Issues

GPTQ-LORA sometimes observed to have nan grad norms in the begining of training, but training proceeds well otherwise.

Metadata

Release files for fms-acceleration-peft 0.4.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for fms-acceleration-peft 0.4.4
File	Interpreter	ABI	Platform	Reset
fms_acceleration_peft-0.4.4-py3-none-any.whl 76.9 kB Oct 1, 2026	Python 3	none	any	Details

Release files / fms_acceleration_peft-0.4.4-py3-none-any.whl

Download URL	fms_acceleration_peft-0.4.4-py3-none-any.whl
Size	76.9 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`b9692ca5e0f60a2daf57b0732f7c3c82fa025ee83af4ff673ade4052b320b7a7`
BLAKE2b-256 checksum How to use checksums	`9b7220e27efb88ae4fe1a0936521028c9b20f8deab9ff265ce1a551135808eb2`
Upload date	Oct 1, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/foundation-model-stack/fms-acceleration
Publishing commit: github.com/foundation-model-stack/fms-acceleration/tree/335d9b7eb12a7b85cd883c9e92f2082fbea6047f

Workflow

Publishing configuration: github.com/foundation-model-stack/fms-acceleration/blob/335d9b7eb12a7b85cd883c9e92f2082fbea6047f/.github/workflows/build-and-publish.yml
Publishing logs: github.com/foundation-model-stack/fms-acceleration/actions/runs/36829259703/attempts/1

Artifact

Subject: fms_acceleration_peft-0.4.4-py3-none-any.whl
SHA-256 checksum: b9692ca5e0f60a2daf57b0732f7c3c82fa025ee83af4ff673ade4052b320b7a7 Verifying attestations