No project description provided
Project description
Optimus: HuggingFace-Aligned 3D-parallel backend
- flash attention 2 support on training
- flash attention 2 support on left-padding generation with kv cache
- fmha on GQA & MQA
- multi model topology support by mpu context
- more model type for experiment (PPL,RM,...)
- GQA & MQA generation (left-padding)
TODO:
- less model control option
- generator based on non-batch flash attention and self-design cuda fused kernel
- Fixed pipeline model
- KV Cache management by pre-malloc and reuse (pre-calculate)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
optimis-0.1.0.tar.gz
(154.0 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
optimis-0.1.0-py3-none-any.whl
(192.8 kB
view details)
File details
Details for the file optimis-0.1.0.tar.gz.
File metadata
- Download URL: optimis-0.1.0.tar.gz
- Upload date:
- Size: 154.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.0 CPython/3.10.12 Linux/6.6.5-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
369d143ec8f02b2992de7e99e4b67ecb27be54cf8052f1b4e43c946736cc655d
|
|
| MD5 |
993494df5ef29ad3543d5d3719e21daf
|
|
| BLAKE2b-256 |
1b335e7189fa84d5673563cfd9e033df2c6c1603c7fbae0d80673b8d4663314e
|
File details
Details for the file optimis-0.1.0-py3-none-any.whl.
File metadata
- Download URL: optimis-0.1.0-py3-none-any.whl
- Upload date:
- Size: 192.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.0 CPython/3.10.12 Linux/6.6.5-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2aeafc302fd03b64b3ff5dd84b2cf2d3add4f4d55581bde98a2bfd26ce3a41c8
|
|
| MD5 |
3f845eac8604a50bb67452b98b766891
|
|
| BLAKE2b-256 |
396113f04618a72ebcca4367b34646fd79ce79598531e1d01691f7e7c8216ed4
|
