No project description provided
Project description
Optimus: HuggingFace-Aligned 3D-parallel backend
- flash attention 2 support on training
- flash attention 2 support on left-padding generation with kv cache
- fmha on GQA & MQA
- multi model topology support by mpu context
- more model type for experiment (PPL,RM,...)
- GQA & MQA generation (left-padding)
TODO:
- less model control option
- generator based on non-batch flash attention and self-design cuda fused kernel
- Fixed pipeline model
- KV Cache management by pre-malloc and reuse (pre-calculate)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
platformers-0.1.0.tar.gz
(153.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
platformers-0.1.0-py3-none-any.whl
(193.5 kB
view details)
File details
Details for the file platformers-0.1.0.tar.gz.
File metadata
- Download URL: platformers-0.1.0.tar.gz
- Upload date:
- Size: 153.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.0 CPython/3.10.12 Linux/6.6.5-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c128085299d30b90f333e682a5ef946cc8863d4abd7b4f2eceb6e3fbc07b1dcc
|
|
| MD5 |
934fc268404018fbd2cca835213ffca3
|
|
| BLAKE2b-256 |
fd9f540a629d7f7670c7bb4662395eb4515aca0f34db91f1b09d516667d4050d
|
File details
Details for the file platformers-0.1.0-py3-none-any.whl.
File metadata
- Download URL: platformers-0.1.0-py3-none-any.whl
- Upload date:
- Size: 193.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.0 CPython/3.10.12 Linux/6.6.5-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d3d183f1a23be4e5714e35e5acdac9046e7ca82766b7f7cbcf2f962e7aff1d2a
|
|
| MD5 |
12425e69c2c7deaa110b7240885b738a
|
|
| BLAKE2b-256 |
7cb300dea7a19eb66f12e897e225901820879bb7fd20472bf71cd6e6b032cafe
|
