A collection of sparse attention mechanisms
Project description
SparseAttnFunctions
Efficient Implementations of Sparse Attention Mechanisms
This repository contains multiple optimized implementations of sparse attention techniques, specifically tailored for CogVideo2B. These modifications enhance computational efficiency and memory usage, making them ideal for large-scale video generation tasks.
Key Features:
• Sparse Attention Variants: Includes several approaches to sparse attention, such as windowed attention, block-sparse attention, and more.
• CogVideo2B Integration: Customized to seamlessly integrate with the CogVideo2B framework, ensuring optimal performance.
• Efficiency: Designed to reduce memory footprint and accelerate computation, especially for high-resolution video generation.
Use Cases:
• Large-scale video generation tasks.
• Applications requiring efficient attention mechanisms for long sequences.
Installation
• pip install .
Contributing:
Contributions are welcome! Please follow the Contribution Guidelines for more details.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file special_attentions_pack-0.3.0.tar.gz.
File metadata
- Download URL: special_attentions_pack-0.3.0.tar.gz
- Upload date:
- Size: 47.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
a66f15a46f878cdce7b5a6a91e2a5c4bc662eb4de708f977499d657c62e46fee
|
|
| MD5 |
7dde580d0e1c7450769025e094c1bdde
|
|
| BLAKE2b-256 |
866a4556c99cd3bca7249f4c90a2974185465966fa4e31dd2ac882b03b3dfe92
|
File details
Details for the file special_attentions_pack-0.3.0-py3-none-any.whl.
File metadata
- Download URL: special_attentions_pack-0.3.0-py3-none-any.whl
- Upload date:
- Size: 55.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
80a48b863f07de300c01ae650c0675fc97c883248bbad5e2339752e91078c8bd
|
|
| MD5 |
4c3b9547999bf1fb73ce1d6283994f20
|
|
| BLAKE2b-256 |
7ae564e026ea4e8b9b3aa35b3a8786f19bd2eb41c6c29c35d7eb08e02fac28bf
|
