A place to store reusable transformer components found around the interwebs
Project description
transformer_nuggets
A place to store reusable transformer components of my own creation or found on the interwebs
Getting Started
Clone the repository:
git clone https://github.com/drisspg/transformer_nuggets.git
Install Package
pip install -e .
Dev Tool Chain
pip install -e ".[dev]"
pre-commit is used to make sure that I don't forget to format stuff, I am going to see if I like this or not. This should be installed when installing the dev tools.
Project Structure
-
benchmarks: Contains scripts and data related to benchmarking the transformer components.
- data: Benchmark data files.
flash.py
: Benchmarking script for Flash.llama.py
: Benchmarking script for Llama.qlora.py
: Benchmarking script for Qlora.fp8_sat_cast.py
: Benchmarks for comparing FP8 saturated casting kernel to eager and compile code.
-
transformer_nuggets: The main directory containing all transformer components/modules.
- flash: Components related to the FlashAttention.
- quant: Implementation of NF4 Tensor and QLora in pure Pytorch
- sdpa: Prototype for updated SDPA interface in Pytorch.
- fp8: Components related interacting with PyTorch FP8 tensors.
- llama: Contains a model def for llama2 models as well as a pretraining script.
- utils: General utility functions and scripts.
benchmark.py
: Benchmark-related utility functions.tracing.py
: Tracing utilities for transformers.
-
test: Contains test scripts for various transformer components.
test_flash.py
: Tests for Flash.test_qlora.py
: Tests for Qlora.test_sdpa.py
: Tests for SDPA.test_fp8.py
: Tests for FP8.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
transformer_nuggets-0.0.1.tar.gz
(52.9 kB
view hashes)
Built Distribution
Close
Hashes for transformer_nuggets-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2ace9ec03f21f9d511f31adbb29c069616fa1319c5c41b4af5e201000b7be661 |
|
MD5 | ffdf2d6a7473963d4bfc2ce2c1f975d8 |
|
BLAKE2b-256 | de27a051c48c614be67252f30caec8a54eb6ee128b99e837520b98db820a3ad8 |
Close
Hashes for transformer_nuggets-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 006bfb92808cedcaeda6c1bcb6f83d5faf4cc57fb7e2e851dbb37c62b9f70879 |
|
MD5 | f2640683443f979bfb0c9126c258cf32 |
|
BLAKE2b-256 | 2eb3ad4c9b45bce5180f04a49631834831e393c2b2ae47fd3f84274f71306410 |