Optimized version of llama3 / gemma.
Project description
llaminate
This project is a showcase for a neural tokenization technique. Since the inputs are compressed and have a smaller shape, the LLM is downsized accordingly.
For example, llama3-8b is brought down to 34 million parameters instead of 8 billion.
Installation
Usage
Resources
Models
Notebooks
Final model:
- pretraining: file / Google Colab
- fine-tuning: file / Google Colab
TODO
See TODO.
Credits
This project winks at llama3 from Meta, but doesn't actually its weights nor code.
License
Licensed under the aGPLv3.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file llaminate-0.6.8.tar.gz.
File metadata
- Download URL: llaminate-0.6.8.tar.gz
- Upload date:
- Size: 5.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/2.0.1 CPython/3.13.2 Linux/6.13.8-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
40b2d316ce0700898d66427b7e910d1b09662f9b57c7e3be635486d1defa6610
|
|
| MD5 |
ccaaad7e28e40753489c7832fceaf631
|
|
| BLAKE2b-256 |
c7ef73247406bf7006aa7e16a6a21d213203eb8d564c20451d7705503897e65a
|
File details
Details for the file llaminate-0.6.8-py3-none-any.whl.
File metadata
- Download URL: llaminate-0.6.8-py3-none-any.whl
- Upload date:
- Size: 7.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/2.0.1 CPython/3.13.2 Linux/6.13.8-arch1-1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
52f1c0b459de847bd2532d04c2d54836490365fdb0f16ea736837977828d4d67
|
|
| MD5 |
89e4823736735f605cf3a31396080249
|
|
| BLAKE2b-256 |
6ff4f7c11dcd1565565168de7b48fa0cc1a913d41cb6aecdf8cce8dbd30897db
|
