BentoML: The easiest way to serve AI apps and models
Project description
BentoML Unsloth integrations
Installation
pip install "bentoml[unsloth]"
Examples.
See train.py
API
To use this integration, one can use bentoml.unsloth.build_bento:
bentoml.unsloth.build_bento(model, tokenizer)
If you model is continued froma fine-tuned checkpoint, then model_name must be passed as well:
bentoml.unsloth.build_bento(model, tokenizer, model_name="llama-3-continued-from-checkpoint")
[!important]
Make sure to save the chat templates to tokenizer instance to make sure generations are correct based on how you setup your data pipeline. See example and documentation for more information.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bentoml_unsloth-0.1.2.tar.gz
(6.0 kB
view details)
Built Distribution
File details
Details for the file bentoml_unsloth-0.1.2.tar.gz.
File metadata
- Download URL: bentoml_unsloth-0.1.2.tar.gz
- Upload date:
- Size: 6.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 9d240000990ae6871b0366c26c46a96e017d252a716076ba004045228c556c1e |
|
| MD5 | 33678f443738719e9f0d667abc7b167c |
|
| BLAKE2b-256 | 5cac88c9e2d8ff2b7d1fe00aaaa1562f298a1680934bfa2e8a24aa9016b6a43f |
File details
Details for the file bentoml_unsloth-0.1.2-py3-none-any.whl.
File metadata
- Download URL: bentoml_unsloth-0.1.2-py3-none-any.whl
- Upload date:
- Size: 6.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 2c3404cf07863fbaff579f40f9d594f0a6e73307dd76c824f34b877ce642abb7 |
|
| MD5 | e16233adb7528829b3b7e1b0598d324a |
|
| BLAKE2b-256 | 7a639b107b9e8737a3ab698911814a9f878fe61b7dab40dd8b9622c8a9a1793a |
