BentoML: The easiest way to serve AI apps and models
Project description
BentoML Unsloth integrations
Installation
pip install "bentoml[unsloth]"
Examples.
See train.py
API
To use this integration, one can use bentoml.unsloth.build_bento
:
bentoml.unsloth.build_bento(model, tokenizer)
If you model is continued froma fine-tuned checkpoint, then model_name
must be passed as well:
bentoml.unsloth.build_bento(model, tokenizer, model_name="llama-3-continued-from-checkpoint")
[!important]
Make sure to save the chat templates to tokenizer instance to make sure generations are correct based on how you setup your data pipeline. See example and documentation for more information.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bentoml_unsloth-0.1.0.tar.gz
(6.0 kB
view hashes)
Built Distribution
Close
Hashes for bentoml_unsloth-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 79b449df4661c92e4c58aaa997f41c3964fc59ed71148eebd8bf2feeb1e73335 |
|
MD5 | 526df9dda94dc71b5a3c0c1bbd26a8e9 |
|
BLAKE2b-256 | 2fe30f72909380c929f0b803511bb0551d12bf59c3e55bfea18bd4b58c8f387f |