LlamaIndex Llms Integration: Huggingface
Installation
-
Install the required Python packages:
%pip install llama-index-llms-huggingface %pip install llama-index-llms-huggingface-api !pip install "transformers[torch]" "huggingface_hub[inference]" !pip install llama-index
-
Set the Hugging Face API token as an environment variable:
export HUGGING_FACE_TOKEN=your_token_here
Usage
Import Required Libraries
import os
from typing import List, Optional
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI
Run a Model Locally
To run the model locally on your machine:
locally_run = HuggingFaceLLM(model_name="HuggingFaceH4/zephyr-7b-alpha")
Run a Model Remotely
To run the model remotely using Hugging Face's Inference API:
HF_TOKEN: Optional[str] = os.getenv("HUGGING_FACE_TOKEN")
remotely_run = HuggingFaceInferenceAPI(
model_name="HuggingFaceH4/zephyr-7b-alpha", token=HF_TOKEN
)
Anonymous Remote Execution
You can also use the Inference API anonymously without providing a token:
remotely_run_anon = HuggingFaceInferenceAPI(
model_name="HuggingFaceH4/zephyr-7b-alpha"
)
Use Recommended Model
If you do not provide a model name, Hugging Face's recommended model is used:
remotely_run_recommended = HuggingFaceInferenceAPI(token=HF_TOKEN)
Generate Text Completion
To generate a text completion using the remote model:
completion_response = remotely_run_recommended.complete("To infinity, and")
print(completion_response)
Set Global Tokenizer
If you modify the LLM, ensure you change the global tokenizer to match:
from llama_index.core import set_global_tokenizer
from transformers import AutoTokenizer
set_global_tokenizer(
AutoTokenizer.from_pretrained("HuggingFaceH4/zephyr-7b-alpha").encode
)
LLM Implementation example
https://docs.llamaindex.ai/en/stable/examples/llm/huggingface/
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file llama_index_llms_huggingface-0.8.0.tar.gz.
File metadata
- Download URL: llama_index_llms_huggingface-0.8.0.tar.gz
- Upload date:
- Size: 7.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6d12048c69e5ffcd23b2c9fe9c67e27d8609b39456c761bbc92ea704f873ae4f
|
|
| MD5 |
c44a58b5715c822cc8189512ea415945
|
|
| BLAKE2b-256 |
5a6cbf28d9ccaa87b21b1209e86bc2bb22bfb886e61a310bc1ef2c6835529634
|
File details
Details for the file llama_index_llms_huggingface-0.8.0-py3-none-any.whl.
File metadata
- Download URL: llama_index_llms_huggingface-0.8.0-py3-none-any.whl
- Upload date:
- Size: 7.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
4c6058e0156dfe7c7a344b9f66e637ae8c5d0b283954a99e0c36e3d8304bb7b7
|
|
| MD5 |
a3cab9bcb92eaaa6ce04b5bbdda9107b
|
|
| BLAKE2b-256 |
eb2c66366d2b2da12716172e0d9a01f8bb97f586f2acef3fd7647a235ddfc4c3
|
