LlamaIndex Llms Integration: Huggingface

Installation

Install the required Python packages:

%pip install llama-index-llms-huggingface
%pip install llama-index-llms-huggingface-api
!pip install "transformers[torch]" "huggingface_hub[inference]"
!pip install llama-index

Set the Hugging Face API token as an environment variable:
```
export HUGGING_FACE_TOKEN=your_token_here
```

Usage

Import Required Libraries

import os
from typing import List, Optional
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.llms.huggingface_api import HuggingFaceInferenceAPI

Run a Model Locally

To run the model locally on your machine:

locally_run = HuggingFaceLLM(model_name="HuggingFaceH4/zephyr-7b-alpha")

Run a Model Remotely

To run the model remotely using Hugging Face's Inference API:

HF_TOKEN: Optional[str] = os.getenv("HUGGING_FACE_TOKEN")
remotely_run = HuggingFaceInferenceAPI(
    model_name="HuggingFaceH4/zephyr-7b-alpha", token=HF_TOKEN
)

Anonymous Remote Execution

You can also use the Inference API anonymously without providing a token:

remotely_run_anon = HuggingFaceInferenceAPI(
    model_name="HuggingFaceH4/zephyr-7b-alpha"
)

Use Recommended Model

If you do not provide a model name, Hugging Face's recommended model is used:

remotely_run_recommended = HuggingFaceInferenceAPI(token=HF_TOKEN)

Generate Text Completion

To generate a text completion using the remote model:

completion_response = remotely_run_recommended.complete("To infinity, and")
print(completion_response)

Set Global Tokenizer

If you modify the LLM, ensure you change the global tokenizer to match:

from llama_index.core import set_global_tokenizer
from transformers import AutoTokenizer

set_global_tokenizer(
    AutoTokenizer.from_pretrained("HuggingFaceH4/zephyr-7b-alpha").encode
)

LLM Implementation example

https://docs.llamaindex.ai/en/stable/examples/llm/huggingface/

Metadata

Release files for llama-index-llms-huggingface 0.8.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llama-index-llms-huggingface 0.8.0
File	Size	Uploaded
llama_index_llms_huggingface-0.8.0.tar.gz	7.9 kB	Aug 31, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for llama-index-llms-huggingface 0.8.0
File	Interpreter	ABI	Platform	Reset
llama_index_llms_huggingface-0.8.0-py3-none-any.whl 7.8 kB Aug 31, 2026	Python 3	none	any	Details

Total release size: 15.6 kB

Release files / llama_index_llms_huggingface-0.8.0.tar.gz

Download URL	llama_index_llms_huggingface-0.8.0.tar.gz
Size	7.9 kB
Tags	Source
SHA-256 checksum How to use checksums	`6d12048c69e5ffcd23b2c9fe9c67e27d8609b39456c761bbc92ea704f873ae4f`
BLAKE2b-256 checksum How to use checksums	`5a6cbf28d9ccaa87b21b1209e86bc2bb22bfb886e61a310bc1ef2c6835529634`
Upload date	Aug 31, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / llama_index_llms_huggingface-0.8.0-py3-none-any.whl

Download URL	llama_index_llms_huggingface-0.8.0-py3-none-any.whl
Size	7.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`4c6058e0156dfe7c7a344b9f66e637ae8c5d0b283954a99e0c36e3d8304bb7b7`
BLAKE2b-256 checksum How to use checksums	`eb2c66366d2b2da12716172e0d9a01f8bb97f586f2acef3fd7647a235ddfc4c3`
Upload date	Aug 31, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`