optimum-rbln 0.1.15

Optimum RBLN is the interface between the Hugging Face Transformers and Diffusers libraries and RBLN accelerators. It provides a set of tools enabling easy model loading and inference on single and multiple rbln device settings for different downstream tasks.

Optimum RBLN

🤗 Optimum RBLN provides an interface between Hugging Face libraries (Transformers, Diffusers) and RBLN Accelerators, including ATOM and REBEL.

This library enables seamless integration between the Hugging Face ecosystem and RBLN's NPU acceleration through a comprehensive toolkit for model loading and inference across single- and multi-Accelerator environments. While we maintain a list of officially validated models and tasks, users can easily adapt other models and tasks with minimal modifications.

Key Features

🚀 High Performance Inference

• Optimized model execution on RBLN NPUs through RBLN SDK compilation
• Support for both single-NPU and multi-NPU inference
• Integrated with RBLN Runtime for optimal performance

🔧 Easy Integration

• Seamless compatibility with Huggingface model hub
• Drop-in replacement for existing Huggingface pipelines
• Minimal code changes required for NPU acceleration

Documentation

Check out the documentation of Optimum RBLN for more advanced usage.

Getting Started

Install from PyPI

To install the latest release of this package:

• Export environment variables to access to RBLN private PyPI.

  export REBEL_PYPI_USERNAME=<username>
  export REBEL_PYPI_PASSWORD=<password>
  

• Install optimum-rbln package:

  pip install --index-url https://pypi.rebellions.in/simple optimum-rbln
  

Install from source

Prerequisites

• Install uv (refer this link for detailed commands)

The below command installs optimum-rbln along with its dependencies.

git clone https://github.com/rebellions-sw/optimum-rbln.git
cd optimum-rbln
./scripts/uv-sync.sh

If you want to install local rebel-compiler as editable mode in uv environment,

uv pip install -e /path/to/rebel_compiler/python

Need Help?

• Join our Developer Community
• Contact maintainers at [e-mail]