GrillyOptimum (Alpha not production ready)

HuggingFace Optimum-compatible Vulkan backend — optional grilly extension.

Features

• from_pretrained() — load any HF Llama model directly
• generate() — HuggingFace-compatible generation interface
• Pipeline integration — use with transformers.pipeline("text-generation")
• Vulkan acceleration — native fp16 inference on RDNA2 GPUs

Quick Start

pip install grillyoptimum

from grillyoptimum import VulkanModelForCausalLM
from transformers import AutoTokenizer

model = VulkanModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# HF-style generation
input_ids = tokenizer.encode("Hello, world!", return_tensors="np")
output = model.generate(input_ids, max_new_tokens=100, temperature=0.7)
print(tokenizer.decode(output[0]))

# Pipeline
from grillyoptimum.pipeline import create_text_generation_pipeline
pipe = create_text_generation_pipeline("meta-llama/Llama-3.2-3B-Instruct")
result = pipe("Explain quantum computing")
print(result[0]["generated_text"])

Requirements

• Python 3.12+
• grilly >= 0.4.0
• grillyinference >= 0.1.0
• transformers

License

MIT