💎 ONYX AI Gemma 4 Engine (E2B Edition)

A high-performance, professional FastAPI wrapper for Gemma Multimodal models with built-in 4-bit quantization and streaming support. Developed by ONYX (RUI Company).

🚀 Features

• Zero Config Integration: Deploy a multimodal AI server in seconds.

• Optimized Performance: Native 4-bit quantization using bitsandbytes for low VRAM/RAM usage.

• Real-time Streaming: Built-in SSE (Server-Sent Events) for smooth, token-by-token generation.

• Hardware Friendly: Optimized for both GPU and high-performance CPU inference.

• 🌊 Streaming Support (Text-Only) The engine supports real-time streaming for text-based conversations only. This allows the model to respond token-by-token using Server-Sent Events (SSE).

• Note: Streaming is currently limited to text inputs and outputs. Multimodal inputs (images/video) are processed using the standard predict endpoint.

📦 Installation

Option 1: Via pip

You can install the engine directly from PyPI: 📦 Installation Option 1: Install via pip

pip install onyx-AI-Gemma4

Option 2: requirements.txt

fastapi
uvicorn
transformers>=4.48.0
torch
accelerate
bitsandbytes
Pillow
torchvision
onyx-AI-Gemma4

💻 Usage ▶ Standard Script

from ONYXAI_Gemma4E2B import OnyxEngine

# Initialize the engine
engine = OnyxEngine(model_id="google/gemma-4-E2B-it")

# Run the server
if __name__ == "__main__":
    engine.run(host="0.0.0.0", port=7860)

🌐 Production

from ONYXAI_Gemma4E2B import OnyxEngine
import uvicorn
import os

engine = OnyxEngine(model_id="google/gemma-4-E2B-it")
app = engine.app

@app.get("/")
def home():
    return {"message": "ONYX Engine is running!"}

if __name__ == "__main__":
    port = int(os.environ.get("PORT", 7860))
    uvicorn.run(app, host="0.0.0.0", port=port)

🛠 API Usage Endpoint

POST /predict

Example Request

{
  "messages": [
    {
      "role": "user",
      "content": "Write a long story about space exploration."
    }
  ],
  "stream": true,
  "temperature": 0.7,
  "max_tokens": 1024
}

🔗 Links Organization: ONYX / RUI Company Author: Eng. Rawan Jassim

© 2026 ONYX. All rights reserved.