⚡ The Problem: GPU-less Deployments

Deploying large AI models or running them on local machines without a dedicated GPU is painfully slow. Matrix multiplications inside torch.nn.Linear layers bottleneck heavily on CPU architectures because they process 32-bit floats natively.

🚀 The Solution: Raafeli

Raafeli automatically transforms your heavy FP32 PyTorch models into highly optimized INT8 (Dynamic Quantized) representations under the hood. All it takes is a single decorator. You do not need to change your architecture, deployment pipeline, or weights.

Quick Start

import torch
import torch.nn as nn
from raafeli import optimize_cpu

# 1. Your heavy model
class HeavyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(1024, 4096)
        self.fc2 = nn.Linear(4096, 1024)
        
    def forward(self, x):
        return self.fc2(torch.relu(self.fc1(x)))

model = HeavyModel()
input_data = torch.randn(1, 1024)

# 2. Decorate your prediction function
@optimize_cpu(model_arg="model", precision="int8")
def predict(model, data):
    return model(data)

# 3. Magic! First run takes a tiny fraction of a second to optimize,
#    all subsequent runs execute in INT8 natively on your CPU!
output = predict(model, input_data)

How It Works

When you call @optimize_cpu, Raafeli hooks into the execution stack. It intercepts the model object passed to your function, and aggressively applies torch.quantization.quantize_dynamic targeting performance-bound layers (like Linear and LSTM).

It caches the optimized model graph back into the object, ensuring the overhead is $0$ on every subsequent call. Your model footprint drops by ~75% and throughput spikes significantly.

Support This Project

Raafeli is an open-source project built out of passion. If it has saved you valuable GPU hours, deployment costs, or debugging time, consider supporting the creator by following on Instagram!

Contributing & Testing

We welcome PRs! To run the test suite locally and verify your changes:

# Clone the repository
git clone https://github.com/ginganomercy/raafeli.git
cd raafeli

# Install with development dependencies
pip install -e .[dev]

# Run tests
pytest tests/