pocket-inference 0.2.0

A lightweight Rust library for running Keras models without the TensorFlow stack.

Binary size: ~450 KB · Designed for resource-constrained environments

Features

Installation

pip install pocket-inference

Prebuilt wheels are published for Linux (x86_64, aarch64) and macOS (Apple Silicon), and cover every Python version from 3.7 onwards via the stable ABI.

Usage

import numpy as np
from pocket_inference import Sequential

# Load a saved Keras model
model = Sequential.load("model.keras")

# Run inference on a single sample
input_data = np.array([[1.0, 2.0, 3.0, 4.0]], dtype=np.float32)
output = model.predict(input_data)
print(output)

# Batch inference
batch = np.random.randn(32, 4).astype(np.float32)
batch_output = model.predict(batch)

# Introspection
print(model.summary())
print(model.num_layers(), model.layer_names())

Roadmap

• Additional layer types (LSTM, GRU, ...)
• More activation functions (LeakyReLU, ELU, GELU, ...)
• GPU acceleration via compute shaders
• Model optimization and pruning tools
• Benchmark suite and performance metrics

Development

git clone https://github.com/James-Wirth/pocket-inference.git
cd pocket-inference
python -m venv .venv && source .venv/bin/activate
pip install maturin
maturin develop --release --features python

License

Dual-licensed under MIT or Apache 2.0.