VaporRAM 💨

VaporRAM is a lightweight, zero-dependency inference engine written in pure C. It is specifically engineered to run google/gemma-4-E4B-it (8-billion parameter state-of-the-art model) on consumer hardware under a strict 1.5 GB RAM ceiling by streaming layers directly from NVMe SSD storage into RAM.

Key Features

• Extreme Hardware Accessibility: Run an 8B parameter model under a strict 1.5 GB RAM ceiling (measured peak RSS: 142.3 MB).
• Sequential Layer Pipeline (SLP): Zero-copy unbuffered O_DIRECT NVMe SSD layer streaming with asynchronous POSIX kernel prefetching hints (POSIX_FADV_WILLNEED).
• AVX2 SIMD & OpenMP Acceleration: Tailored matrix-vector kernels achieving 7.70x speedup over scalar loops (204,795 GFLOPS).
• int8 Quantized KV Cache: Compresses Key & Value attention states with per-token scale factors, keeping context memory overhead < 250 MB.
• OpenAI-Compatible API: Built-in HTTP server supporting /v1/chat/completions, /v1/responses, /v1/models, and /health.
• Web UI & Interactive CLI: Includes an interactive terminal chat mode (vapor chat) and a web dashboard (vapor web).

Hardware & System Requirements

Installation

Option 1: Install via PyPI (Recommended)

pip install vapor-ram

Option 2: Prebuilt Release

Download and extract the latest prebuilt binary tarball:

mkdir vapor-ram && cd vapor-ram
tar xzf vapor-ram-v1.0.1-linux-x86_64.tar.gz

Option 3: Build from Source

Clone the repository and compile using make:

git clone https://github.com/sudsarkar13/vapor-ram.git
cd vapor-ram
make -C c

Usage Guide

The project includes a CLI launcher called vapor (./vapor or python3 vapor).

1. System Diagnostics & Resource Planning

Run diagnostics to check system capabilities and memory budget compliance:

# Run hardware diagnostic checks
./vapor doctor

# View RAM ceiling budget breakdown (< 1.5 GB)
./vapor plan

2. Interactive Terminal Chat

Launch an interactive chat session:

./vapor chat --preset coder

3. One-Shot Prompt Generation

Execute a quick single prompt generation from the command line:

./vapor run "Explain quantum computing in simple terms."

4. OpenAI-Compatible API Server

Start an HTTP server supporting OpenAI endpoints (/v1/chat/completions):

./vapor serve --host 0.0.0.0 --port 8000

Query the API using curl:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-E4B-it",
    "messages": [{"role": "user", "content": "Hello! What can you do?"}]
  }'

5. Web Interface

Start the server and automatically launch the Web UI in your default browser:

./vapor web

Configuration & Preset Flags

You can customize execution using presets or flags:

Project Structure

• c/vapor_engine: Compiled C SIMD inference engine binary.
• vapor: Main Python CLI frontend launcher.
• doctor.py: Installation and hardware diagnostic script.
• openai_server.py: OpenAI-compatible HTTP API server implementation.
• resource_plan.py: Dynamic memory budget calculation planner.
• version.py: Engine version information.
• web/: Frontend dashboard UI static assets.
• docs/: GitHub Pages documentation website and screenshot guides.

License

This project is licensed under the Apache 2.0 License. See the LICENSE file for details.