tigris-ml 0.3.0

TiGrIS - Tiled Graph Inference Scheduler for edge ML deployment

TiGrIS

Tiled Graph Inference Scheduler. An ahead-of-time compiler that tiles ML models to fit embedded devices with hard memory budgets.

Give it an ONNX model and a memory budget. It partitions the compute graph into stages, tiles spatial operations, and emits a flat binary plan that the tigris-runtime executes with zero dynamic allocation.

The problem

On an embedded device with a few hundred KB of SRAM, most interesting models simply don't fit. The usual answer is to shrink the model: quantize harder, prune, pick a smaller architecture, and hope the accuracy hit is acceptable.

TiGrIS takes the other approach. It keeps the model you trained and rearranges the computation so that only a small working set lives in SRAM at any moment. Weights and intermediate spills go to flash or PSRAM. What comes out is a binary plan that the runtime executes as a flat sequence of kernel calls, with no interpreter, no tensor allocator, and no dynamic memory at all.

Quick start

pip install tigris-ml

# Will this model fit in 256KB SRAM + 16MB flash?
tigris analyze mobilenetv2.onnx -m 256K -f 16M

╭──────────────────────── TiGrIS - mobilenetv2 ────────────────────────╮
│ Operators            65                                              │
│ Peak memory (naive)  4.59 MiB                                        │
│ Largest tensor       1x96x112x112 (4.59 MiB)                         │
╰──────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────── SRAM ────────────────────────────────╮
│ Budget              256.00 KiB                                       │
│ Scheduled peak      254.62 KiB (5.4% of naive peak)                  │
│ Stages              42                                               │
│ Need tiling         31 of 42 stages                                  │
╰────────────────  PASS - tiling resolves all stages  ─────────────────╯

The naive peak is 4.59 MiB. TiGrIS schedules it into 256 KiB through temporal partitioning and spatial tiling. analyze runs on your laptop; no hardware required.

From ONNX to embedded

Three steps take a model from ONNX to a C file you can drop into your firmware project:

# 1. Analyze feasibility against a memory budget
tigris analyze model.onnx -m 256K -f 16M

# 2. Compile to a binary plan (weights read-in-place from flash)
tigris compile model.onnx -m 256K -f 16M --xip -o model.tgrs

# 3. Generate a backend-specific C harness for your target
tigris codegen model.tgrs --backend esp-nn -o model.c

The .tgrs plan is target-agnostic: it is the same file whether you run it on an ESP32, a Cortex-M, or a POSIX host for testing. The choice of kernel backend happens at codegen time and decides which kernel library the generated C calls into.

Several kernel backends are available (portable C99, ESP32 family, Cortex-M family); see tigris-runtime for the current list. Switching between them is a --backend flag, not a rewrite.

What you get

tigris compile writes a single .tgrs file that contains the operator schedule, tile parameters, quantization tables, and the weights. This file goes on flash at deployment time.

tigris codegen produces a small C harness that locates the plan on flash at runtime and hands it to the runtime:

• declarations for the input/output buffers and the arena
• a tigris_run_once() entry point that sets up memory and calls the runtime
• backend-specific glue for finding the plan: partition mmap on ESP-IDF, an extern flash symbol on Cortex-M, a file path on POSIX

Link the harness against tigris-runtime and your chosen kernel library, flash the .tgrs alongside the firmware, and you have a working inference binary.

Further reading

• Getting started: installation, first compile, deploying to ESP32
• Introducing TiGrIS: design, benchmarks, how tiling works
• CLI reference: every flag, every subcommand

Development

git clone https://github.com/raws-labs/tigris
cd tigris
pip install -e ".[dev]"
pytest