ModelForge

ModelForge turns supported machine-learning artifacts into deployable inference-service projects.

The current MVP supports ONNX models end to end: model inspection, runtime selection, conservative optimization planning, local benchmarking, and generation of a FastAPI/Docker service. TorchScript inspection is available for artifacts you explicitly trust.

Requirements

• Python 3.11 or newer
• ONNX Runtime support: install the onnx extra
• Docker is optional and is only needed to build/run generated containers

Install

Install the project in editable mode with ONNX support:

pip install -e ".[onnx]"

Available optional extras:

• .[onnx] — ONNX inspection and ONNX Runtime inference
• .[pytorch] — trusted TorchScript/PyTorch inspection
• .[gpu] — GPU-enabled ONNX Runtime package
• .[all] — all optional integrations
• .[dev] — test, lint, and type-check tooling

Confirm the CLI is installed:

modelforge --help

Quick start

An example ONNX model is included at examples/models/model.onnx.

modelforge inspect examples/models/model.onnx
modelforge benchmark examples/models/model.onnx
modelforge build examples/models/model.onnx --output ./model-service

The build output contains the model, FastAPI application, Dockerfile, runtime requirements, manifest, README, and a smoke-test placeholder.

CLI commands

Inspect a model

modelforge inspect path/to/model.onnx

This reports detected model format, tensor inputs/outputs, graph metadata, and parameter count where available.

Show the optimization plan

modelforge optimize path/to/model.onnx

ModelForge records graph, precision, quantization, and batching decisions. Unsupported or unsafe optimizations are reported as skipped rather than applied silently.

Benchmark inference

modelforge benchmark path/to/model.onnx

For ONNX models with fully known numeric shapes, ModelForge creates zero-filled synthetic inputs and labels the result accordingly. Supply representative inputs through the Python API for production-representative measurements.

Generate a service

modelforge build path/to/model.onnx --output ./model-service

Use --verbose for diagnostic logging or --quiet to suppress normal logs:

modelforge --verbose inspect path/to/model.onnx

serve is reserved for the local-serving workflow and is not yet implemented in the MVP.

Run a generated service

After building an ONNX CPU service:

cd model-service
pip install -r requirements.txt
uvicorn app.main:app --host 0.0.0.0 --port 8000

Endpoints:

• GET /health — readiness status
• GET /metadata — generated ModelForge manifest
• POST /predict — inference

For a model whose input is named data_0, submit JSON like:

curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"inputs":{"data_0":[[[[0.0]]]]}}'

The tensor shape and dtype must match the model metadata returned by modelforge inspect or the generated /metadata endpoint.

Docker

From the generated service directory:

docker build -t my-model-service .
docker run --rm -p 8000:8000 my-model-service

The generated CPU Dockerfile uses python:3.12-slim, runs the application as a non-root user, and includes a healthcheck. It does not install unrelated frameworks such as TensorFlow or PyTorch.

Python API

from modelforge import ModelForge, benchmark_model, build_service, inspect_model

metadata = inspect_model("examples/models/model.onnx")
print(metadata.inputs)

benchmark = benchmark_model("examples/models/model.onnx")
print(benchmark.latency_ms)

result = build_service(
    "examples/models/model.onnx",
    output_dir="./model-service",
)
print(result.output_dir)

forge = ModelForge("examples/models/model.onnx")
plan_metadata, hardware, plan = forge.plan()

Configuration

BuildConfig can be constructed from a YAML file:

model:
  path: examples/models/model.onnx

optimization:
  precision: auto
  quantization: auto
  graph_optimization: auto

runtime:
  backend: auto
  provider: auto

container:
  generate: true
  gpu: auto

from pathlib import Path
from modelforge.config import BuildConfig

config = BuildConfig.from_yaml(Path("modelforge.yaml"))

Configuration parsing is available now; complete CLI configuration-file merging is planned for a future release.

Security notes

• Do not treat arbitrary model files as safe.
• Generic PyTorch checkpoints can deserialize Python objects. ModelForge refuses them unless you set trusted_artifact=True and you control the source.
• Prefer ONNX or TorchScript for portable inference artifacts.
• Generated containers do not include secrets; pass configuration through environment variables or your deployment platform.

Development

pip install -e ".[dev,onnx]"
pytest

The repository's included ONNX model is used by the integration tests. Docker validation depends on local Docker permissions; if Docker Buildx is unavailable, generated assets are still tested at the file level.

MVP limitations

ModelForge intentionally reports unsupported work instead of pretending it ran:

• Only ONNX has an end-to-end generated service path today.
• FP16 and GPU execution require compatible hardware/runtime support and are not enabled automatically on CPU-only hosts.
• Static INT8 requires calibration data; without it, ModelForge records that static quantization was not performed.
• TensorFlow, scikit-learn, TensorRT, cloud deployment, and Kubernetes are planned extension points, not current integrations.