ovkit

OpenVINO inference in 3 lines. One Model class, clean Results, 30+ ready models — with AUTO/NPU/GPU devices, async throughput, and INT8.

Docs · 한국어 문서 · Model catalog · Examples

from ovkit import Model

r = Model("detect")("image.jpg")[0]   # download -> convert -> cache -> run
r.save("out.jpg")                     # boxes drawn; r.boxes.xyxy / .conf / .cls

Or without writing Python at all:

ovkit run detect image.jpg            # prints results, saves image_out.jpg

Install

git clone https://github.com/leeyunjai82/ovkit.git && cd ovkit
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e .

Extras: ".[quant]" (INT8/NNCF) · ".[genai]" (LLM/STT) · ".[all]". Python 3.10+.

Supported tasks

Every task below runs end-to-end through the same 3 lines — swap the alias:

The registry exposes one well-tested model per capability (35 total); the HF mirror hosts the full Apache-2.0 OMZ set (other tiers, int8, sparse variants) — surfacing a variant is a one-line edit (catalog). ovkit list shows everything with descriptions.

Devices

Single images run synchronously; stream=True uses an AsyncInferQueue for video/webcam throughput. INT8: model.quantize(calib_images) (NNCF).

Benchmarks

python scripts/benchmark.py        # prints a paste-ready CPU/GPU/NPU table

Usage

from ovkit import Model

model = Model("face_detection")              # alias, name, .xml, or .onnx
results = model("photo.jpg", conf=0.25)      # image / ndarray / folder / video
for r in model.predict(0, stream=True):      # webcam (lazy generator)
    annotated = r.plot()

print(Model("age_gender")("face.jpg")[0].text)   # "age 31 · male 98%"

ovkit run detect image.jpg --save out.jpg   # one-shot inference
ovkit run age_gender face.jpg --device NPU
ovkit list                                  # aliases + models with descriptions
ovkit info face_detection                   # source / task / license
ovkit download detect                       # warm the cache
ovkit devices                               # available OpenVINO devices

from ovkit.genai import pipeline

llm = pipeline("llm")                        # tinyllama_chat from the mirror
print(llm.generate("Explain OpenVINO in one sentence.", max_new_tokens=64))

stt = pipeline("stt")                        # whisper_base
print(stt.generate(audio_16k_mono_float32))

pip install -r examples/requirements.txt
python examples/web_app.py                   # http://127.0.0.1:8000

Adding a model

Models are data, not code — one manifest entry (src/ovkit/manifests/):

my_model:
  src: hf
  repo: leeyunjai/ovkit-models
  filename: detect/my_model/model.xml
  task: detect
  description: Shown by `ovkit list`.
  license: apache-2.0            # must be permissive — enforced at load time

Resolution: alias → local path → cache (~/.cache/ovkit) → download → convert → cache, with atomic writes, sha256 checks, upstream fallback, and OVKIT_OFFLINE=1. Maintainer tooling (mirror build / verify / self-check / benchmark) lives in scripts/ — see the guide.

License

ovkit is Apache-2.0 and license-clean by design: only permissive (Apache/MIT/BSD) models and libraries — no AGPL model stacks, no non-commercial weights; every manifest entry must declare a permissive license (enforced at load time).