Skip to main content

okf-embed — Jina v5 text embeddings (Rust core, PyO3)

Exact port of EmbeddingEngine._encode: task prefix → tokenize (8192) → ONNX forward → last-token pooling → L2 → Matryoshka truncate → re-normalise. Pinned against a numpy/transformers replication of that pipeline by tests/test_parity.py (max abs diff ≤ 1e-5, cosine ≥ 0.999999).

The only embedding backend. There is no Python fallback stack, no embedding_backend selector, and no optimum/transformers in the runtime path — a mid-run stack switch would silently mix vector spaces in one index, so the design is fail-fast instead.

Install

Published to PyPI — okfgraph pulls it in automatically (platform wheels for Linux / Windows / macOS-arm64, Python 3.11–3.13):

uv sync          # editable path source, builds via maturin

From source (needs a Rust toolchain 1.85+ and maturin; maturin develop needs pip, which uv venvs lack — build the wheel and install it instead):

cd rust/okf-embed
maturin build --release
uv pip install --python <venv> target/wheels/okf_embed-*.whl --reinstall

Runtime: ONNX Runtime discovery

ort loads dynamically (load-dynamic, same pin as bobine: 2.0.0-rc.13). Resolution order: ORT_DYLIB_PATH first (user override always wins), else the pip-installed onnxruntime/onnxruntime-gpu build when unset: Windows uses capi/onnxruntime.dll, macOS uses capi/libonnxruntime.dylib, and Linux prefers versioned capi/libonnxruntime.so.* with capi/libonnxruntime.so as fallback. GPU DLL warming and Windows DLL-directory setup are best-effort and never fatal. OKFgraph's resolve_ort_dylib() runs before the native module is imported and exposes the choice as OKFRouter.ort_dylib, so both bobine and okf-embed share one ORT binary — no version/CUDA drift between ingest and import.

Lifecycle: lazy session, cheap tokenizer

JinaV5.open (model download + ONNX session build) is the single expensive step. OKFRouter therefore holds a lazy proxy: construction validates the wheel import and device string eagerly, but the session opens on the first real encode — PPR search, budgeted reads, diff, and doctor stay cold.

JinaTokenizer.open fetches only tokenizer.json for exact token counts without the session. The truncation policy is shared, so counts are identical to the session path (verified). A failed session open is cached and re-raised — configuration errors fail fast once, not once per encode.

Explicit local files (air-gapped)

JinaV5.open_files(onnx_path, tokenizer_path) and JinaTokenizer.open_files(tokenizer_path) skip every download. The sidecar (model.onnx_data-style) must sit next to the ONNX file — ORT resolves it relative to the model path, same as the HF cache layout. OKFRouter(model_path=..., tokenizer_path=...) uses them (both or neither; missing files raise FileNotFoundError at construction). Same bytes in → same vectors out (test-pinned against HF acquisition).

Session/threading policy (measured, Phase 6)

Tuning is Level3, intra = physical-cores/2, inter = 1 — kept because it measured fastest, not because it was inherited. Reference box: Windows, 32 logical cores, CPU-only ORT 1.29, warm model cache, best-of-5 reps on 4 fixed docs (short → ~400 tokens):

Config Session cold open encode_batch (4 docs) Notes
Level3, intra=16, inter=1 (current) 4.7 s 375 ms kept
Level1, intra=16, inter=1 5.5 s 433 ms (+15%) slower and bit-different vectors
Level3, intra=32, inter=1 4.5 s 411 ms (+10%) full-logical loses to phys/2 (SMT contention)
encode_one vs 1× encode_batch 389 vs 375 ms one boundary crossing saves ~3%; sequential stays
Tokenizer-only cold open 0.5 s 9× cheaper than session open; budgeted reads stay cold

Two consequences:

  • Do not mix tuning in one index. Level1 vs Level3 fuse the graph differently, so bits differ (hashes diverged at 1e-8 formatting). Same model + same build + same tuning, or re-embed.
  • Sequential batching stays. Padded batching would waste attention on variable-length docs to save ~14 ms of boundary overhead — not worth the numerics risk.

Methodology: scratch script + temporary env-knob patch (both reverted; only this table committed). Re-measure on new hardware/ORT before changing the policy.

Pitfall: stale onnxruntime.dll on Windows

This dev machine carries C:\Windows\System32\onnxruntime.dll (v1.17.1). With ORT_DYLIB_PATH unset, ort loads it and dies with BadVersion { version_str: "1.17.1" }, followed by an abort at shutdown (STATUS_STACK_BUFFER_OVERRUN from ort's exit handler — fallout, not the root cause). resolve_ort_dylib() exists precisely so normal entry points never hit this; bare JinaV5.open without it does. Same pitfall as bobine (see its docs/benchmarks.md).

Failure policy

Level Behaviour
Install The wheel is a core dependency of OKFgraph; if it is missing or fails to import, the router raises a clear RuntimeError with the install hint — never an ImportError from deep inside, never a silent fallback.
Device Accelerators are opportunistic: auto/cuda use CUDA when the loaded ORT registers the EP, else warn (stderr) + CPU. used_cuda reports the outcome. Never fatal. Unknown provider names warn and are skipped; registration failure degrades to CPU.
Encode Fail fast. No fallback at encode time — vectors must stay bit-comparable within one index.
Tokenizer No transformers in the runtime path, anywhere: internal tokenize + count_tokens() (== tokenizer.encode(t, add_special_tokens=False)) feed the context-window guard.

Contract notes

  • Session IO is discovered at load (input_ids + attention_mask required, token_type_ids fed only if declared — v5's export doesn't declare it, which is where generic runners fail). Output prefers last_hidden_state.
  • truncate_dim validated like the router (32–1024, warning off the Matryoshka ladder). MAX_LENGTH (8192) is exposed for the window guard.
  • Batch encoding is sequential by design (padded batches waste attention compute on variable-length docs). GIL is released during encode.
  • input_ids/attention_mask feed as int64; pooling takes the last attended token (mask_sum - 1, clamped ≥ 0).

Testing

  • Rust unit tests (18, pure — no network, no dylib, no tokenizer file): device parsing, model-id parsing, provider-matrix mapping, task-prefix idempotence, the L2 → truncate → re-normalise math, contract constants, and open() validation firing before I/O.

    cd rust/okf-embed && cargo test --locked
    

    Runs in CI (Ubuntu, --locked) alongside OKFgraph's pytest jobs.

  • Python parity (tests/test_parity.py, marked slow): Rust output vs a numpy/transformers replication across dims × tasks × texts, ≤ 1e-5. Needs the omni extra (transformers rides in via sentence-transformers).

  • Python e2e (tests/test_rust_backend.py, tests/test_rust_e2e.py): wheel import, count_tokens contract, encode against the real model.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

okf_embed-0.2.0-cp313-cp313-win_amd64.whl (5.8 MB view details)

Uploaded CPython 3.13Windows x86-64

okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl (7.2 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.38+ x86-64

okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl (6.3 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

okf_embed-0.2.0-cp312-cp312-win_amd64.whl (5.8 MB view details)

Uploaded CPython 3.12Windows x86-64

okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl (7.2 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.38+ x86-64

okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl (6.3 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

okf_embed-0.2.0-cp311-cp311-win_amd64.whl (5.8 MB view details)

Uploaded CPython 3.11Windows x86-64

okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl (7.2 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.38+ x86-64

okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl (6.3 MB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

File details

Details for the file okf_embed-0.2.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: okf_embed-0.2.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 5.8 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for okf_embed-0.2.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 d07a20fbb23d92358abf1d36e7ba4ef1b11062293fa0e350eb6fecfa5172c02a
MD5 8b8c79a26a71b3426016edfce898e4fe
BLAKE2b-256 b036c22616502204ef3bbec5968a6d79f4cb3da9f71e7442cb41c75bdfd3c462

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp313-cp313-win_amd64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl.

File metadata

File hashes

Hashes for okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl
Algorithm Hash digest
SHA256 1d489b520be6b2093fe6b2ec4ea47d2507a5633e49919030395471436a606bec
MD5 a8539b579d0afa523e523a04fbb696d8
BLAKE2b-256 b4202a97e0ea899d47aa0809fd44a82ad3727efb735798f6541e5b65bcea8129

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 af440c548f045fd47f11d4e68a69722d8585e1baa0a53d8d0cea3450e0e96e02
MD5 4d38cdc7d8ba474c4f80df556aed2ad2
BLAKE2b-256 3c791e010b189c52f959d4f69ec77b3877bad17ae99c0f20d5389c72294e8f68

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: okf_embed-0.2.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 5.8 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for okf_embed-0.2.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 bfabb48170ceb7705663e81d04c1c98ab239a76b4bbb2df53834ffc59a10b658
MD5 1a1e55d57bce3058e9d083cf45ae120f
BLAKE2b-256 ee188b228227f0621d1105d897a7ff38b0f001d5ceb53aff11326beccaca48fd

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp312-cp312-win_amd64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl.

File metadata

File hashes

Hashes for okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl
Algorithm Hash digest
SHA256 54f7a8a5855047a1d4f000ae29a8a11adffc91c15ea0a415e87086757fe98ee8
MD5 92768f0284718cc6c9daf757493298a8
BLAKE2b-256 1eef67fc1e4e03b993c2c3f378c593c4accc65d1fe611895c36eb42de3fb35ae

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 6f6d1776bb14c16e24b627e06f3fa68319c3306ce0182bb5e27e1f4cd838f358
MD5 dc003278a4b804980a31e460b0c8ae09
BLAKE2b-256 ec188de6648509b2a45bcf6b71bdc1ea96395de9ea4e33ffc37a76d927adbc46

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: okf_embed-0.2.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 5.8 MB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for okf_embed-0.2.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 fa9cde1b64adf8b93721089a72153f28530fe5dda8c1259404a288fecd564fe2
MD5 afc6b5aedf81c6ccc0a2ba97ce427d34
BLAKE2b-256 ff2b4ad82be1e6f789b16ee0cbf60d3e82632516464cd3b5e77868bbd18c14fc

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp311-cp311-win_amd64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl.

File metadata

File hashes

Hashes for okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl
Algorithm Hash digest
SHA256 2a9b89ccd6fd1819c45d474aad141322cc7046ae630210b3446ef00409da81cb
MD5 ff6bc018f4085c8d855ac61d5f43a47d
BLAKE2b-256 42849400971f374b7aa6c5339c63d115990e1e8d95ebf8e405e171c120edef4c

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 73f6c05d2865a462bbff2a3ed94abf24e2581ea9d11d9c45b44f942f5eed8db9
MD5 fdf62546ecdbac3723f52d93d4ee0d89
BLAKE2b-256 db77c7a107080cee0d8631691254e1a3f91a02d76316acd7a216df853e5cb98f

See more details on using hashes here.

Provenance

The following attestation bundles were made for okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: release.yml on opticsWolf/OKFgraph

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.2.0 This release

9 files

0.1.1

9 files

0.1.0

9 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page