okf-embed — Jina v5 text embeddings (Rust core, PyO3)
Exact port of EmbeddingEngine._encode: task prefix → tokenize (8192) →
ONNX forward → last-token pooling → L2 → Matryoshka truncate → re-normalise.
Pinned against a numpy/transformers replication of that pipeline by
tests/test_parity.py (max abs diff ≤ 1e-5, cosine ≥ 0.999999).
The only embedding backend. There is no Python fallback stack, no
embedding_backend selector, and no optimum/transformers in the runtime
path — a mid-run stack switch would silently mix vector spaces in one
index, so the design is fail-fast instead.
Install
Published to PyPI — okfgraph pulls it in automatically (platform wheels
for Linux / Windows / macOS-arm64, Python 3.11–3.13):
uv sync # editable path source, builds via maturin
From source (needs a Rust toolchain 1.85+ and maturin; maturin develop
needs pip, which uv venvs lack — build the wheel and install it instead):
cd rust/okf-embed
maturin build --release
uv pip install --python <venv> target/wheels/okf_embed-*.whl --reinstall
Runtime: ONNX Runtime discovery
ort loads dynamically (load-dynamic, same pin as bobine: 2.0.0-rc.13).
Resolution order: ORT_DYLIB_PATH first (user override always wins), else the
pip-installed onnxruntime/onnxruntime-gpu build when unset: Windows uses
capi/onnxruntime.dll, macOS uses capi/libonnxruntime.dylib, and Linux
prefers versioned capi/libonnxruntime.so.* with capi/libonnxruntime.so as
fallback. GPU DLL warming and Windows DLL-directory setup are best-effort and
never fatal. OKFgraph's resolve_ort_dylib() runs before the native module is
imported and exposes the choice as OKFRouter.ort_dylib, so both bobine and okf-embed
share one ORT binary — no version/CUDA drift between ingest and import.
Lifecycle: lazy session, cheap tokenizer
JinaV5.open (model download + ONNX session build) is the single expensive
step. OKFRouter therefore holds a lazy proxy: construction validates the
wheel import and device string eagerly, but the session opens on the first
real encode — PPR search, budgeted reads, diff, and doctor stay cold.
JinaTokenizer.open fetches only tokenizer.json for exact token counts
without the session. The truncation policy is shared, so counts are identical
to the session path (verified). A failed session open is cached and re-raised
— configuration errors fail fast once, not once per encode.
Explicit local files (air-gapped)
JinaV5.open_files(onnx_path, tokenizer_path) and
JinaTokenizer.open_files(tokenizer_path) skip every download. The
sidecar (model.onnx_data-style) must sit next to the ONNX file — ORT
resolves it relative to the model path, same as the HF cache layout.
OKFRouter(model_path=..., tokenizer_path=...) uses them (both or neither;
missing files raise FileNotFoundError at construction). Same bytes in →
same vectors out (test-pinned against HF acquisition).
Session/threading policy (measured, Phase 6)
Tuning is Level3, intra = physical-cores/2, inter = 1 — kept because it
measured fastest, not because it was inherited. Reference box: Windows,
32 logical cores, CPU-only ORT 1.29, warm model cache, best-of-5 reps on
4 fixed docs (short → ~400 tokens):
| Config | Session cold open | encode_batch (4 docs) |
Notes |
|---|---|---|---|
| Level3, intra=16, inter=1 (current) | 4.7 s | 375 ms | kept |
| Level1, intra=16, inter=1 | 5.5 s | 433 ms (+15%) | slower and bit-different vectors |
| Level3, intra=32, inter=1 | 4.5 s | 411 ms (+10%) | full-logical loses to phys/2 (SMT contention) |
4× encode_one vs 1× encode_batch |
— | 389 vs 375 ms | one boundary crossing saves ~3%; sequential stays |
| Tokenizer-only cold open | 0.5 s | — | 9× cheaper than session open; budgeted reads stay cold |
Two consequences:
- Do not mix tuning in one index. Level1 vs Level3 fuse the graph differently, so bits differ (hashes diverged at 1e-8 formatting). Same model + same build + same tuning, or re-embed.
- Sequential batching stays. Padded batching would waste attention on variable-length docs to save ~14 ms of boundary overhead — not worth the numerics risk.
Methodology: scratch script + temporary env-knob patch (both reverted; only this table committed). Re-measure on new hardware/ORT before changing the policy.
Pitfall: stale onnxruntime.dll on Windows
This dev machine carries C:\Windows\System32\onnxruntime.dll (v1.17.1).
With ORT_DYLIB_PATH unset, ort loads it and dies with
BadVersion { version_str: "1.17.1" }, followed by an abort at shutdown
(STATUS_STACK_BUFFER_OVERRUN from ort's exit handler — fallout, not the
root cause). resolve_ort_dylib() exists precisely so normal entry points
never hit this; bare JinaV5.open without it does. Same pitfall as bobine
(see its docs/benchmarks.md).
Failure policy
| Level | Behaviour |
|---|---|
| Install | The wheel is a core dependency of OKFgraph; if it is missing or fails to import, the router raises a clear RuntimeError with the install hint — never an ImportError from deep inside, never a silent fallback. |
| Device | Accelerators are opportunistic: auto/cuda use CUDA when the loaded ORT registers the EP, else warn (stderr) + CPU. used_cuda reports the outcome. Never fatal. Unknown provider names warn and are skipped; registration failure degrades to CPU. |
| Encode | Fail fast. No fallback at encode time — vectors must stay bit-comparable within one index. |
| Tokenizer | No transformers in the runtime path, anywhere: internal tokenize + count_tokens() (== tokenizer.encode(t, add_special_tokens=False)) feed the context-window guard. |
Contract notes
- Session IO is discovered at load (
input_ids+attention_maskrequired,token_type_idsfed only if declared — v5's export doesn't declare it, which is where generic runners fail). Output preferslast_hidden_state. truncate_dimvalidated like the router (32–1024, warning off the Matryoshka ladder).MAX_LENGTH(8192) is exposed for the window guard.- Batch encoding is sequential by design (padded batches waste attention compute on variable-length docs). GIL is released during encode.
input_ids/attention_maskfeed as int64; pooling takes the last attended token (mask_sum - 1, clamped ≥ 0).
Testing
-
Rust unit tests (18, pure — no network, no dylib, no tokenizer file): device parsing, model-id parsing, provider-matrix mapping, task-prefix idempotence, the L2 → truncate → re-normalise math, contract constants, and
open()validation firing before I/O.cd rust/okf-embed && cargo test --locked
Runs in CI (Ubuntu,
--locked) alongside OKFgraph's pytest jobs. -
Python parity (
tests/test_parity.py, markedslow): Rust output vs a numpy/transformers replication across dims × tasks × texts, ≤ 1e-5. Needs theomniextra (transformers rides in via sentence-transformers). -
Python e2e (
tests/test_rust_backend.py,tests/test_rust_e2e.py): wheel import, count_tokens contract, encode against the real model.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file okf_embed-0.2.0-cp313-cp313-win_amd64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp313-cp313-win_amd64.whl
- Upload date:
- Size: 5.8 MB
- Tags: CPython 3.13, Windows x86-64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d07a20fbb23d92358abf1d36e7ba4ef1b11062293fa0e350eb6fecfa5172c02a
|
|
| MD5 |
8b8c79a26a71b3426016edfce898e4fe
|
|
| BLAKE2b-256 |
b036c22616502204ef3bbec5968a6d79f4cb3da9f71e7442cb41c75bdfd3c462
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp313-cp313-win_amd64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp313-cp313-win_amd64.whl -
Subject digest:
d07a20fbb23d92358abf1d36e7ba4ef1b11062293fa0e350eb6fecfa5172c02a - Sigstore transparency entry: 2816091283
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl
- Upload date:
- Size: 7.2 MB
- Tags: CPython 3.13, manylinux: glibc 2.38+ x86-64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1d489b520be6b2093fe6b2ec4ea47d2507a5633e49919030395471436a606bec
|
|
| MD5 |
a8539b579d0afa523e523a04fbb696d8
|
|
| BLAKE2b-256 |
b4202a97e0ea899d47aa0809fd44a82ad3727efb735798f6541e5b65bcea8129
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp313-cp313-manylinux_2_38_x86_64.whl -
Subject digest:
1d489b520be6b2093fe6b2ec4ea47d2507a5633e49919030395471436a606bec - Sigstore transparency entry: 2816091185
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl
- Upload date:
- Size: 6.3 MB
- Tags: CPython 3.13, macOS 11.0+ ARM64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
af440c548f045fd47f11d4e68a69722d8585e1baa0a53d8d0cea3450e0e96e02
|
|
| MD5 |
4d38cdc7d8ba474c4f80df556aed2ad2
|
|
| BLAKE2b-256 |
3c791e010b189c52f959d4f69ec77b3877bad17ae99c0f20d5389c72294e8f68
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp313-cp313-macosx_11_0_arm64.whl -
Subject digest:
af440c548f045fd47f11d4e68a69722d8585e1baa0a53d8d0cea3450e0e96e02 - Sigstore transparency entry: 2816090737
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp312-cp312-win_amd64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp312-cp312-win_amd64.whl
- Upload date:
- Size: 5.8 MB
- Tags: CPython 3.12, Windows x86-64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
bfabb48170ceb7705663e81d04c1c98ab239a76b4bbb2df53834ffc59a10b658
|
|
| MD5 |
1a1e55d57bce3058e9d083cf45ae120f
|
|
| BLAKE2b-256 |
ee188b228227f0621d1105d897a7ff38b0f001d5ceb53aff11326beccaca48fd
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp312-cp312-win_amd64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp312-cp312-win_amd64.whl -
Subject digest:
bfabb48170ceb7705663e81d04c1c98ab239a76b4bbb2df53834ffc59a10b658 - Sigstore transparency entry: 2816091008
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl
- Upload date:
- Size: 7.2 MB
- Tags: CPython 3.12, manylinux: glibc 2.38+ x86-64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
54f7a8a5855047a1d4f000ae29a8a11adffc91c15ea0a415e87086757fe98ee8
|
|
| MD5 |
92768f0284718cc6c9daf757493298a8
|
|
| BLAKE2b-256 |
1eef67fc1e4e03b993c2c3f378c593c4accc65d1fe611895c36eb42de3fb35ae
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp312-cp312-manylinux_2_38_x86_64.whl -
Subject digest:
54f7a8a5855047a1d4f000ae29a8a11adffc91c15ea0a415e87086757fe98ee8 - Sigstore transparency entry: 2816091432
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl
- Upload date:
- Size: 6.3 MB
- Tags: CPython 3.12, macOS 11.0+ ARM64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6f6d1776bb14c16e24b627e06f3fa68319c3306ce0182bb5e27e1f4cd838f358
|
|
| MD5 |
dc003278a4b804980a31e460b0c8ae09
|
|
| BLAKE2b-256 |
ec188de6648509b2a45bcf6b71bdc1ea96395de9ea4e33ffc37a76d927adbc46
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp312-cp312-macosx_11_0_arm64.whl -
Subject digest:
6f6d1776bb14c16e24b627e06f3fa68319c3306ce0182bb5e27e1f4cd838f358 - Sigstore transparency entry: 2816090606
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp311-cp311-win_amd64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp311-cp311-win_amd64.whl
- Upload date:
- Size: 5.8 MB
- Tags: CPython 3.11, Windows x86-64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
fa9cde1b64adf8b93721089a72153f28530fe5dda8c1259404a288fecd564fe2
|
|
| MD5 |
afc6b5aedf81c6ccc0a2ba97ce427d34
|
|
| BLAKE2b-256 |
ff2b4ad82be1e6f789b16ee0cbf60d3e82632516464cd3b5e77868bbd18c14fc
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp311-cp311-win_amd64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp311-cp311-win_amd64.whl -
Subject digest:
fa9cde1b64adf8b93721089a72153f28530fe5dda8c1259404a288fecd564fe2 - Sigstore transparency entry: 2816091079
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl
- Upload date:
- Size: 7.2 MB
- Tags: CPython 3.11, manylinux: glibc 2.38+ x86-64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2a9b89ccd6fd1819c45d474aad141322cc7046ae630210b3446ef00409da81cb
|
|
| MD5 |
ff6bc018f4085c8d855ac61d5f43a47d
|
|
| BLAKE2b-256 |
42849400971f374b7aa6c5339c63d115990e1e8d95ebf8e405e171c120edef4c
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp311-cp311-manylinux_2_38_x86_64.whl -
Subject digest:
2a9b89ccd6fd1819c45d474aad141322cc7046ae630210b3446ef00409da81cb - Sigstore transparency entry: 2816090853
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type:
File details
Details for the file okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl.
File metadata
- Download URL: okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl
- Upload date:
- Size: 6.3 MB
- Tags: CPython 3.11, macOS 11.0+ ARM64
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
73f6c05d2865a462bbff2a3ed94abf24e2581ea9d11d9c45b44f942f5eed8db9
|
|
| MD5 |
fdf62546ecdbac3723f52d93d4ee0d89
|
|
| BLAKE2b-256 |
db77c7a107080cee0d8631691254e1a3f91a02d76316acd7a216df853e5cb98f
|
Provenance
The following attestation bundles were made for okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl:
Publisher:
release.yml on opticsWolf/OKFgraph
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
okf_embed-0.2.0-cp311-cp311-macosx_11_0_arm64.whl -
Subject digest:
73f6c05d2865a462bbff2a3ed94abf24e2581ea9d11d9c45b44f942f5eed8db9 - Sigstore transparency entry: 2816090933
- Sigstore integration time:
-
Permalink:
opticsWolf/OKFgraph@fff1413439520c15ab609652e1e88135b4cb017c -
Branch / Tag:
refs/tags/v0.2.11 - Owner: https://github.com/opticsWolf
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@fff1413439520c15ab609652e1e88135b4cb017c -
Trigger Event:
push
-
Statement type: