Victor Native Extensions

High-performance Rust implementations of CPU-intensive operations for the Victor agentic AI framework.

Features

  • Deduplication (dedup): Rolling hash-based content deduplication using xxHash3
  • Similarity (similarity): SIMD-optimized cosine similarity for embeddings
  • JSON Repair (json_repair): Fast JSON repair with streaming parser
  • Hashing (hashing): High-performance signature hashing for loop detection

Performance

Operation Python Rust Speedup
Block deduplication 1.0ms 0.02ms ~50x
Cosine similarity (384d, 100 vectors) 5.0ms 1.0ms ~5x
JSON repair 0.5ms 0.05ms ~10x
Signature hashing 0.2ms 0.02ms ~10x

Building

Prerequisites

  • Rust 1.70+ (install via rustup)
  • Python 3.10+
  • maturin (pip install maturin)

Development Build

cd rust

# Build and install in development mode
maturin develop

# Build with optimizations
maturin develop --release

# Build wheel for distribution
maturin build --release

Install Pre-built Wheel

# From the rust directory after building
pip install target/wheels/victor_native-*.whl

Usage

The native extensions are automatically used when available:

from victor.processing.native import (
    rolling_hash_blocks,
    batch_cosine_similarity,
    repair_json,
    compute_signature,
    is_native_available,
)

# Check if native extensions are loaded
print(f"Native available: {is_native_available()}")

# Deduplication
blocks = rolling_hash_blocks(content, min_block_length=50)
for hash_str, block, is_duplicate in blocks:
    if is_duplicate:
        print(f"Duplicate block: {block[:50]}...")

# Similarity
query = [0.1, 0.2, ...]  # 384-dim embedding
corpus = [[...], [...], ...]  # List of embeddings
similarities = batch_cosine_similarity(query, corpus)

# JSON repair
fixed = repair_json("{'key': 'value', 'active': True}")
# '{"key": "value", "active": true}'

# Signature hashing
sig = compute_signature("read_file", {"path": "/test.py"})

Fallback Behavior

If the native extension is not available (not installed or incompatible platform), the victor.processing.native module automatically falls back to pure Python implementations with equivalent functionality.

from victor.processing.native import is_native_available

if is_native_available():
    print("Using Rust implementation (fast)")
else:
    print("Using Python fallback (compatible)")

Architecture

rust/
├── Cargo.toml          # Rust package configuration
├── pyproject.toml      # maturin build configuration
├── README.md           # This file
└── src/
    ├── lib.rs          # PyO3 module definition
    ├── dedup.rs        # Deduplication module
    ├── similarity.rs   # Cosine similarity module
    ├── json_repair.rs  # JSON repair module
    └── hashing.rs      # Signature hashing module

Testing

# Run Rust tests
cargo test

# Run Python integration tests
pytest ../tests/unit/test_native.py -v

License

Apache License 2.0

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for vjsingh from gravatar.com vjsingh

Credits

Author: Vijaykumar Singh

License

Apache Software License (Apache-2.0)

Requires

Python >=3.10

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

victor_native-0.1.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

victor_native-0.1.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.3 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ ARM64

victor_native-0.1.0-cp312-cp312-win_amd64.whl (1.1 MB view details)

Uploaded CPython 3.12Windows x86-64

victor_native-0.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

victor_native-0.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.3 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ ARM64

victor_native-0.1.0-cp312-cp312-macosx_11_0_arm64.whl (1.2 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

victor_native-0.1.0-cp312-cp312-macosx_10_12_x86_64.whl (1.3 MB view details)

Uploaded CPython 3.12macOS 10.12+ x86-64

victor_native-0.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

victor_native-0.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.3 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ ARM64

File details

Details for the file victor_native-0.1.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1f7f78151bfe7019645bed0b2f689d515f13b8360605e86a26843ce5dd5234cb
MD5 3cc11ac2e00c64b51b189f3badfaed0b
BLAKE2b-256 dd1e68cc7395f0e215ff03bd5cffa054b1c92a3e15057944b51b6205171141aa

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ab357459695aa688af5267eeec1ebb1a9b712898656587fbbec042b996b76146
MD5 17aeb240aa10e33e5f0be2ccd84b879e
BLAKE2b-256 d7f1a440ffe3a7883d48c3c3b08b42eafd32e13d4ecc09397e0d7396c4416577

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 4ae2fcc4cbd4956bddd4359ec734706f877876c14c4050730166bf07e4b550d2
MD5 40e1d1e77f37714bd50e4a451b8c87b1
BLAKE2b-256 dc8cdf2debe2354668dd6225e644c4421f368e6d59a71d2c85dcf2e416219612

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp312-cp312-win_amd64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bcf3e4ed5125e47248075de2e1e49317a17c3f54bbe22de50beb16a1a3dfbd7c
MD5 d127ca0c6abc92bb98cd581ab50d2078
BLAKE2b-256 c406301a9013414d9228adc240d1917214740d300fe667644ebafbd25eb09386

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 8a2f0eb716ac566884591bcf56e06b7b2dfe24bd94bca575f9c8eaca5a4c218d
MD5 540735fba6eb2b0862050772252f361c
BLAKE2b-256 a6d6c97269b15020c89a5285c8dd863deb563d01b24c8c72da27032b7aadf258

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 17a09dd1f6dec89f919f682c8261987b73c71b0d6fdfae4e573a4ff98690d8af
MD5 cda393053d99e70d910ea1af04731632
BLAKE2b-256 81b10c6d4a95e8c28a6c07597d237b43602d4d311c43e4b6b6e8207010194c09

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 e8cc6baeebcb4b1b24dc0290c8b3da8b1d467cda2a9860b3b902d0020be56705
MD5 06cecfe919485342989dbc427ae1676e
BLAKE2b-256 c4e2c54ccaef8fdfb96ed38ee44b15155941f3a31c62c044a09a9d16e3ef3620

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp312-cp312-macosx_10_12_x86_64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7fcf220b87062a0fc68d340fb1484462210b3705ee4d7c3479b4798fd8f67977
MD5 dd4e317d23d2395e5613ae6e00d99efb
BLAKE2b-256 904724879262e5586cd6ea703382e049e674cc51f14115f4d129837a2b89d6aa

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file victor_native-0.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for victor_native-0.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 4b2b7d7f1ae39ad2cfa82f5c8a45a58b6b9d56ed4bd5638c42ba030d36e497ef
MD5 e661a83a45028726732b3d42dd3b044d
BLAKE2b-256 5163a5685bb271cb1a2a921cbd145b057b10bceb2e684aa3a594537d34c25600

See more details on using hashes here.

Provenance

The following attestation bundles were made for victor_native-0.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on anvai-labs/victor

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page