Victor Rust Workspace

Rust libraries, the standalone edge runtime, and high-performance Python extensions for Victor.

Packages

victor-edge depends on the other three Rust crates, so crates.io publication is intentionally ordered as protocol → state → tools → edge.

Features

• Deduplication (dedup): Rolling hash-based content deduplication using xxHash3
• Similarity (similarity): SIMD-optimized cosine similarity for embeddings
• JSON Repair (json_repair): Fast JSON repair with streaming parser
• Hashing (hashing): High-performance signature hashing for loop detection

Performance

Building

Prerequisites

• Rust 1.70+ (install via rustup)
• Python 3.12+
• maturin (pip install maturin)

Development Build

cd rust

# Build and install in development mode
maturin develop

# Build with optimizations
maturin develop --release

# Build wheel for distribution
maturin build --release

Install Pre-built Wheel

# From the rust directory after building
pip install target/wheels/victor_native-*.whl

Usage

The native extensions are automatically used when available:

from victor.processing.native import (
    rolling_hash_blocks,
    batch_cosine_similarity,
    repair_json,
    compute_signature,
    is_native_available,
)

# Check if native extensions are loaded
print(f"Native available: {is_native_available()}")

# Deduplication
blocks = rolling_hash_blocks(content, min_block_length=50)
for hash_str, block, is_duplicate in blocks:
    if is_duplicate:
        print(f"Duplicate block: {block[:50]}...")

# Similarity
query = [0.1, 0.2, ...]  # 384-dim embedding
corpus = [[...], [...], ...]  # List of embeddings
similarities = batch_cosine_similarity(query, corpus)

# JSON repair
fixed = repair_json("{'key': 'value', 'active': True}")
# '{"key": "value", "active": true}'

# Signature hashing
sig = compute_signature("read_file", {"path": "/test.py"})

Fallback Behavior

If the native extension is not available (not installed or incompatible platform), the victor.processing.native module automatically falls back to pure Python implementations with equivalent functionality.

from victor.processing.native import is_native_available

if is_native_available():
    print("Using Rust implementation (fast)")
else:
    print("Using Python fallback (compatible)")

Architecture

rust/
├── Cargo.toml          # Workspace configuration and shared version
├── pyproject.toml      # maturin build configuration
├── README.md           # This file
└── crates/
    ├── protocol/
    ├── state/
    ├── tools/
    ├── edge-runtime/
    └── python-bindings/

Releasing crates.io packages

Rust workspace releases use their own rust-vX.Y.Z tags. Before tagging, update the workspace version and every internal dependency version in the crate manifests, then run:

cd rust
cargo fmt --all --check
cargo test --workspace --locked
cargo package -p victor-protocol --locked
cargo package -p victor-state --locked
cargo package -p victor-tools --locked
cargo package -p victor-edge --locked

All four package checks resolve the internal Victor dependencies through crates.io, providing the same consumer-side validation used by the release workflow.

Push an annotated rust-vX.Y.Z tag only after the release commit reaches main. The Publish Rust crates workflow validates that the tag and workspace versions match, then publishes the dependency chain. It requires the Actions secret CARGO_REGISTRY_TOKEN to be available to the repository. Re-running the workflow is safe: crate versions already present on crates.io are skipped and verified. After publication, the workflow installs victor-edge back from crates.io and runs its version command.

Testing

# Run Rust tests
cargo test

# Run Python integration tests
pytest ../tests/unit/test_native.py -v

License

Apache License 2.0