cryptotensors-koalavault-vllm

KoalaVault Key Provider for CryptoTensors - A secure key management provider for encrypted model deployment with vLLM.

Note: This package is distributed as pre-compiled binaries. The Rust source code is proprietary and not publicly available.

Overview

cryptotensors-koalavault-vllm is a Key Provider implementation for the CryptoTensors library, specifically designed for secure model deployment with vLLM. It provides:

• Secure Key Management: Automatic key retrieval from KoalaVault servers
• Attestation Support: Framework integrity verification for secure deployments
• vLLM Integration: Seamless integration with vLLM for encrypted model serving
• Native Performance: Rust-based implementation with Python bindings
• Binary Distribution: Pre-compiled native libraries for easy installation

Installation

From PyPI

pip install cryptotensors-koalavault-vllm

The package includes pre-compiled binaries for common platforms. No compilation or Rust toolchain is required.

Quick Start

Basic Usage

import cryptotensors

# Initialize the KoalaVault provider
cryptotensors.init_key_provider(
    "koalavault-vllm",
    api_key="your-api-key",
    model_owner="model-owner",
    model_name="model-name",
    model_path="/path/to/model"
)

# Now you can use cryptotensors to load encrypted models
from cryptotensors import safe_open

with safe_open("model.safetensors", framework="pt") as f:
    tensor = f.get_tensor("layer.weight")

With vLLM

The provider is typically initialized automatically when using the KoalaVault vLLM container:

docker run --rm \
  -e KOALAVAULT_API_KEY=your-api-key \
  -v /path/to/models:/models \
  koalavault/vllm:latest \
  --koalavault-api-key your-api-key \
  --koalavault-model owner/model-name \
  --model /models/model-name

Architecture

This package implements the KeyProvider trait from the CryptoTensors core library. When initialized, it:

1. Authenticates with the KoalaVault server using the provided API key
2. Performs Attestation to verify the framework integrity
3. Retrieves Keys for decrypting the encrypted model tensors
4. Manages Keys securely in memory without exposing them to Python

API Reference

Python API

cryptotensors.init_key_provider(name, **config)

Initialize and activate a key provider.

Parameters:

• name (str): Provider name, must be "koalavault-vllm"
• api_key (str): KoalaVault API key
• model_owner (str): Model owner identifier
• model_name (str): Model name identifier
• model_path (str): Path to the model directory

Example:

cryptotensors.init_key_provider(
    "koalavault-vllm",
    api_key="sk-...",
    model_owner="my-org",
    model_name="my-model",
    model_path="/models/my-model"
)

Rust API

The Rust crate provides the KoalaVaultProvider struct that implements the KeyProvider trait. The crate is distributed as a binary library (cdylib) and can be used via FFI:

use cryptotensors::{KeyProvider, CryptoTensorsError};

// Provider is created via FFI function exported from the binary
// extern "C" fn cryptotensors_create_provider() -> *mut dyn KeyProvider

Note: The Rust source code for this provider is proprietary. Only pre-compiled binary libraries are distributed via crates.io.

Security Features

• Signature Verification: All provider libraries are cryptographically signed and verified before loading
• Key Isolation: Cryptographic keys never pass through the Python interpreter
• Attestation: Framework integrity is verified before key retrieval
• Secure Storage: Keys are stored in memory with automatic cleanup

Requirements

• Python >= 3.9
• cryptotensors core library (>= 0.2.0)

Distribution

This package is distributed as pre-compiled binary wheels for:

• Linux (x86_64, aarch64)
• macOS (x86_64, arm64)
• Windows (x86_64)

The Rust source code is proprietary and not available for public inspection or modification. Only the compiled binary libraries are distributed via PyPI and crates.io.

License

This package is distributed under a proprietary license. The binary libraries are provided for use with the CryptoTensors ecosystem. See the package metadata for specific license terms.

Links

• CryptoTensors Core Library (Open Source)
• KoalaVault Documentation
• PyPI Package
• Crates.io Package

Support

For issues and questions, please contact KoalaVault support or refer to the official documentation.