kyrex 0.2.0

The operating system for Python dependencies and environments.

Kyrex

The operating system for Python dependencies and environments.

---

Table of Contents

• Requirements
• Installation
• Quick Example
• Why Kyrex?
• Features
• Architecture
• Commands
• Performance
• Comparison
• Security
• FAQ
• Project Structure
• Roadmap
• Project Status
• Documentation Links
• Contributing
• License

---

Requirements

Building or installing Kyrex from source requires the following system dependencies:

Supported Operating Systems

• Windows: Windows 10, Windows 11, or Windows Server (x86_64).
• Linux: Ubuntu, Debian, Red Hat, Fedora, Arch Linux, etc. (x86_64, aarch64).
• macOS: Apple Silicon (M1/M2/M3) and Intel-based architectures.

System Dependencies

• Python: Version 3.8 or newer.
• Git: Installed and registered in your system PATH.
• Rust Toolchain: Stable compiler (rustc) and package manager (cargo).

Installing Rust

Linux & macOS

Install using the official rustup script:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Windows

Install via winget package manager:

winget install Rustlang.Rustup

Alternatively, download and run the installer directly from rustup.rs.

Ensure the installation was successful and available in your terminal:

rustc --version
cargo --version

---

Installation

1. Install from PyPI

To install the pre-built distribution of Kyrex directly:

pip install kyrex

2. Install from Source

To compile and install Kyrex locally:

git clone https://github.com/aaryanrwt/kyrex.git
cd kyrex
pip install maturin
maturin build --release
pip install target/wheels/kyrex-*.whl

3. Development Installation

To set up an editable local workspace for code changes:

git clone https://github.com/aaryanrwt/kyrex.git
cd kyrex
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\Activate.ps1
pip install maturin click rich packaging tomli pytest
maturin develop

---

Quick Example

# Resolve, download, and cache dependencies globally
kyrex install numpy pandas torch

# Share your local cache over the LAN
kyrex share --port 8990

# Compress the active virtual environment into a portable bundle
kyrex bundle --output dev_env.kyx

# Recreate the virtual environment on another machine from the bundle
kyrex restore dev_env.kyx --path .restored_venv

---

Why Kyrex?

Modern Python environment setups repeatedly download identical artifacts, consuming bandwidth and delaying execution.

• Redundant Downloads: Multi-machine deployments and team workspaces frequently fetch the exact same wheel files from PyPI.
• Large Dependency Bottlenecks: Heavy data science and ML runtimes (such as PyTorch and CUDA binaries) consume gigabytes of storage and take minutes to download.
• Flaky Replication: Moving exact environment configurations to offline or air-gapped staging servers without heavy containers is complex.
• Hardware Integration: Configuring and mapping Python wheels to host GPU configurations and active CUDA toolkits requires manual matching of links and tags.

Kyrex acts as an environment fabric layer on top of standard package managers:

• Content-Addressable Storage (CAS): Wheels are indexed and cached globally by their cryptographic SHA-256 hash. Environments link directly to these cache copies.
• P2P LAN Distribution: Uses mDNS to discover peers on the local network. Packages are served and pulled over local network connections, bypassing PyPI.
• Self-Contained Archives: .kyx bundles archive the environment state, dependency structure, and exact wheels into a compressed file for offline replication.
• Zero-Setup CUDA Remapping: Scans system drivers and paths to override package requests automatically with GPU-optimized wheel binaries.

---

Features

Category	Capability	Description
Cache	Content-Addressable Storage	Caches wheel binaries globally using SHA-256 hashes to prevent duplicate storage.
Networking	mDNS P2P Sharing	Automatically advertises and discovers local peers on the network to pull cached wheels over the LAN.
Environment	Teleportation (.kyx)	Bundles environment manifests and wheels into a compressed archive for instant offline restoration.
Performance	Hard-Link Installation	Links wheels directly from the local CAS to virtual environments, avoiding duplicate copies and file writes.
Security	Subprocess Safety	Standardizes list-based subprocess calls and verifies hashes at system boundaries to prevent directory escapes.
Diagnostics	Environment Doctor	Audits cache databases, validates file integrity, and identifies virtual environment drift.
Machine Learning	CUDA Auto-Mapping	Detects GPU details on the host and automatically overrides installation indexes to fetch compatible CUDA wheels.

---

Architecture

graph TD
    PyPI[PyPI Registry] -->|Download| Resolver[uv Resolver]
    Resolver -->|Cache Write| CAS[Local CAS Cache]
    Resolver -->|Query/Fetch| LAN[LAN Peers]
    CAS -->|Hard-Link| Venv[Virtual Environment]
    LAN -->|Download/Cache| CAS
    Venv -->|Bundle| Kyx[Bundle .kyx]
    Kyx -->|Restore| Destination[Restore Anywhere]

---

Commands

Command	Usage	Description
kyrex install	kyrex install requests	Resolves, downloads, caches, and installs packages into .venv.
kyrex add	kyrex add numpy	Alias for kyrex install.
kyrex bundle	kyrex bundle -o env.kyx	Packages the active environment and cached wheels into a .kyx archive.
kyrex restore	kyrex restore env.kyx	Unpacks wheels and recreates the virtual environment.
kyrex share	kyrex share --port 8990	Starts the P2P wheel distribution server on the local network.
kyrex clone	kyrex clone <repo_url>	Clones a repository and automatically bootstraps its dependencies.
kyrex bootstrap	kyrex bootstrap	Initializes .venv and installs packages detected in the project path.
kyrex doctor	kyrex doctor	Validates CAS registry file integrity and checks for environment drift.

---

Performance

Note: Benchmarks will be populated under identical CPU and network configurations in future releases.

Operation	pip	uv	Kyrex (Cold)	Kyrex (Warm Cache)	Kyrex (LAN Peer)
Install requests	TBD	TBD	TBD	TBD	TBD
Install torch (2.5GB)	TBD	TBD	TBD	TBD	TBD
Restore Environment	N/A	N/A	TBD	TBD	TBD

---

Comparison

Feature	pip	uv	Poetry	PDM	Conda	Kyrex
Fast Installs		✓				✓
Offline Installs		✓				✓
Local Cache		✓			✓	✓
LAN P2P Sharing						✓
Bundle Support						✓
GPU-Aware Installs						✓
Environment Restoration						✓

---

Security

Kyrex implements the following measures:

• SHA-256 Validation: Strictly validates that all package hashes match exactly ^[a-fA-F0-9]{64}$ before accessing or writing files in the CAS.
• Unsafe Path Protection: Prevents path traversal and directory escape attempts during bundle extraction or file imports.
• Subprocess Isolation: Runs all external commands (like git or uv) as structured lists without shell invocation (shell=False) to prevent command injection.
• SQLite Database Parameters: Executes database queries using strictly parameterized arguments to prevent SQL injection.
• Hash Checks: Verifies target file integrity on install to detect corrupted or tampered local cache files.

---

FAQ

Why Rust?

Rust provides the thread safety and memory model required to compute cryptographic hashes and write Content-Addressable Storage (CAS) files concurrently, alongside high-performance tar/zstd archiving capabilities for portable .kyx bundles.

Does Kyrex replace pip or uv?

No. Kyrex leverages uv under the hood as its primary resolution engine. It sits on top of these tools to manage caching layers, P2P network sharing, bundle packaging, and hardware auto-mapping.

Does it work offline?

Yes. Once packages are cached in the global CAS, kyrex install --offline installs dependencies and recreates environments completely without internet connectivity.

Does it support Windows?

Yes. Kyrex is fully compatible with Windows 10/11 and Windows Server platforms, utilizing standard registry and system calls to identify GPU configurations.

Is the bundle portable?

The .kyx bundle is portable across machines running the same host operating system and architecture. It bundles the package wheels and environment metadata. Bundles created on Linux should not be restored directly on Windows.

Can I use it in CI?

Yes. Teleporting environments via .kyx bundles allows pipelines to unpack and restore dependency environments in seconds, bypassing resolution and download phases.

---

Project Structure

kyrex/
├── Cargo.toml                  # Rust compilation configuration.
├── pyproject.toml              # Project dependency and packaging configuration.
├── src/                        # Rust source code.
│   ├── lib.rs                  # PyO3 Python extension bindings.
│   ├── cas.rs                  # Content-Addressable Storage file engine.
│   ├── db.rs                   # SQLite cache registry interface.
│   └── bundle.rs               # Environment bundling (.kyx) archive logic.
├── kyrex/                      # Python source code.
│   ├── __init__.py             # Initialization.
│   ├── core.py                 # Core paths and Python-to-Rust wrappers.
│   ├── hardware.py             # System spec and CUDA configuration mapping.
│   ├── p2p.py                  # P2P Server and mDNS local sharing.
│   ├── resolver.py             # uv resolution pipeline integration.
│   └── cli.py                  # CLI command parser.
├── tests/                      # Unit and integration test suite.
├── images/                     # Project graphic assets.
└── docs/                       # Project documentation assets.

---

Roadmap

Planned features for future releases:

• Incremental Bundle Updates: Only transfer differences and new packages when updating .kyx bundles.
• Cross-Platform Bundle Validation: Automatically detect and prompt users if attempting to restore a Linux bundle on macOS or Windows.
• CI Cache Integrations: Provide configurations and caching scripts for GitHub Actions, GitLab CI, and Azure Pipelines.
• Cryptographic Bundle Signing: Introduce digital signatures for .kyx packages to verify source authenticity.
• Plugin API: Support custom command-line extensions.

---

Project Status

Kyrex is currently in active development. The public API may evolve before v1.0. Feedback and contributions are welcome.

---

Documentation Links

For deeper technical information, consult the reference guides:

• Architecture Details
• mDNS Networking & P2P Protocols
• Bundle Package Format (.kyx)
• Command Reference
• Security Audits & Threat Model

---

Contributing

Guidelines on how to set up development instances, compile local modifications, and submit pull requests are detailed in [CONTRIBUTING.md](file:///c:/Users/Aaryan Rawat/Documents/kyrex/CONTRIBUTING.md).

---

License

MIT