kyrex 1.0.0

The operating system for Python dependencies and environments.

Kyrex

Kyrex is a Python environment infrastructure layer for caching, portability, offline restoration, and reproducible environments.

---

Kyrex is a decentralized environment fabric designed to streamline dependency packaging and transfer. By caching wheels in a Content-Addressable Storage (CAS) database, sharing packages across local networks (LAN), and bundling complete environments, Kyrex enables developers to replicate configurations instantly and offline.

Table of Contents

• Requirements
• Installation
• Quick Start
• Why Kyrex Exists
• Design Principles
• Who is Kyrex For?
• Architecture
• Core Features
• CLI Reference
• Comparison with Alternative Tools
• Security Model & Threat Model
• Testing
• Validation Status
• Compatibility Matrix
• Performance Philosophy
• FAQ
• Known Limitations
• Development Principles
• Release Roadmap
• Long-Term Vision
• Release Validation
• Contributing
• License

---

Requirements

Target Operating Systems

Kyrex v1.0 targets support for:

• Windows: Windows 10, Windows 11, or Windows Server (x86_64).
• Linux: Ubuntu, Debian, Red Hat, Fedora, Arch Linux, etc. (x86_64, aarch64).
• macOS: Apple Silicon (M1/M2/M3) and Intel-based architectures.

Note: Official support is granted only after completing automated CI validation and manual smoke testing for that platform.

Prerequisites

Before installing Kyrex, ensure the following tools and system configurations are available:

Required Runtimes

• Python: Python 3.10–3.13 (3.14 will be supported once upstream dependencies officially support it.) Check via: python --version
• Git: Installed and registered in your system PATH. Check via: git --version
• Rust Toolchain: Stable compiler (rustc) and package manager (cargo). Check via: rustc --version and cargo --version

Platform Build Tools (Required only when compiling from source)

• Windows: Install Visual Studio Build Tools, MSVC C++ Compiler, and Windows SDK.
• Linux: Install build-essential, gcc, clang, and pkg-config.
• macOS: Install Xcode Command Line Tools via:

  xcode-select --install
  

Optional Helper Packages

• uv: Check via uv --version.
• maturin: Check via maturin --version (required for local extension compilation).

Network & Port Access

• TCP Port 8990: Required only for local LAN sharing server (kyrex share). Must be allowed in firewall.
• mDNS Multicast: Multicast traffic must be enabled on the local network for P2P peer discovery.

Storage Specifications

• Hardware: SSD (Solid State Drive) is highly recommended for optimal hard-link restoration speeds.
• Free Disk Space:
  • Minimum: 2 GB
  • Recommended: 10 GB+ (depending on the size and count of cached machine learning wheel binaries).
• Internet Connection: Required only for initial installation, dependency resolution, or downloading uncached packages. All subsequent environment builds can run completely offline.

---

Installation

1. Install from PyPI

To install the pre-built distribution of Kyrex directly:

pip install kyrex

2. Install from Source

To compile and install Kyrex locally:

git clone https://github.com/aaryanrwt/kyrex.git
cd kyrex
pip install maturin
maturin build --release
pip install target/wheels/kyrex-*.whl

3. Development Installation

To set up an editable local workspace for code changes:

git clone https://github.com/aaryanrwt/kyrex.git
cd kyrex
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\Activate.ps1
pip install maturin click rich packaging tomli pytest zeroconf uv
maturin develop

---

Quick Start

# Resolve, download, and cache dependencies globally
kyrex install numpy pandas torch

# Share your local cache over the LAN
kyrex share --port 8990

# Compress the active virtual environment into a portable bundle
kyrex bundle --output dev_env.kyx

# Recreate the virtual environment on another machine from the bundle
kyrex restore dev_env.kyx --path .restored_venv

---

Why Kyrex Exists

Modern Python development involves repeated downloads of identical package binaries across different local virtual environments, developer machines, and CI/CD pipelines. This redundancy wastes internet bandwidth, slows down team onboarding, and creates fragility when deploying to staging or air-gapped systems. Kyrex treats Python virtual environments not as disposable, arbitrary directory trees, but as verifiable, relocatable, and shareable software layers.

---

Design Principles

Kyrex follows five core principles:

• Reliability before features: Prioritize absolute stability, deterministic behaviors, and rigorous pre-release validation.
• Reproducible environments: Ensure virtual environments can be reconstituted offline identically across workstations.
• Offline-first workflows: Minimize external network dependency by caching wheels in a global content-addressed local store.
• Security by default: Cryptographic verification on CAS file writes and zero-trust sandbox execution limits.
• Performance through Rust: Focus core filesystem, database queries, and bundler compression loops inside performance-optimized Rust.

---

Who is Kyrex For?

• Data Scientists & ML Engineers: Working with large binary dependencies (like PyTorch or CUDA wheels) who want to avoid reinstall delays and automatic hardware mapping.
• Research Labs & Universities: Setting up compute clusters where gigabytes of packages need to be distributed across many machines without saturated gateways.
• CI/CD Platforms: Replicating staging environments rapidly using local offline bundles rather than rebuilding environments from scratch.
• Offline & Air-gapped Developers: Working without internet connectivity (defense, banking, healthcare) who need a portable, self-contained way to package dependencies.

Who should NOT use Kyrex?

• Users who need a lightweight pip replacement for tiny projects with minimal, pure-Python dependencies (where standard pip or uv is already sufficient).
• Single-use, disposable scripts that do not require environment persistence or portability.

---

Architecture

graph TD
    PyPI[PyPI Registry] -->|Download| Resolver[uv Resolver]
    Resolver -->|Cache Write| CAS[Local CAS Cache]
    Resolver -->|Query/Fetch| LAN[LAN Peers]
    CAS -->|Hard-Link| Venv[Virtual Environment]
    LAN -->|Download/Cache| CAS
    Venv -->|Bundle| Kyx[Bundle .kyx]
    Kyx -->|Restore| Destination[Restore Anywhere]

For a detailed analysis of components (Rust, SQLite, CAS, .kyx), execution flows, and design choices, see docs/architecture.md.

---

Core Features

Category	Capability	Description
Cache	Content-Addressable Storage	Caches wheel binaries globally using SHA-256 hashes to prevent duplicate storage.
Networking	mDNS P2P Sharing	Automatically advertises and discovers local peers on the network to pull cached wheels over the LAN.
Environment	Teleportation (.kyx)	Bundles environment manifests and wheels into a compressed archive for instant offline restoration.
Performance	Hard-Link Installation	Links wheels directly from the local CAS to virtual environments, avoiding duplicate copies and file writes.
Security	Subprocess Safety	Standardizes list-based subprocess calls and verifies hashes at system boundaries to prevent directory escapes.
Diagnostics	Environment Doctor	Audits cache databases, validates file integrity, and identifies virtual environment drift.
Machine Learning	CUDA Auto-Mapping	Detects GPU details on the host and automatically overrides installation indexes to fetch compatible CUDA wheels.

---

CLI Reference

kyrex install

Resolves and installs packages into a target virtual environment.

kyrex install [PACKAGES]... [OPTIONS]

• --offline: Disables PyPI lookups, installing only from local CAS wheels.
• --path, -p TEXT: Path to the virtual environment (default: .venv).

kyrex bundle

Archives the current virtual environment and all its cached wheels into a single .kyx bundle.

kyrex bundle --output TEXT [OPTIONS]

• --path, -p TEXT: Path to the virtual environment directory to bundle.

kyrex restore

Restores a virtual environment from a .kyx bundle.

kyrex restore [BUNDLE_FILE] [OPTIONS]

• --path, -p TEXT: Path to recreate the virtual environment.

kyrex share

Starts the P2P wheel distribution server on the local network.

kyrex share [OPTIONS]

• --port, -p INTEGER: Port to bind the HTTP sharing server.

kyrex clone

Clones a git repository and automatically bootstraps its dependencies.

kyrex clone [REPO_URL] [DEST_DIR]

kyrex bootstrap

Initializes a virtual environment and installs dependencies detected in the current path (pyproject.toml or requirements.txt).

kyrex bootstrap

kyrex doctor

Diagnoses and repairs local cache DBs, verifying file hashes and identifying environment drift.

kyrex doctor

CLI Terminal Output Demonstrations

Below are example outputs showing how Kyrex presents diagnostics and peer sharing status in the terminal:

1. Checking Cache Health (kyrex doctor --fix)

$ kyrex doctor --fix

 ╔══════════════════════════════════════════════════════════════════════════╗
 ║                           Kyrex Diagnostics                              ║
 ╚══════════════════════════════════════════════════════════════════════════╝

Running Kyrex Diagnostics...

-> Checking SQLite catalog path: C:\Users\Aaryan Rawat\.cache\kyrex\metadata.db
-> Checking Content-Addressable Storage (CAS): C:\Users\Aaryan Rawat\.cache\kyrex\cas

! Warning: Found 2 orphaned wheel files in CAS not registered in SQLite.
v Doctor repair option (--fix) active.
-> Re-indexing orphaned wheels into SQLite database catalog...
v Registered wheel: click-8.4.2-py3-none-any.whl (128 KB)
v Registered wheel: rich-15.0.0-py3-none-any.whl (320 KB)

v All integrity checks passed. Database and cache are healthy.

2. Starting P2P Sharing (kyrex share)

$ kyrex share --port 8990

 ╔══════════════════════════════════════════════════════════════════════════╗
 ║                         Kyrex Peer-to-Peer Fabric                        ║
 ╚══════════════════════════════════════════════════════════════════════════╝

-> Advertising local package sharing service on port 8990...
-> mDNS Service Broadcast: _kyrex._tcp.local. (host: Aaryan-PC)
-> HTTP wheel server listening on http://127.0.0.1:8990

⚡ Peer fabric is active. Press Ctrl+C to stop sharing.

Comparison with Alternative Tools

Kyrex does not replace dependency resolvers; it adds a portability and caching layer on top of them. Below is a factual comparison of features and capabilities:

Feature	pip	uv	Poetry	PDM	Conda	Kyrex
Core Role	Package Installer	Package Resolver	Project Manager	Project Manager	System Environment	Environment Fabric
Deduplication Caching	❌ (HTTP cache)	✅ (Content-Address)	❌ (HTTP cache)	❌ (HTTP cache)	✅ (Hard links)	✅ (CAS Hard-link)
Offline Restore	❌ (Manual flags)	✅ (Local cache)	❌ (Manual flags)	❌ (Manual flags)	✅ (Local channels)	✅ (Zero-setup local)
LAN Peer Sharing	❌	❌	❌	❌	❌	✅ (mDNS Discovery)
Relocatable Bundles	❌	❌	❌	❌	⚠ (Conda-pack)	✅ (.kyx Archives)
GPU Auto-Mapping	❌	❌	❌	❌	❌	✅ (CUDA detection)
Reproducible lockfile	❌	✅	✅	✅	❌	✅ (.kyx.lock)

---

Security Model & Threat Model

Kyrex is engineered with a multi-layered security model to protect system hosts and developer environments.

System Protections

• SHA-256 CAS Verification: All wheels are indexed and verified by their cryptographic SHA-256 hash to prevent package spoofing.
• Strict Input Sanitization: Python and Rust API boundaries reject any hash parameters that do not strictly match the 64-character hexadecimal format (^[a-fA-F0-9]{64}$).
• Path Traversal Prevention: Bundle extraction and file read/write routines validate destination paths to block directory escape and tar-slip vulnerability exploits.
• Subprocess Isolation: External helper programs (such as git or uv) are executed as structured lists with shell=False. This eliminates shell injection vectors.
• Parameterized SQL Queries: All metadata transactions in the SQLite catalog use parameterized queries, neutralizing SQL injection attempts.

Threat Model

Kyrex protects against:

• Path traversal and directory escapes during bundle extraction or CAS file registration.
• Command injection through structured subprocess lists running with shell=False.
• Corrupted environment bundles or tampered files through cryptographic verification.
• Local cache database corruption or tampered package hashes.

Kyrex does not currently protect against:

• Malicious packages intentionally published to and downloaded from upstream PyPI registries.
• Supply-chain attacks originating from compromised source code repositories before they are built into wheels.
• Compromised host platforms where the root system has already been broken.

---

Testing

Kyrex contains an extensive test suite verifying the correctness of core operations across Rust and Python layers.

Unit Tests

• CAS engine: Verifies file hashing and directory tree layout.
• SQLite metadata: Asserts table initialization and environment registrations.
• Bundle creation: Packs environment manifest and wheels into compressed archives.
• Bundle restoration: Recreates environments from archived .kyx packages.
• Hash validation: Confirms boundary checks and format constraints are enforced.

Integration Tests

• CLI installation: Tests click routing and subprocess environment linking.
• Bundle restore: Reconstitutes complete environments on clean simulated systems.
• Cache rebuild: Restores missing database paths from orphaned local wheels.
• Doctor repair: Runs automatic doctor fixes using --fix.

Security Tests

• Path traversal: Asserts traversal attempts are detected and blocked.
• Invalid hashes: Validates that non-hexadecimal or incorrect length inputs raise value errors.
• Corrupted bundles: Verifies that tampered archives trigger verification failure.
• Injection attempts: Confirms that subprocess arguments ignore shell meta-character injections.

Execute the test suite locally:

python -m pytest -v

---

Validation Status

Latest release v1.0.0 has been successfully verified under the following conditions:

• Fresh Windows installation validation.
• Python 3.13.7 virtual environment bootstrapping.
• Maturin compilation and PyPI-compatible wheel creation.
• CLI subcommands registration (install, bundle, restore, share, doctor, clone, bootstrap).
• Cryptographic verification on CAS writes and SQLite indexing.
• Portable .kyx bundle creation and restoration.
• Command-line version check and diagnostic checks.

---

Compatibility Matrix

Kyrex v1.0 categorizes platform and interpreter compatibility as follows:

1. Officially Verified

Verified via direct local validation and testing on host hardware:

• Windows 11 (x86_64)
• Python 3.13

2. Experimental

Built, compiled, and tested via the GitHub Actions CI matrix:

• Windows 10
• Ubuntu Linux
• Debian Linux
• Fedora Linux
• Arch Linux
• macOS Intel
• macOS Apple Silicon
• Python 3.10–3.12

3. Planned for Upcoming Releases

• Additional Linux distributions validation.
• Extended Python interpreter compatibility (such as Python 3.14).
• Target platform ARM optimizations.
• Large-scale enterprise validation.

---

Performance Philosophy

Kyrex is designed around three goals:

• Minimize repeated package downloads over the internet.
• Maximize local cache reuse across workspaces.
• Reduce setup time for Python virtual environments.

Performance benchmarks will be published after standardized testing on identical hardware and network configurations. We do not publish synthetic benchmarks.

---

FAQ

Why Rust?

Rust provides the memory safety, concurrent execution models, and compiler optimizations needed to compute cryptographic hashes and write Content-Addressable Storage (CAS) files concurrently, alongside high-performance tar/zstd archiving capabilities for portable .kyx bundles.

Does Kyrex replace pip or uv?

No. Kyrex leverages uv under the hood as its primary resolution engine. It sits on top of these tools to manage caching layers, P2P network sharing, bundle packaging, and hardware auto-mapping.

Does it work offline?

Yes. Once packages are cached in the global CAS, kyrex install --offline installs dependencies and recreates environments completely without internet connectivity.

Does it support Windows?

Yes. Kyrex is fully compatible with Windows 10/11 and Windows Server platforms, utilizing standard registry and system calls to identify GPU configurations.

Is the bundle portable?

The .kyx bundle is portable across machines running the same host operating system and architecture. It bundles the package wheels and environment metadata. Bundles created on Linux should not be restored directly on Windows.

Can I use it in CI?

Yes. Teleporting environments via .kyx bundles allows pipelines to unpack and restore dependency environments in seconds, bypassing resolution and download phases.

Where are packages stored globally?

Packages are stored in a central global cache directory at ~/.cache/kyrex/cas/ grouped by the first two characters of their SHA-256 hash.

How does P2P discovery work?

Kyrex uses mDNS (via the zeroconf protocol) to broadcast a local service named _kyrex._tcp.local.. Discovered peers are queried over HTTP for their package manifests.

Does P2P download verify wheel integrity?

Yes. When Kyrex downloads a wheel from a LAN peer, it calculates its SHA-256 hash and compares it against the index-resolved hash to prevent tampering.

Can I use Kyrex with requirements.txt?

Yes. Kyrex's bootstrap command parses requirements.txt or pyproject.toml configurations to resolve and install dependencies.

What happens if the peer disconnects during P2P download?

Kyrex automatically falls back to fetching the wheel from PyPI if a peer download fails or times out.

Does Kyrex compile source packages?

No. Kyrex works with pre-built wheel distributions (.whl). If a package only has source distributions (.tar.gz), it is compiled and resolved into a wheel by the underlying uv engine before caching.

How does CUDA auto-mapping work?

Kyrex checks system drivers, paths, and registry keys to identify the system's CUDA version. If installing torch, it overrides the target index url to the PyTorch wheels index matching the CUDA version.

Can I clean the cache?

Yes. You can use the kyrex doctor command to verify cache integrity and remove corrupted packages.

How is metadata managed?

All global metadata, registered environments, and package-to-hash mappings are stored in a SQLite database file located at ~/.cache/kyrex/metadata.db.

---

Known Limitations

• Windows wheels are currently published directly.
• Linux/macOS wheels are generated through CI.
• Bundle portability depends on platform-compatible binary wheels.
• Large environments may require significant disk space.
• Peer discovery currently operates within the same local network.

---

Development Principles

Every feature added to Kyrex must satisfy:

• Performance: High speed through Rust-compiled IO and SQLite indexing.
• Reliability: Deterministic environment recreation from portable archives.
• Reproducibility: Verifiable builds with lock-free file sharing.
• Security: Cryptographic verification and secure subprocess execution.
• Minimal Configuration: Out-of-the-box setups without manual configurations.

---

Release Roadmap

For a detailed technical roadmap and market gap analysis, see docs/ROADMAP.md.

Note: These are planned areas of exploration and may evolve based on community feedback.

v1.0 (Stable Release)

• Content-Addressable Storage (CAS) cache engine.
• Portable .kyx environment bundling.
• Declarative .kyx.lock lockfile formats.
• ThreadPool parallel downloads & local P2P LAN sharing.
• Diagnostics exporter (doctor --report) and automatic recovery (doctor --fix).

---

v1.1 (Planned)

• Full cross-platform validation (Windows 10, Ubuntu, Debian, Fedora, Arch, macOS Intel & Apple Silicon).
• Complete Python 3.10–3.12 validation.
• Multi-platform bundle validations.
• Custom plugin system templates.
• Shell autocompletion profiles.

---

v1.2 (Planned)

• Distributed cache mirroring.
• Authenticated peer sharing.

---

Long-Term Vision

Kyrex aims to become a foundational infrastructure layer for Python dependency management and environment portability. Rather than being another package installer, Kyrex provides a unified layer for caching, environment portability, offline development, LAN package distribution, reproducible builds, and hardware-aware dependency management. The goal is to make Python environments fast, reliable, and reproducible across laptops, workstations, CI pipelines, and enterprise infrastructure.

---

Release Validation

Kyrex undergoes validation before every public release.

See the RELEASE_VALIDATION_REPORT.md and FINAL_RELEASE_REPORT.md files for test logs, compatibility matrices, and validation summaries.

---

Contributing

Contributions of all sizes are welcome.

Whether you're fixing bugs, improving documentation, proposing new features, or optimizing performance, we'd love your help.

See CONTRIBUTING.md for setup instructions, coding guidelines, testing requirements, and the pull request workflow.

---

Acknowledgements

Kyrex builds upon the excellent work of the Python and Rust ecosystems. Special thanks to the communities behind:

• Python
• Rust
• PyO3
• Maturin
• uv
• SQLite
• PyPI

Without these projects, Kyrex would not be possible.

---

License

Kyrex is licensed under the MIT License. See the LICENSE file for complete details.

---

Build once. Cache forever. Restore anywhere.