polyglot-bridge 0.1.2

High-performance ML data pipeline accelerator - Rust-powered computational functions with seamless Python integration

🌉 The Polyglot Bridge

High-Performance ML Data Pipeline Accelerator

Stop waiting for your data preprocessing. The Polyglot Bridge delivers Rust's blazing performance with Python's simplicity—no Rust knowledge required.

---

🚀 Why The Polyglot Bridge?

If you're an AI/ML engineer tired of slow Python preprocessing bottlenecks, this library is for you.

The Problem

Pure Python is slow for computationally intensive operations. Your ML training pipeline spends more time preprocessing data than actually training models.

The Solution

The Polyglot Bridge provides Rust-powered computational functions that seamlessly integrate into your Python workflow:

• ✅ Drop-in replacement for slow Python operations
• ✅ Zero Rust knowledge required - just pip install and go
• ✅ Automatic parallelization - leverages all CPU cores without threading code
• ✅ Type-safe - full type hints and IDE autocomplete support

---

⚡ Performance: The Numbers Don't Lie

Real-world benchmarks comparing pure Python vs The Polyglot Bridge:

Operation	Dataset Size	Python	Rust	Speedup
Matrix Multiply	50×50	36.1 ms	0.38 ms	95x faster 🔥
Matrix Multiply	100×100	245.9 ms	2.8 ms	88x faster 🔥
Sum of Squares	10,000	1.3 ms	0.4 ms	3.3x faster ⚡
Sum of Squares	100,000	23.3 ms	9.3 ms	2.5x faster ⚡

Average Speedup: 22.5x | Maximum Speedup: 95x

Benchmarks run on Windows with Python 3.14 and Rust 1.93. Your results may vary based on hardware.

---

📦 Installation

pip install polyglot-bridge

Requirements:

• Python 3.8 or later
• No Rust installation needed!

---

🎯 Quickstart

Basic Usage

import polyglot_bridge

# Sum of squares - 3x faster than pure Python
numbers = [1.0, 2.0, 3.0, 4.0, 5.0]
result = polyglot_bridge.sum_of_squares(numbers)
print(result)  # 55.0

# Matrix multiplication - up to 95x faster!
a = [[1.0, 2.0], [3.0, 4.0]]
b = [[5.0, 6.0], [7.0, 8.0]]
result = polyglot_bridge.matrix_multiply(a, b)
print(result)  # [[19.0, 22.0], [43.0, 50.0]]

# Parallel transformation - automatic multi-core processing
data = list(range(100000))
result = polyglot_bridge.parallel_transform(data, 2.5)
# Transforms 100k elements using all CPU cores

Real-World ML Pipeline Example

import polyglot_bridge
import numpy as np

# Feature preprocessing for ML pipeline
def preprocess_features(raw_features, transformation_matrix):
    """
    Accelerated feature transformation using Rust
    
    Before: 245ms with pure Python (100×100 matrix)
    After: 2.8ms with Polyglot Bridge (88x faster!)
    """
    # Convert numpy to list (zero-copy in practice)
    features_list = raw_features.tolist()
    transform_list = transformation_matrix.tolist()
    
    # Lightning-fast matrix multiplication
    transformed = polyglot_bridge.matrix_multiply(features_list, transform_list)
    
    return np.array(transformed)

# Your training loop now spends time training, not preprocessing!

---

🎨 API Reference

sum_of_squares(numbers: List[float]) -> float

Compute the sum of squares for a list of numbers.

Performance: ~3x faster than pure Python

Example:

result = polyglot_bridge.sum_of_squares([1.0, 2.0, 3.0])
# Returns: 14.0

Raises:

• ValueError: If input list is empty
• RuntimeError: If computation overflows

---

matrix_multiply(a: List[List[float]], b: List[List[float]]) -> List[List[float]]

Multiply two matrices using optimized Rust implementation.

Performance: Up to 95x faster than pure Python for 50×50 matrices

Example:

a = [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]  # 2×3 matrix
b = [[7.0, 8.0], [9.0, 10.0], [11.0, 12.0]]  # 3×2 matrix
result = polyglot_bridge.matrix_multiply(a, b)
# Returns: [[58.0, 64.0], [139.0, 154.0]]  # 2×2 matrix

Raises:

• ValueError: If matrices are empty or dimensions don't match

---

parallel_transform(data: List[float], factor: float) -> List[float]

Transform data in parallel by multiplying each element by a factor.

Performance: Automatic parallelization across all CPU cores

Example:

data = [1.0, 2.0, 3.0, 4.0, 5.0]
result = polyglot_bridge.parallel_transform(data, 2.0)
# Returns: [2.0, 4.0, 6.0, 8.0, 10.0]

Raises:

• ValueError: If input list is empty

---

🏗️ Architecture

The Polyglot Bridge uses:

• Rust Edition 2024 for cutting-edge language features
• PyO3 for seamless Python-Rust interoperability
• Rayon for automatic parallelization
• Iterator-based algorithms for optimal performance

┌─────────────────────────────────────┐
│      Python Application             │
└──────────────┬──────────────────────┘
               │ Zero-cost FFI
               ▼
┌─────────────────────────────────────┐
│      PyO3 Bindings Layer            │
│  • Type conversion                  │
│  • Error translation                │
└──────────────┬──────────────────────┘
               │
               ▼
┌─────────────────────────────────────┐
│      Rust Core Library              │
│  • Optimized algorithms             │
│  • Parallel processing (Rayon)      │
│  • Zero-copy operations             │
└─────────────────────────────────────┘

---

🧪 Testing

The library includes comprehensive test coverage:

• 30 Python integration tests (100% pass rate)
• 12 Rust unit tests with property-based testing
• Hypothesis-powered property tests for correctness
• Algorithmic equivalence verified across implementations

Run tests:

# Python tests
pytest tests/

# Rust tests
cargo test

# Benchmarks
python python/benchmarks.py

---

🛠️ Development

Building from Source

# Clone the repository
git clone https://github.com/nirvagold/polyglot-bridge.git
cd polyglot-bridge

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install maturin
pip install maturin

# Build and install in development mode
maturin develop --release

# Run tests
pytest tests/
cargo test

Project Structure

polyglot-bridge/
├── src/
│   ├── lib.rs              # PyO3 bindings
│   ├── core/
│   │   ├── math.rs         # Mathematical operations
│   │   └── parallel.rs     # Parallel processing
│   └── error.rs            # Error types
├── tests/
│   ├── test_python.py      # Python integration tests
│   └── test_benchmarks.py  # Algorithmic equivalence tests
├── benches/
│   └── criterion.rs        # Rust benchmarks
├── python/
│   └── benchmarks.py       # Python vs Rust benchmarks
└── polyglot_bridge.pyi     # Type stubs for IDE support

---

📊 Benchmarks Deep Dive

Methodology

All benchmarks compare fair, optimized implementations:

• Pure Python uses list comprehensions and generator expressions
• Rust uses iterator chains and Rayon parallelization
• Multiple iterations with outlier removal for accuracy

When to Use The Polyglot Bridge

Best for:

• ✅ Matrix operations (50-100x speedup)
• ✅ Large-scale numerical computations
• ✅ ML feature preprocessing pipelines
• ✅ Batch data transformations

Not ideal for:

• ❌ Very small datasets (< 1000 elements) - FFI overhead dominates
• ❌ Operations that are already vectorized with NumPy
• ❌ I/O-bound operations

---

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---

🙏 Acknowledgments

• Built with PyO3 for Python-Rust interoperability
• Powered by Rayon for parallelization
• Inspired by the need for faster ML data pipelines

---

📬 Contact

Have questions or suggestions? Open an issue on GitHub!

---

Stop waiting. Start accelerating. 🚀

⭐ Star this repo if The Polyglot Bridge saves you time!