Rivusio

A blazing-fast, type-safe data processing pipeline framework in Python, designed for seamless integration of both synchronous and asynchronous operations, complete with robust parallel execution for ultimate performance and scalability.

🚀 Key Features

• Type Safety: Comprehensive type hints and runtime validation
• Flexible Processing: Support for both sync and async operations
• High Performance: Parallel execution strategies and optimized processing
• Stream Processing: Built-in support for various windowing strategies
• Robust Error Handling: Configurable retries and error recovery
• Monitoring: Built-in metrics collection and monitoring
• Configuration: Type-safe configuration using Pydantic
• Extensible: Plugin system for custom components
• Composable: Easy pipeline composition with the >> operator
• Comprehensive Documentation: Extensive guides and examples

🛠️ Installation

pip install rivusio

🎯 Quick Start

from typing import Dict, List
from rivusio import AsyncBasePipe, SyncBasePipe, AsyncPipeline

# Define a simple transformation pipe
class NumberFilterPipe(AsyncBasePipe[Dict[str, int], List[int]]):
    async def __call__(self, data: Dict[str, int]) -> List[int]:
        return [v for v in data.values() if v > 100]

# Create and use a pipeline
async def main():
    pipeline = AsyncPipeline([NumberFilterPipe()])
    
    # Process data
    data = {"a": 150, "b": 50, "c": 200}
    result = await pipeline(data)  # [150, 200]
    print(result)

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

📚 Documentation

Visit our comprehensive documentation for:

• Detailed guides and tutorials
• API reference
• Best practices
• Advanced examples
• Configuration options

🔧 Development Setup

# Clone the repository
git clone https://github.com/zbytealchemy/rivusio.git
cd rivusio

# Install dependencies
poetry install

# Run tests
poetry run pytest

# Build documentation
poetry install --with docs
make docs

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details on:

• Code of Conduct
• Development process
• Pull request guidelines
• Testing requirements

📋 Requirements

• Python 3.10+
• Poetry for dependency management
• Git for version control

📈 Performance

Rivusio is designed for high performance:

• Efficient batch processing
• Parallel execution capabilities
• Optimized stream processing
• Minimal overhead

🔒 Security

• Regular security audits
• Type-safe operations
• Input validation
• Comprehensive testing

📄 License

MIT License - see LICENSE file for details.

🙏 Acknowledgments

Special thanks to all contributors who have helped make Rivusio better.

Built with ❤️ by ZbyteAlchemy