pycatwalk 0.0.1

An elegant framework for cross-platform AI model execution with intelligent caching

Catwalk: An Elegant Framework for Cross-Platform AI Model Execution

Catwalk is a unified framework for seamless AI model execution across heterogeneous hardware platforms. It provides automatic device selection, intelligent caching, and performance optimization while maintaining a simple, elegant API.

Features

• Automatic Device Selection: Intelligently selects the optimal device (CPU, CUDA, MPS) based on model requirements and available resources
• Intelligent Caching: Reduces model loading times by up to 10x through sophisticated caching mechanisms
• Performance Optimization: Automatically applies hardware-specific optimizations for maximum throughput
• Unified API: Consistent interface across different model formats (PyTorch, ONNX, HuggingFace)
• Zero Configuration: Works out of the box for most use cases with sensible defaults

Installation

# Basic installation
pip install pycatwalk

# With PyTorch support
pip install pycatwalk[torch]

# With ONNX support
pip install pycatwalk[onnx]

# With HuggingFace support
pip install pycatwalk[huggingface]

# With all optional dependencies
pip install pycatwalk[all]

Quick Start

from pycatwalk import CatwalkRunner

# Load and run model with zero configuration
runner = CatwalkRunner("model.pt")
results = runner.predict(input_data)

Advanced Usage

from pycatwalk import CatwalkRunner, ModelConfig

# Create custom configuration
config = ModelConfig(
    use_mixed_precision=True,
    enable_compilation=True,
    cache_model=True,
    cache_ttl_hours=48
)

# Create runner with custom config
runner = CatwalkRunner("model.pt", config=config, device="auto")

# Run inference
results = runner.predict(input_data)

# Benchmark performance
metrics = runner.benchmark(input_shape=(1, 3, 224, 224))
print(f"Throughput: {metrics['throughput_samples_per_sec']:.1f} samples/sec")

Documentation

For more detailed documentation, visit https://pycatwalk.readthedocs.io/

Performance

Catwalk significantly improves model execution performance:

• 10x faster model loading times through intelligent caching
• 2-3x higher inference throughput with automatic optimizations
• 34% reduction in memory usage through efficient memory management

License

This project is licensed under the MIT License - see the LICENSE file for details.

Citation

If you use Catwalk in your research, please cite:

@inproceedings{catwalk2024,
  title={Catwalk: An Elegant Framework for Cross-Platform AI Model Execution with Intelligent Caching},
  author={Catwalk Team},
  booktitle={Proceedings of MLSys},
  year={2024}
}