Matrice Common Library

matrice_common is a high-performance Python package providing reusable utilities for Matrice.ai services. It offers production-ready components for authentication, API communication, streaming, and more.

🚀 Quick Start

Installation

pip install --index-url https://test.pypi.org/simple/ matrice_common

Basic Usage

from matrice_common.rpc import RPC
from matrice_common.session import create_session

# Initialize RPC client
rpc = RPC(
    access_key="your_access_key",
    secret_key="your_secret_key"  # pragma: allowlist secret
)

# Make API requests
response = rpc.get("/v1/endpoint")
data = rpc.post("/v1/resource", payload={"key": "value"})

# Create a session
session = create_session(
    access_key="your_access_key",
    secret_key="your_secret_key"  # pragma: allowlist secret
)

# Create a project
project = session.create_classification_project(
    name="My Project",
    description="AI classification project"
)

✨ Key Features

🔐 Authentication & Security

Token-based authentication with automatic refresh
Secure credential management
Environment-based configuration (prod/staging/dev)

🌐 RPC Client

Synchronous and asynchronous HTTP methods
Automatic token management
Built-in error handling and retry logic
Type-safe API interactions

📊 Streaming

Unified Interface: Single API for Kafka and Redis streaming
Async Support: Full async/await compatibility
Metrics & Monitoring: Built-in performance tracking
Auto-Reconnection: Resilient connection handling

🔧 Utilities

Comprehensive error logging (Sentry + Kafka)
Error deduplication
Automatic dependency installation
Caching decorators
Type hints throughout

📦 Session Management

Project lifecycle management

📖 Documentation

Comprehensive documentation is available in DOCUMENTATION.md, including:

Installation Guide - Setup and requirements
Authentication - Token management and security
RPC Client - API communication
Session Management - Project lifecycle
Streaming - Kafka and Redis streaming
Frame Optimization - Intelligent transmission
Error Handling - Logging and monitoring
Utilities - Helper functions
API Reference - Complete API docs
Testing - Test suite information
Examples - Code examples

💡 Examples

Async API Calls

import asyncio
from matrice_common.rpc import RPC

async def fetch_data():
    rpc = RPC(access_key="...", secret_key="...")

    # Concurrent requests
    results = await asyncio.gather(
        rpc.get_async("/v1/users"),
        rpc.get_async("/v1/projects"),
        rpc.get_async("/v1/datasets")
    )

    return results

asyncio.run(fetch_data())

Streaming with Kafka

from matrice_common.stream.matrice_stream import MatriceStream, StreamType

# Create stream
stream = MatriceStream(
    stream_type=StreamType.KAFKA,
    access_key="...",
    secret_key="..."
)

# Setup and use
stream.setup(topic_or_stream_name="my-topic")
stream.add_message({"data": "value", "timestamp": "2025-01-01T12:00:00Z"})

# Receive messages
message = stream.get_message(timeout=30)
if message:
    print(f"Received: {message}")

stream.close()

Error Handling

from matrice_common.utils import log_errors, AppError, ErrorType, get_deduplication_config

# Configure deduplication (or use environment variables)
# export MATRICE_ERROR_DEDUPLICATION_ENABLED=true
# export MATRICE_ERROR_CACHE_TTL_SECONDS=1800  # 30 minutes

# Use service_name to properly track errors per service
@log_errors(service_name="my_service", raise_exception=False)
def process_data(data):
    """Automatically logs errors to Sentry and Kafka with deduplication."""
    if not data:
        raise ValueError("Data cannot be empty")

    # Process data
    return result

# Errors are automatically logged and deduplicated
result = process_data(my_data)

# Check deduplication config
print(get_deduplication_config())
# Output: {'enabled': True, 'ttl_seconds': 900, 'max_cache_size': 1000, 'current_cache_size': 0}

🧪 Testing

The library includes a comprehensive test suite with high coverage.

Running Tests

# Install development dependencies
pip install -r requirements-dev.txt

# Run all tests
pytest

# Run with coverage report
pytest --cov=src/matrice_common --cov-report=html

# Run specific test module
pytest tests/unit/test_rpc.py -v

# Run with verbose output
pytest -v --tb=short

Test Coverage

token_auth.py: 100% coverage
utils.py: 52% coverage
rpc.py: Comprehensive unit tests
Integration tests for all major workflows

View detailed coverage:

pytest --cov --cov-report=html
open htmlcov/index.html  # View in browser

🏗️ Project Structure

py_common/
├── src/matrice_common/      # Source code
│   ├── rpc.py               # RPC client
│   ├── token_auth.py        # Authentication
│   ├── utils.py             # Utilities and error handling
│   ├── session.py           # Session management
│   ├── stream/              # Streaming modules
│   │   ├── matrice_stream.py
│   │   ├── kafka_stream.py
│   │   └── redis_stream.py
│   └── optimize/            # Frame optimization
│       ├── cache_manager.py
│       ├── frame_comparators.py
│       ├── frame_difference.py
│       └── transmission.py
├── tests/                   # Test suite
│   ├── conftest.py         # Test fixtures
│   ├── unit/               # Unit tests
│   └── integration/        # Integration tests
├── DOCUMENTATION.md        # Comprehensive documentation
├── README.md              # This file
├── setup.py               # Package setup
├── pyproject.toml         # Project configuration
├── pytest.ini             # Pytest configuration
└── requirements-dev.txt   # Development dependencies

🔧 Development

Setup Development Environment

# Clone repository
git clone <repository-url>
cd py_common

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install development dependencies
pip install -r requirements-dev.txt

# Install package in editable mode
pip install -e .

Code Quality

# Format code
black src/

# Lint code
flake8 src/

# Type checking
mypy src/

Building the Package

# Build package
python setup.py bdist_wheel sdist

# Build with PyArmor obfuscation (optional)
python setup.py build

# Skip obfuscation
SKIP_PYARMOR_OBFUSCATION=true python setup.py bdist_wheel

🌍 Environment Variables

Core Configuration

Variable	Description	Required	Default
`MATRICE_ACCESS_KEY_ID`	API access key	Yes	-
`MATRICE_SECRET_ACCESS_KEY`	API secret key	Yes	-
`ENV`	Environment (prod/staging/dev)	No	prod
`MATRICE_ACTION_ID`	Current action ID	No	-
`MATRICE_SESSION_ID`	Current session ID	No	-
`SKIP_PYARMOR_OBFUSCATION`	Skip code obfuscation	No	false

Error Logging & Deduplication

Variable	Description	Required	Default
`MATRICE_ERROR_DEDUPLICATION_ENABLED`	Enable error deduplication	No	true
`MATRICE_ERROR_CACHE_TTL_SECONDS`	Deduplication time window (seconds)	No	900 (15 min)
`MATRICE_ERROR_CACHE_MAX_SIZE`	Max unique errors to track	No	1000

Setting Environment Variables

# Linux/Mac
export MATRICE_ACCESS_KEY_ID="your_access_key"
export MATRICE_SECRET_ACCESS_KEY="your_secret_key"  # pragma: allowlist secret
export ENV="staging"

# Windows PowerShell
$env:MATRICE_ACCESS_KEY_ID="your_access_key"
$env:MATRICE_SECRET_ACCESS_KEY="your_secret_key" <!-- pragma: allowlist secret -->
$env:ENV="staging"

# Python
import os
os.environ['MATRICE_ACCESS_KEY_ID'] = 'your_access_key'
os.environ['MATRICE_SECRET_ACCESS_KEY'] = 'your_secret_key' <!-- pragma: allowlist secret -->

📊 Module Overview

Core Modules

Module	Purpose	Key Features
rpc.py	API Communication	Sync/Async HTTP, auto-auth, retry logic
token_auth.py	Authentication	Token management, auto-refresh
utils.py	Utilities	Error logging, caching, helpers
session.py	Session Management	Project lifecycle, CRUD operations

Streaming Modules

Module	Purpose	Backend
matrice_stream.py	Unified Streaming	Kafka/Redis
kafka_stream.py	Kafka Streaming	Apache Kafka
redis_stream.py	Redis Streaming	Redis Streams

Optimization

Module	Purpose	Features
frame_comparators.py	Frame Comparison	SSIM, perceptual hashing
frame_difference.py	Difference Detection	Change detection
transmission.py	Optimized Transfer	Smart transmission

🤝 Contributing

We welcome contributions! Please follow these guidelines:

Fork the repository
Create a feature branch: git checkout -b feature/my-feature
Write tests for new functionality
Ensure tests pass: pytest
Format code: black src/
Submit pull request

Contribution Guidelines

Maintain test coverage above 80%
Follow PEP 8 style guide
Add type hints to all functions
Write clear docstrings (Google style)
Update documentation for new features

📝 License

This project is licensed under the MIT License - see the LICENSE.txt file for details.

🙏 Acknowledgments

Built for Matrice.ai services
Uses industry-standard libraries (requests, aiohttp, Kafka, Redis)
Inspired by modern Python best practices

📞 Support

Documentation: DOCUMENTATION.md
Issues: GitHub Issues
Email: support@matrice.ai

🗺️ Roadmap

Additional streaming backends (RabbitMQ, NATS)
GraphQL support
WebSocket streaming
Real-time metrics dashboard
CLI tool for common operations

Made with ❤️ by the Matrice.ai Team

Last Updated: 2025-01-30 | Version: 0.0.2

Release files for matrice-common 0.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for matrice-common 0.3.1
File	Size	Uploaded
matrice_common-0.3.1.tar.gz	260.4 kB	Jul 29, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for matrice-common 0.3.1
File	Interpreter	ABI	Platform	Reset
matrice_common-0.3.1-py3-none-any.whl 295.9 kB Jul 29, 2026	Python 3	none	any	Details

Total release size: 556.2 kB

Release files / matrice_common-0.3.1.tar.gz

Download URL	matrice_common-0.3.1.tar.gz
Size	260.4 kB
Tags	Source
SHA-256 checksum How to use checksums	`79bbd38d5b2b76bd64c39284c596700ffea33dca8c3872fdf51394a8ad248e05`
BLAKE2b-256 checksum How to use checksums	`e14243ff9c70d69b2af4aa50eebd86ad98e56da2d9595468510777973b02ad3a`
Upload date	Jul 29, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.12.10`

Release files / matrice_common-0.3.1-py3-none-any.whl

Download URL	matrice_common-0.3.1-py3-none-any.whl
Size	295.9 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`a7b10e70a12f1e54a711e8f1064aa004b1cde7263ddb583b844cf172945724f5`
BLAKE2b-256 checksum How to use checksums	`79d4c14baa6c8576421e93929cfa87941f9143f69c1fd34cdbdc37bf04ab2072`
Upload date	Jul 29, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.2.0 CPython/3.12.10`