ryutils 9.0.5

A collection of utilities for Python

My Common Python Utils

A comprehensive collection of Python utilities built over time for various projects. This library provides essential tools for HTTP requests, caching, logging, system monitoring, alerts, and more.

Installation

pip install ryutils

Features

🌐 HTTP Requests & Caching

• RequestsHelper - Advanced HTTP client with retry logic, caching, and comprehensive logging
• JsonCache - Lightweight JSON-based cache with expiration support
• Request caching - Automatic caching of GET requests with configurable expiration
• Retry strategies - Exponential backoff for failed requests
• Request logging - Comprehensive logging of all HTTP requests and responses

📊 System Monitoring & Profiling

• ResourceProfiler - CPU and memory usage profiling decorator
• System stats - Real-time system resource monitoring
• Performance tracking - Function execution time and resource consumption

📝 Logging & Output

• log - Advanced logging system with colored output and multiple handlers
• Multi-threaded logging - Per-thread log files with configurable prefixes
• Verbose output - Configurable verbosity levels for different components
• CSV logging - Structured logging to CSV files

🚨 Alerting System

• Slack alerts - Send notifications to Slack channels
• Discord alerts - Send notifications to Discord webhooks
• Mock alerts - Testing and development alert system
• Alert factory - Unified interface for different alert types

📱 SMS & Communication

• TextBelt SMS - Send SMS messages via TextBelt API
• Multi-region support - US, Canada, and international SMS support

🔧 Utilities

• Dictionary utilities - Advanced dictionary manipulation and comparison
• Path utilities - File and directory path helpers
• URL shortening - TinyURL integration for URL shortening
• Text processing - Text chunk reading and processing utilities
• Format utilities - Data formatting and conversion helpers

🔒 Security & Networking

• SSH tunneling - Modern SSH tunnel implementation using paramiko
• Proxy capture - Mitmproxy integration for request/response capture
• Header extraction - Extract headers and cookies from captured requests

🧵 Concurrency

• Worker threads - Queue-based worker thread implementation
• Thread-safe operations - Thread-safe caching and logging

Quick Start

HTTP Requests with Caching

from ryutils.requests_helper import RequestsHelper
from ryutils.verbose import Verbose

# Create a requests helper with caching
helper = RequestsHelper(
    base_url="https://api.example.com",
    verbose=Verbose(),
    cache_expiry_seconds=3600,  # 1 hour cache
    cache_file=Path("cache.json")
)

# GET request (automatically cached)
data = helper.get("/users", {"page": 1})

# POST request (clears cache for the endpoint)
result = helper.post("/users", {"name": "John"})

System Resource Profiling

from ryutils.system_stats import profile_function

@profile_function
def my_function():
    # Your code here
    return "result"

# Function execution will be automatically profiled
result = my_function()

Alerting

from ryutils.alerts.factory import AlertFactory
from ryutils.alerts.alert_types import AlertType

# Create alert factory
factory = AlertFactory()

# Send Slack alert
factory.send_alert(
    AlertType.SLACK,
    message="System is running smoothly",
    webhook_url="https://hooks.slack.com/..."
)

# Send Discord alert
factory.send_alert(
    AlertType.DISCORD,
    message="Deployment completed",
    webhook_url="https://discord.com/api/webhooks/..."
)

Logging

from ryutils import log

# Setup logging
log.setup(
    log_dir="./logs",
    log_level="INFO",
    main_thread_name="main"
)

# Use colored logging
log.print_ok("Success message")
log.print_warn("Warning message")
log.print_fail("Error message")
log.print_bright("Info message")

Dictionary Utilities

from ryutils.dict_util import (
    check_dict_keys_recursive,
    patch_missing_keys_recursive,
    flatten_dict,
    safe_get
)

# Check for missing keys recursively
missing = check_dict_keys_recursive(template_dict, data_dict)

# Patch missing keys
patched = patch_missing_keys_recursive(template_dict, data_dict)

# Flatten nested dictionary
flat = flatten_dict(nested_dict)

# Safe dictionary access
value = safe_get(data, "key.subkey", default="default")

SMS Notifications

from ryutils.sms.pytextbelt import Textbelt

# Create SMS client
sms = Textbelt()

# Send SMS
recipient = sms.Recipient(
    phone="[phone]",
    key="your-textbelt-key",
    region="us"
)

sms.send(recipient, "Hello from Python!")

SSH Tunneling

from ryutils.modern_ssh_tunnel import ModernSSHTunnel

# Create SSH tunnel
tunnel = ModernSSHTunnel(
    ssh_host="ssh.example.com",
    ssh_port=22,
    ssh_username="user",
    ssh_pkey="path/to/private/key",
    remote_host="remote.example.com",
    remote_port=3306,
    local_port=13306
)

# Start tunnel
tunnel.start()

# Use tunnel (e.g., connect to remote database on localhost:13306)
# ... your code ...

# Stop tunnel
tunnel.stop()

Configuration

Verbose Configuration

from ryutils.verbose import Verbose

verbose = Verbose(
    requests=True,           # Log HTTP requests
    requests_url=True,      # Log request URLs
    requests_response=True, # Log responses
    request_cache=True,     # Log cache operations
    alerts=True             # Log alert operations
)

Cache Configuration

from ryutils.json_cache import JsonCache

cache = JsonCache(
    cache_file=Path("cache.json"),
    expiry_seconds=3600,    # 1 hour expiration
    verbose=Verbose()
)

Dependencies

The library uses minimal external dependencies:

• requests - HTTP requests
• paramiko - SSH tunneling
• psutil - System monitoring
• memory-profiler - Memory profiling
• pyshorteners - URL shortening
• slack-sdk - Slack integration
• discord-webhook - Discord integration
• pytz - Timezone handling

Development

Setup Development Environment

# Clone the repository
git clone https://github.com/droneshire/my-py-utils.git
cd my-py-utils

# Create virtual environment
make init

# Install development dependencies
make install_dev

# Run tests
make test

# Run linting
make lint

Available Make Commands

• make init - Initialize virtual environment
• make install - Install production dependencies
• make install_dev - Install development dependencies
• make test - Run all tests
• make lint - Run linting (black, mypy, pylint)
• make format - Format code with black and isort
• make clean - Clean up temporary files

Testing

The library includes comprehensive tests for all major components:

# Run all tests
make test

# Run specific test module
make test TESTMODULE=dict_util_test

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Run the test suite
6. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For questions, issues, or contributions, please use the GitHub Issues page.