chutils 3.2.0

Набор простых и удобных утилит для Python, который избавляет от рутины при работе с конфигурацией и логированием в новых проектах.

Русская версия

chutils: Stop the Routine!

chutils is a set of simple utilities for Python designed to eliminate the repetitive setup of configuration, logging, and secrets in your projects.

Start a new project and focus on what matters, not the routine.

Full documentation is available on our website (currently in Russian).

The Problem

Every time you start a new project, you have to solve the same tasks:

• How to conveniently read settings from a configuration file?
• How to configure logging to write messages to both the console and a file with daily rotation?
• How to securely store API keys without hardcoding them in the code?
• How to make it all work "out of the box" without manually defining paths?

chutils offers ready-made solutions for all these problems.

Key Features

• ✨ Zero Configuration: The library automatically finds your project root and the config.yml or config.ini file. It uses lazy initialization — no heavy operations until you actually need them.
• ⚙️ Flexible Configuration: Support for YAML and INI formats. Simple functions for retrieving typed data.
• ✍️ Advanced Logger: The setup_logger() function configures logging to the console and rotating files out of the box. Includes automatic secret masking and smart console width detection for IDEs (PyCharm, etc.). It returns a custom logger with additional debug levels (devdebug, mediumdebug).
• 🔒 Secure Secret Storage: The secret_manager module provides a simple interface for saving and retrieving secrets via the system keyring, with a fallback to .env files.
• 🚀 CLI Booster: Turn any function into a CLI tool in seconds using the @cli_command decorator with automatic type mapping and docstring parsing.
• ⏰ Painless Datetime: Always-aware UTC time utilities, smart parsing, and human-readable time intervals.
• 📡 Distributed Tracing: Seamless OpenTelemetry integration with @trace decorator and automatic log correlation.
• 🔍 Config Diagnostics: Interactive trace of configuration sources and priorities via config debug command.
• 🌐 Remote Configuration: Load settings from HTTP/HTTPS endpoints with background polling and fallback support.
• 🔄 Hot-Reload: Support for automatic configuration reloading on file changes without restart (requires pip install chutils[watch]).
• 🛡️ Secure Paths: Prevent Path Traversal attacks by safely resolving file paths against a base directory using resolve_safe_path().
• 🤖 AI Linter: Run static analysis checks on your codebase to ensure AI readiness (strict type hints, structured docstrings, API map sync, and file dependency synchronization) via chutils dev ai-lint.
• ✅ Data Validation (chutils.validation): An all-in-one Pydantic-based data and JSON validation module featuring @validate_call decorator for function argument checks and rich exception formatting.
• 🩺 Diagnostics API (chutils.diagnostics): An asynchronous, thread-safe health checking framework with built-in checks (keyring, config) and integrations for popular web frameworks (FastAPI, Flask).
• ⚡ Async Ready: Most core functions have asynchronous versions (prefixed with a) for non-blocking execution.
• 🚀 Ready to Use: Just install and use.

Command Line Interface (CLI)

The library provides a chutils console command for convenient secret management without writing code.

Secret Management

# Save a secret to the system storage (keyring)
chutils secrets set MY_API_KEY "super-secret-value"

# Delete a secret
chutils secrets delete MY_API_KEY

# Explicitly specify service name
chutils secrets set DB_PASSWORD "12345" --service my_custom_app

Installation

poetry add chutils

Or using pip:

pip install chutils

Examples

In the /examples folder, you will find ready-to-run scripts demonstrating the library's key features. Each example focuses on a specific task.

Quick Start

1. Working with Configuration

1. (Optional) Create a config.yml file in your project root:

   # config.yml
   Database:
     host: localhost
     port: 5432
   

2. Get values in your code:

   from chutils import get_config_value, get_config_int
   
   db_host = get_config_value("Database", "host", fallback="127.0.0.1")
   db_port = get_config_int("Database", "port", fallback=5432)
   

   Validation via Pydantic (Optional)

   For strict typing and validation, you can use Pydantic models (requires pip install chutils[pydantic]):

   from pydantic import BaseModel
   from chutils import get_config
   
   class AppConfig(BaseModel):
       app_name: str
       version: str
   
   # Returns an instance of AppConfig
   cfg = get_config(model=AppConfig)
   print(cfg.app_name)
   

   You can also validate specific sections:

   from chutils import get_config_section
   db_cfg = get_config_section("Database", model=MyDbModel)
   

   Overriding Configuration with Local Files (config.local.yml)

   You can create a config.local.yml next to your main file. Values from the local file will override corresponding values from the main file. This is perfect for local development or storing sensitive data (ensure *.local.* is in your .gitignore).

2. Hot-Reload

You can make your application react to configuration file changes in real-time.

from chutils import start_config_watcher, on_config_change, get_config_value


def reload_logic():
    print("Configuration updated!")
    # Update app state here
    db_url = get_config_value("Database", "url")


# Register callback
on_config_change(reload_logic)

# Start watcher (requires watchdog package)
start_config_watcher()

To use this feature, install watchdog: pip install chutils[watch]

3. Logging Setup

1. Configure and use the logger:

   from chutils import setup_logger_from_config, ChutilsLogger
   
   # Automatically reads settings from [Logging] section in config.yml
   logger: ChutilsLogger = setup_logger_from_config()
   
   logger.info("Application started.")
   logger.devdebug("Deep debug message (level 9).")
   

   Structured Logging (JSON)

   If you need to output logs in JSON format for ELK, Splunk, or cloud logging (requires pip install chutils[json]):

   # Via code
   logger = setup_logger(json_format=True)
   
   # Or via config in the [Logging] section
   # json_format: true
   

   Contextual Logging (ContextVar)

   You can bind metadata to the current execution context (thread or coroutine), and it will be automatically included in all log messages.

   from chutils import setup_logger, bind_context
   
   logger = setup_logger()
   
   # Bind request ID and user to the current context
   bind_context(request_id="REQ-123", user="admin")
   
   logger.info("Action performed")
   # Text: ... [request_id=REQ-123 user=admin] Action performed
   # JSON: {..., "message": "Action performed", "context": {"request_id": "REQ-123", "user": "admin"}}
   

   Controlling Logging via Environment Variables

   • CH_LOG_JSON=true: Forces JSON format.
   • CH_LOG_NO_TIME=true: Removes the date/time from the log format (for clean Docker logs).
   • CH_LOG_NO_FILE=true: Disables creating log files.

   These variables have highest priority and override any code or config settings.

3. Secret Management

SecretManager provides a unified abstraction for managing secrets using a custom chain of providers (SecretProvider) and automatic fallback logic.

By default, SecretManager searches for secrets in the following order: **Keyring > .env File > Environment Variables **.

from chutils import SecretManager

secrets = SecretManager("my_awesome_app")

# Save secret (will be written to the first writable provider in the chain)
secrets.save_secret("API_KEY", "secret-value-123")

# Retrieve secret
key = secrets.get_secret("API_KEY")

Built-in Providers

You can configure the providers list explicitly:

from chutils import SecretManager
from chutils.secret_manager.providers import KeyringProvider, DotEnvProvider, EnvProvider

secrets = SecretManager(
    service_name="my_app",
    providers=[
        KeyringProvider(),
        DotEnvProvider(),
        EnvProvider()
    ]
)

Cloud Secret Managers

The library supports transparent integration with AWS Secrets Manager and Google Secret Manager:

from chutils import SecretManager
from chutils.secret_manager.providers import AWSSecretManagerProvider, GCPSecretManagerProvider, EnvProvider

# Initialize cloud providers (requires installation of chutils[aws,gcp])
aws_provider = AWSSecretManagerProvider(region_name="us-east-1")
gcp_provider = GCPSecretManagerProvider(project_id="my-gcp-project")

# Create a SecretManager with the chain: AWS -> GCP -> Environment Variables.
# If AWS or GCP calls fail due to network/permissions or the secret is missing,
# SecretManager logs a warning and automatically falls back to EnvProvider.
secrets = SecretManager(
    service_name="my_service",
    providers=[aws_provider, gcp_provider, EnvProvider()]
)

Install cloud dependencies:

pip install chutils[aws]   # For AWS (boto3)
pip install chutils[gcp]   # For GCP (google-cloud-secret-manager)
pip install chutils[all]   # For all optional packages

Disabling Keyring (Optional)

In environments like Docker or CI/CD where keyring is unavailable, you can suppress warnings and skip the check:

• Set CH_DISABLE_KEYRING_WARNING=true in environment.
• Or add disable_keyring: true under secrets section in config.yml.

4. Safe Path Resolution

Safely resolve relative and absolute paths against a base directory to prevent directory traversal vulnerabilities:

from chutils.fs import resolve_safe_path
from chutils.exceptions import PathTraversalError

try:
    # Resolves path relative to the base directory and checks if it's safe
    safe_path = resolve_safe_path("user_file.txt", base_dir="./sandbox")
    print(f"Safe absolute path: {safe_path}")
except PathTraversalError as e:
    print(f"Path traversal detected! Attempted: {e.context.get('attempted_path')}")

API Overview

Configuration (chutils.config)

• get_config_value(section, key, fallback=None, required=False) / aget_config()
• get_config_int, get_config_boolean, get_config_list, get_config_path (all support fallback and required parameters)
• save_config_value(section, key, value, notify=True) / asave_config_value()
• Use notify=False to update the file without triggering Hot-Reload callbacks.

Logging (chutils.logger)

• setup_logger(name, log_level, log_file_name, rotation_type, compress, ...)
• Levels: logger.devdebug (9), logger.mediumdebug (15), and all standard ones.

Secret Management (chutils.secret_manager)

• SecretManager(service_name, prefix)
• save_secret / asave_secret
• get_secret(key, fallback=None, required=False) / aget_secret(...)
• delete_secret / adelete_secret

Environment Manifest (chutils.env)

• BaseEnvManifest: Declarative specification of environment variables using Pydantic typing, automatic secret masking, and load() classmethod.

Data Validation (chutils.validation)

• validate_data(model, data): Validates dictionaries or JSON strings against Pydantic models, raising ChutilsValidationError.
• @validate_call: Automatically validates function arguments using Pydantic.

Health Checking (chutils.diagnostics)

• DiagnosticsManager: Registers and аsynchronously executes system health checks with timeouts.
• Web Framework helpers: get_fastapi_health_handler and get_flask_health_handler.

Decorators (chutils.decorators)

• @log_function_details: Logs arguments, execution time, and result (uses DEVDEBUG level).
• @timeout(seconds, fallback): Limits function execution time. Supports sync/async and optional fallback.
• @retry: Automatically retries a function if it fails. Supports sync/async, backoff, jitter, and exception filtering.
• @cli_command: Turns any function into a standalone CLI script with automatic argument parsing.

File System (chutils.fs)

• resolve_safe_path(path, base_dir): Safely resolves path and checks for Path Traversal attempts.
• ensure_dir(path): Ensures directory exists.

Time & Dates (chutils.time)

• utc_now(): Returns a timezone-aware UTC datetime.
• parse_datetime(value): Smart parsing of strings and timestamps into UTC.
• humanize_timedelta(dt): Returns human-readable strings like "5 minutes ago" or "tomorrow".

Example of @retry usage:

from chutils.decorators import retry


@retry(retries=3, delay=1.0, backoff=2.0)
def fetch_data():
    # Will be retried up to 3 times on any Exception
    ...

Command Line Interface (CLI)

chutils includes a set of tools to speed up development and debugging.

1. Initialize Project

Quickly set up a new project with a default configuration and .gitignore rules:

# Interactive mode
chutils init

# Non-interactive mode (default values)
chutils init -y

2. Validate Configuration

Check if your configuration files match your Pydantic models:

# Automatically finds 'Settings' class in context.py or config.py
chutils validate

# Explicitly specify the model path
chutils validate --model src.settings:AppConfig

3. Debug Search Paths

See exactly where chutils is looking for configuration files:

# Human-readable output
chutils show-paths

# JSON output for automation
chutils show-paths --json

4. Manage Secrets

Manage your system keyring secrets directly from the terminal:

# Set a secret
chutils secrets set API_KEY "your-secret-value" --service my_app

# Delete a secret
chutils secrets delete API_KEY --service my_app

5. Debug Configuration

Trace exactly where each configuration value comes from:

chutils config debug

6. AI-Readiness Linter

Perform a static audit of your codebase to ensure it is optimized for AI developers and agents:

# Run linter on the current project
chutils dev ai-lint

# Run linter in strict mode with ignored paths
chutils dev ai-lint --strict --ignore "temp/,build/"

See docs/ai_lint.md for more details.

7. Environment Variables Synchronization

Keep .env and .env.example files in sync with format preservation, comments, and values masking:

# Compare .env and .env.example keys without making changes
chutils dev sync-env --dry-run

# Synchronize files automatically
chutils dev sync-env --yes

# Specify custom paths
chutils dev sync-env --env-path .env.dev --example-path .env.dev.example

License

The project is distributed under the MIT License.