Skip to main content

A YAML-based logging library with rich formatting for container and Kubernetes environments

Project description

LogNimbus

lognimbus

Upload Python Package

LogNimbus is a YAML-based logging library with rich formatting for container and Kubernetes environments.

Features

  • Human-Readable Format: YAML's indentation-based structure makes logs easy to read and write, facilitating quick understanding.
  • Structured Data: Supports hierarchical logging with nested structures, allowing for detailed and organized logs.
  • Tool Compatibility: Seamlessly integrates with modern logging and monitoring tools like Fluentd and Logstash, which efficiently parse YAML.
  • Unified Configuration: Aligns with YAML-based configuration management tools (e.g., Kubernetes, Helm), ensuring consistency across your stack.
  • Extensible and Flexible: Easily add custom fields and adapt log structures to evolving application requirements.
  • Machine-Readable: Simplifies automated log analysis and processing due to its structured nature.
  • Error Reduction: Schema validation helps maintain consistent and error-free log entries.
  • Prometheus Monitoring: Integrated Prometheus metrics to monitor logging activity.
  • Contextual Logging: Includes context in log messages for better traceability.
  • Sensitive Data Masking: Masks sensitive fields in the log data to protect sensitive information.

Roadmap

LogNimbus Roadmap

Feature Status Description
Human-Readable Format Implemented YAML's indentation-based structure makes logs easy to read and write.
Structured Data Implemented Supports hierarchical logging with nested structures.
Tool Compatibility Implemented Seamlessly integrates with tools like Fluentd and Logstash.
Unified Configuration Implemented Aligns with YAML-based configuration management tools like Kubernetes and Helm.
Extensible and Flexible Implemented Easily add custom fields and adapt log structures.
Machine-Readable Implemented Simplifies automated log analysis and processing due to its structured nature.
Error Reduction Implemented Schema validation helps maintain consistent and error-free log entries.
Prometheus Monitoring Implemented Integrated Prometheus metrics to monitor logging activity.
Contextual Logging Implemented Includes context in log messages for better traceability.
Sensitive Data Masking Implemented Masks sensitive fields in the log data to protect sensitive information.
Dynamic Configuration Reloading Implemented Allows reloading configuration at runtime without restarting the application.
Integration with Logging Module Implemented Provides a handler to integrate with Python’s built-in logging module.
Error Notifications via Slack Implemented Sends error notifications to Slack using a webhook URL.
Asynchronous Logging Implemented Implemented to avoid blocking operations.
Log Rotation and Retention Implemented Supports log rotation based on size and retention policies.
Time-Based Log Rotation Planned Rotate logs based on time intervals (daily, weekly, monthly).
Nested Contexts Planned Support nested contexts for more granular contextual information.
Custom Retention Policies Planned Allow defining custom retention policies for old logs.
Error Notifications via Email Planned Integrate with email notification systems.
Retry Mechanism for Logging Planned Implement a retry mechanism for logging failures.
Advanced Log Filters Planned Allow users to define filters to include or exclude certain log messages.
Search and Query Interface Planned Provide an interface to search and query logs.
Custom Metrics Planned Allow users to define custom Prometheus metrics related to their application.
Health Check Endpoint Planned Provide an endpoint to monitor the health of the logging system.
Encryption for Log Files Planned Encrypt log files to ensure data security.
Audit Logging Planned Provide tamper-evident logging for audit purposes.
Interactive Log Viewer Planned Develop a web-based or terminal-based interactive log viewer for exploring logs.
Customizable Log Formats Planned Allow users to define custom log formats and templates.

Installation

pip install lognimbus

Usage

Basic Usage

You can start using LogNimbus with minimal configuration, which defaults to console logging only.

from lognimbus import YamlLogger

# Initialize the logger with default settings (console logging only)
logger = YamlLogger()

# Log messages at different levels
logger.debug(msg="This is a debug message")
logger.info(msg="This is an info message")
logger.warning(msg="This is a warning message")
logger.error(msg="This is an error message")
logger.critical(msg="This is a critical message")

Advanced Usage

Override default settings and enable additional features as needed.

from lognimbus import YamlLogger, LogContext

# Initialize the logger with custom settings
logger = YamlLogger(
    log_file='app_logs.yml',
    log_rotation_enabled=True,
    max_log_size=5242880,  # 5MB
    backup_count=3,
    prometheus_enabled=True,
    slack_webhook_url='https://hooks.slack.com/services/your/webhook/url'
)

# Log messages at different levels
logger.debug(msg="This is a debug message")
logger.info(msg="This is an info message")
logger.warning(msg="This is a warning message")
logger.error(msg="This is an error message")
logger.critical(msg="This is a critical message")

# Log a message with additional data
additional_data = {
    'User': {'id': 12345, 'name': "John Doe"},
    'Operation': 'Data Processing'
}
logger.info(msg="Data processing completed", data=additional_data)

# Add contextual information and log messages with context
with LogContext(request_id='12345', user_id='john_doe'):
    logger.info(msg="User login attempt")
    try:
        # Simulate a login operation
        raise ValueError("Invalid credentials")
    except Exception as e:
        logger.log_exception(e, msg="Login failed")

# Log a message with sensitive data
sensitive_data = {
    'password': 'secret_password',
    'credit_card_number': '1234-5678-9876-5432'
}
logger.info(msg="User payment processing", data=sensitive_data)

Integration with Python's Built-in Logging Module

You can integrate LogNimbus with Python's built-in logging module for seamless logging.

from lognimbus import YamlLogger, LogNimbusHandler
import logging

# Initialize the logger
logger = YamlLogger()

# Integrate with Python's built-in logging module
log_handler = LogNimbusHandler(logger)
logging.basicConfig(level=logging.DEBUG, handlers=[log_handler])
python_logger = logging.getLogger(__name__)

# Log messages at different levels using the standard logging module
python_logger.debug("This is a debug message")
python_logger.info("This is an info message")
python_logger.warning("This is a warning message")
python_logger.error("This is an error message")
python_logger.critical("This is a critical message")

Configuratio

You can also configure LogNimbus using a YAML configuration file. Here is an example configuration file (lognimbus_config.yml):

lognimbus:
  log_file: "app_logs.yml"
  additional_log_file: "backup_logs.yml"
  console_logging: true
  log_level: "INFO"
  log_rotation:
    enabled: true
    max_size: 10485760  # 10MB
    backup_count: 5
  prometheus_port: 8000
  sensitive_data_masking:
    enabled: true
    fields:
      - "password"
      - "credit_card_number"
  notifications:
    slack_webhook_url: "https://hooks.slack.com/services/your/webhook/url"  # Optional

To use the configuration file:

from lognimbus import YamlLogger

# Initialize the logger from a configuration file
logger = YamlLogger(config_file='lognimbus_config.yml')

# Log messages as usual
logger.info(msg="Configured via YAML file")

Why YAML-Based Logging?

YAML-based logging offers several advantages:

  • Readability: Easy for humans to read and write.
  • Structured Data: Captures complex data in a clear, hierarchical format.
  • Compatibility: Works well with modern logging tools and container management systems.
  • Flexibility: Easily extendable with custom fields.
  • Consistency: Uniform format for both configuration and logs in cloud-native environments.

LogNimbus makes logging in YAML format simple and effective, enhancing both human readability and machine parsing capabilities, making it an excellent choice for modern development practices and cloud-native applications.

"Buy Me A Coffee"

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lognimbus-0.1.2.tar.gz (8.4 kB view details)

Uploaded Source

Built Distribution

lognimbus-0.1.2-py3-none-any.whl (9.2 kB view details)

Uploaded Python 3

File details

Details for the file lognimbus-0.1.2.tar.gz.

File metadata

  • Download URL: lognimbus-0.1.2.tar.gz
  • Upload date:
  • Size: 8.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for lognimbus-0.1.2.tar.gz
Algorithm Hash digest
SHA256 5fc7cfaf68e26cdd41ab8fd6417274854faa0d9ca5e42ff747cc69f18504693b
MD5 2b33efe7a96a3f1384f86670c5281160
BLAKE2b-256 c130d49f2adc3659172ef67b5231ff5913b95ca817b59ffe71167783f6c9ed55

See more details on using hashes here.

File details

Details for the file lognimbus-0.1.2-py3-none-any.whl.

File metadata

  • Download URL: lognimbus-0.1.2-py3-none-any.whl
  • Upload date:
  • Size: 9.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for lognimbus-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 a4738dcb690e54483fffa2f3127450dc329c86f88422a59b0bf040f7717a0060
MD5 9335c01fafadd85ff78f6d5682ab4646
BLAKE2b-256 47011c91d08d9fadb6e66552652ff3ded04642f395dce6bbb52819e745d7ade4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page