Skip to main content

Multiprocessing-safe domain-based logging framework with QueueListener architecture

Project description

repo = LoggingRepository(config)
queue = repo.get_queue()

Worker processes:

worker_repo = LoggingRepository(config, use_listener=False, queue=queue)
worker_repo.task.info("Worker started")

All processes log safely to the same listener and files.


Debug Control

Only .debug() messages are conditional via debug_enabled:

config.debug_enabled = False
repo.app.debug("Won't appear")
repo.app.info("Will appear")

Useful for toggling debug messages in production.


Output Structure

Logs are written in:

logs/
 ├─ app.log         # domain logs
 ├─ db.log
 ├─ redis.log
 ├─ debug.log       # per-level logs
 ├─ info.log
 ├─ warning.log
 └─ error.log

Console output is colorized by level:

  • DEBUG → blue
  • INFO → green
  • WARNING → yellow
  • ERROR → red

Files remain clean.


Graceful Shutdown

The repository automatically shuts down on process exit and supports manual shutdown:

repo.shutdown()

Typical Usage Example

repo.app.info("Service starting")

try:
    connect_db()
except Exception:
    repo.db.exception("DB connection failed")

repo.redis.debug("Cache size: %d", cache_size)

Multiprocessing workers:

def worker(queue):
    repo = LoggingRepository(config, use_listener=False, queue=queue)
    repo.task.info("Worker task started")

Project Structure

miu_logger/
 ├─ repository.py         # main LoggingRepository
 ├─ listener.py           # QueueListener and handler setup
 ├─ logger_factory.py     # logger creation functions
 ├─ conditional.py        # ConditionalLogger
 ├─ filters.py            # Logger filters
 ├─ formatters.py         # ColoredFormatter
 ├─ config.py             # LogConfig definition
 └─ stubgen.py            # stub generator for IDE autocomplete

When to Use

  • Services with many subsystems
  • Multiprocessing ingestion pipelines
  • Long-running daemons
  • Kubernetes / systemd services
  • Need for clear operational logs

When Not to Use

  • Single-file scripts
  • No multiprocessing
  • No domain separation required

Plain logging is sufficient in those cases.


License

MIT


Author

Bruno Miura

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

miu_logger-0.1.6.tar.gz (7.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

miu_logger-0.1.6-py3-none-any.whl (8.2 kB view details)

Uploaded Python 3

File details

Details for the file miu_logger-0.1.6.tar.gz.

File metadata

  • Download URL: miu_logger-0.1.6.tar.gz
  • Upload date:
  • Size: 7.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.9

File hashes

Hashes for miu_logger-0.1.6.tar.gz
Algorithm Hash digest
SHA256 e5784e6c1a8a2d687232c75e3cd1c132bc380a35167ad5c9a4b5b48323d20954
MD5 9e474fc539c6be167fc8719994312fc6
BLAKE2b-256 002fd08b30d62c288f943786b1901073a58926d37e31e4a703aad1d59b3abea4

See more details on using hashes here.

File details

Details for the file miu_logger-0.1.6-py3-none-any.whl.

File metadata

  • Download URL: miu_logger-0.1.6-py3-none-any.whl
  • Upload date:
  • Size: 8.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.9

File hashes

Hashes for miu_logger-0.1.6-py3-none-any.whl
Algorithm Hash digest
SHA256 53e113a4c6460e277fbfc0b4c611421c680143cbd1b1e6c56ab5db030268ba62
MD5 a8eda49b2f2fa10d46c0357d96c45e0f
BLAKE2b-256 5ee59aa0afe0d029f50e641072cd0a4de79b42e72f4ff983ce3112a2d5f57edc

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page