aiologging
Asynchronous logging library for Python (3.9–3.14). The API mirrors the standard logging module — same method names, signatures, hierarchy and semantics — with the logging methods being coroutines. Records are created at the call site and put on a queue; a background consumer fans them out to per-handler workers that perform the I/O, so await logger.info(...) never waits for a file write or an HTTP request.
Features
- logging-compatible API: same methods, levels, hierarchy, filters and
LogRecordsemantics as the standardloggingmodule - Background consumer: handler I/O happens off the calling coroutine's path in per-handler workers, so one slow handler never delays the others; everything starts lazily and survives event loop changes
- Configurable delivery:
awaitresolves on enqueue (default) or after handlers processed the record (delivery="await") - Configurable backpressure: bounded queue with
block(default),drop_newordrop_oldoverflow policies - Stdlib bridge:
captureStdlib()routes third-party library logs (aiohttp, sqlalchemy, ...) through the same async handlers - Async Handlers: non-blocking I/O for streams, files (with size/time rotation), HTTP endpoints with extensible authentication, and Telegram chats
- Buffered Handlers: batch processing for high-volume logging
- Performance Metrics: built-in metrics for loggers, handlers and the queue
- Strict Type Checking: full mypy support with type hints
Installation
Basic Installation
pip install aiologging
With Optional Dependencies
# For file handlers
pip install aiologging[aiofiles]
# For HTTP handlers (pick one backend)
pip install aiologging[aiohttp]
pip install aiologging[httpx]
# For Protobuf support
pip install aiologging[protobuf]
# All dependencies
pip install aiologging[all]
# Development dependencies
pip install aiologging[dev]
Quick Start
import asyncio
import aiologging
async def main():
aiologging.basicConfig(level=aiologging.INFO)
logger = aiologging.getLogger("app")
await logger.info("Application started")
await logger.warning("Something might be wrong")
await logger.error("An error occurred: %s", "details")
# once, before the loop goes away: drain the queue, close handlers
await aiologging.shutdown()
asyncio.run(main())
Module-level convenience functions work like in standard logging:
await aiologging.warning("Logged on the root logger")
How it works
await logger.info(...) synchronously creates the LogRecord (so caller info, %-formatting and exc_info behave exactly like standard logging) and puts it on a bounded queue. A background task drains the queue and fans each record out to per-handler dispatch queues, each drained by its own worker — so a slow handler (say, an HTTP sink honouring a long retry-after) delays only its own queue, never the other handlers. Per-handler ordering is preserved. By default the await resolves as soon as the record is enqueued — logging is nearly free for the calling coroutine.
Lifecycle:
- the consumer and the handler workers start lazily on the first logged record and are rebuilt transparently if the event loop changes (e.g. one loop per test) — including handler resources such as HTTP sessions and open files;
await aiologging.flush(timeout=...)waits until everything queued has been handled (optionally bounded; raisesasyncio.TimeoutErroron expiry);await aiologging.shutdown(timeout=...)drains the queue and closes all handlers — call it once at application exit;timeoutbounds the drain phase to fit e.g. a container's termination grace period;aiologging.flush_sync(timeout=5.0)is the synchronous emergency drain for contexts without a running loop (afinallyafterasyncio.run, plain sync code): it delivers queued and buffered records on a private event loop and returnsFalseinstead of raising when the budget expires;- at interpreter exit an automatic drain spends up to 2 seconds delivering whatever is left, then warns on stderr about anything undelivered. Configure with
basicConfig(atexit_flush=...)oraiologging.set_atexit_flush();0disables the drain. atexit does not run when the process dies from an unhandled signal — handle SIGTERM (e.g.sys.exit) for this to cover container shutdowns.
Delivery and backpressure
aiologging.basicConfig(
level=aiologging.INFO,
queue_size=10_000, # queue capacity
overflow="block", # "block" | "drop_new" | "drop_old"
delivery="enqueue", # "enqueue" | "await"
)
# per-logger override for critical logs: the await resolves only
# after the handlers have processed the record
audit = aiologging.getLogger("app.audit")
audit.delivery = "await"
overflow="block"(default): when the queue is full,await logger.info(...)waits for free space — no records are lost.overflow="drop_new"/"drop_old": the call never waits; the incoming or the oldest record is discarded and counted in metrics.
Overflow treats all records alike, so a burst of DEBUG/INFO can push an ERROR out of a full queue. The opt-in LevelAwareDrop policy sacrifices low-severity records instead (mirroring rsyslog discardSeverity / logback discardingThreshold, both opt-in there too):
aiologging.basicConfig(
drop_policy=aiologging.LevelAwareDrop(
discard_below=logging.WARNING, # DEBUG/INFO are expendable
watermark=0.8, # act only when the queue is 80%+ full
),
)
Above the watermark fill ratio, arriving records below discard_below are discarded; on a completely full queue the oldest expendable queued record is evicted to make room for the arriving one; with no expendable victims the configured overflow policy applies unchanged. The policy covers the main queue and the per-handler dispatch queues; any object implementing DropPolicyProtocol can replace the level rule.
Capturing stdlib logging
Third-party libraries log via the standard logging module. Route their records through your async handlers:
aiologging.basicConfig(level=aiologging.INFO, capture_stdlib=True)
# or:
aiologging.captureStdlib()
Bridged records are routed through the aiologging hierarchy under the same logger name, so per-name handlers and propagation work as usual. The bridge is thread-safe and buffers records emitted before the event loop starts.
For alerting channels the bridge can also deliver critical records inline — synchronously, before they are queued — so the pager message leaves even when the record is emitted right before the process dies (an except block before a raise, a SIGTERM handler):
aiologging.captureStdlib(inline_level=logging.ERROR)
# or: aiologging.basicConfig(capture_stdlib=True, inline_level=logging.ERROR)
Records at or above inline_level are offered to handlers that support synchronous delivery (emit_sync, currently AsyncTelegramHandler, which POSTs via urllib in a helper thread joined with the deadline). The inline path is bounded — inline_timeout (default 3 s) per record, plus a token bucket of inline_burst sends (default 1) refilling at inline_rate per minute (default 2) — so a burst of errors cannot stall the calling thread. On any failure the record simply stays on the normal queue path; a handler that received it inline is skipped by its queue worker, other handlers deliver as usual. Note the FIFO trade-off: an inline record reaches the chat ahead of older queued records — include %(asctime)s in the formatter so readers can reorder.
Examples
Complete runnable examples live in the examples/ directory:
- examples/basic_usage.py — levels,
basicConfig,exc_info, delivery modes,flush/shutdown - examples/stdlib_capture.py — routing third-party (stdlib
logging) records through async handlers, including from threads and before the loop starts - examples/file_logging.py — file handler, size- and time-based rotation
- examples/http_logging.py — JSON batches over HTTP with a custom authenticator (includes a local test collector)
- examples/telegram_logging.py — sending records to a Telegram chat, including rate-limit handling (includes a local Bot API stand-in)
- examples/config_usage.py — configuring loggers from a dictionary
- examples/emergency_flush.py — delivery at process death:
shutdown(timeout=...),flush_sync, the automatic atexit drain - examples/inline_alerts.py — inline (synchronous) Telegram delivery of critical bridged records via
captureStdlib(inline_level=...) - examples/drop_policy.py — level-aware overflow with
LevelAwareDrop: errors survive queue pressure at the cost of excess INFO context
Handlers
Stream Handler
import sys
import aiologging
async def main():
logger = aiologging.getLogger("app")
logger.addHandler(aiologging.AsyncStreamHandler(sys.stdout))
await logger.info("This goes to stdout")
await aiologging.shutdown()
File Handler (requires aiofiles)
import aiologging
async def main():
logger = aiologging.getLogger("app")
logger.addHandler(aiologging.AsyncFileHandler("app.log"))
await logger.info("This goes to app.log")
await aiologging.shutdown()
Rotating File Handler (requires aiofiles)
import aiologging
async def main():
logger = aiologging.getLogger("app")
# Size-based rotation
logger.addHandler(aiologging.AsyncRotatingFileHandler(
"app.log",
max_bytes=1024*1024, # 1MB
backup_count=5,
))
# Time-based rotation
logger.addHandler(aiologging.AsyncTimedRotatingFileHandler(
"timed.log",
when="midnight",
backup_count=7,
))
await logger.info("This will be rotated")
await aiologging.shutdown()
HTTP Handler (requires aiohttp or httpx)
import aiologging
async def main():
logger = aiologging.getLogger("app")
logger.addHandler(aiologging.AsyncHttpHandler(
"https://api.example.com/logs",
headers={"Authorization": "Bearer token"},
))
await logger.info("This will be sent via HTTP")
await aiologging.shutdown()
HTTP handlers work on top of either aiohttp or httpx. By default
aiohttp is used when installed, with a fallback to httpx. The backend
can also be selected explicitly:
http_handler = aiologging.AsyncHttpHandler(
"https://api.example.com/logs",
backend="httpx", # or "aiohttp"
)
Telegram Handler (requires aiohttp or httpx)
Sends log records to a Telegram chat via the Bot API. Buffered
records are combined into as few sendMessage calls as possible,
each within the 4096-character limit; a 429 Too Many Requests
response is retried after the delay the Bot API returns in
retry_after.
import aiologging
async def main():
logger = aiologging.getLogger("app")
logger.addHandler(aiologging.AsyncTelegramHandler(
token="123456:ABC-DEF...", # from @BotFather
chat_id="-1001234567890", # or "@channelname"
level=aiologging.ERROR,
parse_mode=None, # or "HTML" / "MarkdownV2"
))
await logger.error("This will be sent to Telegram")
await aiologging.shutdown()
Custom Authentication
import aiologging
async def oauth_authenticator(session, request_data):
"""Custom OAuth authentication."""
token = await refresh_oauth_token()
return {"Authorization": f"Bearer {token}"}
http_handler = aiologging.AsyncHttpHandler(
"https://api.example.com/logs",
authenticator=oauth_authenticator,
)
HTTP Handler Formats
JSON Handler
json_handler = aiologging.AsyncHttpJsonHandler(
"https://api.example.com/logs"
)
Text Handler
text_handler = aiologging.AsyncHttpTextHandler(
"https://api.example.com/logs"
)
Protobuf Handler (requires protobuf)
proto_handler = aiologging.AsyncHttpProtoHandler(
"https://api.example.com/logs"
)
Universal Handler (auto-detect format)
universal_handler = aiologging.AsyncHttpHandler(
"https://api.example.com/logs",
format_type="application/json" # Optional: auto-detected if not specified
)
Advanced Usage
Custom Filters
import aiologging
class CustomFilter:
def filter(self, record):
return "important" in record.getMessage()
logger = aiologging.getLogger("app")
logger.addFilter(CustomFilter())
Custom Formatters
Formatters live on handlers, exactly like in standard logging:
import logging
import aiologging
handler = aiologging.AsyncStreamHandler()
handler.setFormatter(logging.Formatter(
"%(asctime)s [%(levelname)s] %(name)s: %(message)s"
))
aiologging.getLogger("app").addHandler(handler)
Error Handling
import aiologging
async def error_handler(record, exception):
"""Custom error handler for failed log operations."""
print(f"Failed to log {record.getMessage()}: {exception}")
handler = aiologging.AsyncStreamHandler()
handler.error_handler = error_handler
Performance Metrics
from aiologging.logger import _logger_manager
logger = aiologging.getLogger("app")
print(logger.get_metrics()) # per-logger counters
print(_logger_manager.get_metrics()) # queue length, dropped records
for handler in logger.handlers:
print(handler.get_metrics()) # per-handler counters
Batch Processing
import aiologging
from aiologging.types import BatchConfig
http_handler = aiologging.AsyncHttpHandler(
"https://api.example.com/logs",
batch_config=BatchConfig(
batch_size=100,
flush_interval=5.0,
max_retries=3,
),
)
Configuration Management
Configuration from Dictionary
import asyncio
import aiologging
config = {
"version": 1,
"loggers": {
"myapp": {
"level": "INFO",
"handlers": ["console", "file"]
}
},
"handlers": {
"console": {
"class": "stream",
"level": "INFO",
"stream": "stdout"
},
"file": {
"class": "file",
"level": "DEBUG",
"filename": "app.log",
"mode": "a"
}
}
}
aiologging.configure_from_dict(config)
async def main():
logger = aiologging.get_configured_logger("myapp")
await logger.info("This uses configured logger")
await aiologging.shutdown()
asyncio.run(main())
A handler's "class" is either a built-in name (stream, file,
http, http_json, telegram, rotating_file,
timed_rotating_file) or a dotted path to your own AsyncHandler
subclass, as in stdlib logging.config.dictConfig:
config["handlers"]["custom"] = {
"class": "myapp.logging.MyHandler", # imported at build time
"level": "WARNING",
"endpoint": "https://example.com", # extra keys -> constructor
}
Configuration from File
aiologging.configure_from_file("logging_config.json")
Migration from Standard Logging
# before
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("app")
logger.info("Message")
# after
import aiologging
aiologging.basicConfig(level=aiologging.INFO)
logger = aiologging.getLogger("app")
await logger.info("Message") # + await
await aiologging.shutdown() # once, at application exit
Key differences:
- Await the logging methods —
debug/info/.../critical,logandexceptionare coroutines; everything else (setLevel,addHandler,getChild, ...) stays synchronous. - Call
await aiologging.shutdown()at exit — the queue must be drained while the event loop is still running. - Handlers are async — use the
Async*handler classes (or bridge stdlib records withcaptureStdlib()).
API Reference
Logger Methods
Async (require await):
await logger.log(level, msg, *args, **kwargs)await logger.debug/info/warning/error/critical(msg, *args, **kwargs)await logger.exception(msg, *args, exc_info=True, **kwargs)
All accept the standard keyword arguments: exc_info, extra, stack_info, stacklevel.
Sync (identical to logging.Logger):
setLevel(level),getEffectiveLevel(),isEnabledFor(level)addHandler(h)/removeHandler(h)/hasHandlers()addFilter(f)/removeFilter(f)/filter(record)getChild(suffix)/getChildren()makeRecord(...),findCaller(stack_info, stacklevel)- attributes:
name,level,parent,propagate,handlers,filters,disabled
Module Functions
getLogger(name=None)— hierarchical loggers; no name returns the root loggerbasicConfig(level, format, datefmt, handlers, force, queue_size, overflow, delivery, drop_policy, capture_stdlib, inline_level, atexit_flush)await flush(timeout=None)— wait until every queued record has been handledawait shutdown(timeout=None)— drain the queue and close all handlersflush_sync(timeout=5.0)— synchronous emergency drain without a running loop; returnsTruewhen fully deliveredset_atexit_flush(timeout)— budget of the automatic drain at interpreter exit (default 2.0; 0 disables)disable(level)— likelogging.disablecaptureStdlib(capture=True, level=NOTSET, inline_level=None, inline_timeout=3.0, inline_burst=1, inline_rate=2.0)— bridge stdlib logging records;inline_levelenables synchronous inline delivery of critical recordsawait debug/info/warning/error/exception/critical/log(...)— root-logger convenience coroutines
Handler Classes
AsyncHandler- Base async handlerAsyncStreamHandler- Stream output handlerAsyncFileHandler- File output handler (requires aiofiles)AsyncRotatingFileHandler- Size-based rotation (requires aiofiles)AsyncTimedRotatingFileHandler- Time-based rotation (requires aiofiles)AsyncHttpHandler- Universal HTTP handler (requires aiohttp or httpx)AsyncHttpTextHandler- Plain text HTTP handler (requires aiohttp or httpx)AsyncHttpJsonHandler- JSON HTTP handler (requires aiohttp or httpx)AsyncHttpProtoHandler- Protobuf HTTP handler (requires aiohttp or httpx, plus protobuf)StdlibBridgeHandler- Synclogging.Handlerforwarding records into aiologging
Configuration Classes
BatchConfig- Batch processing configurationFileConfig- File handler configurationHttpConfig- HTTP handler configurationLoggerConfig- Logger configurationRotationConfig- Rotation configuration for file handlers
Exception Classes
AiologgingError- Base exception for all aiologging errorsHandlerError- Base exception for handler errorsConfigurationError- Configuration-related errorsDependencyError- Missing optional dependenciesAuthenticationError- Authentication failuresNetworkError- Network-related errorsFileError- File operation errorsRotationError- File rotation errorsBatchError- Batch processing errorsFormatterError- Formatting errorsLoggerError- Logger operation errorsContextError- Context manager errors
Performance Considerations
- Delivery mode: the default
enqueuemakes logging nearly free for the caller; reservedelivery="await"for records that must be confirmed - Backpressure: pick the overflow policy consciously —
blocknever loses records,drop_*never stalls the application - Buffering: use buffered/batching handlers for high-volume logging
- Metrics Collection: enable metrics to monitor drops and errors
- Rate Limiting: use rate limiters to prevent log flooding
- Shutdown: always
await aiologging.shutdown()before exit so nothing queued is lost
Testing
# Install development dependencies
pip install aiologging[dev]
# Run tests
pytest
# Run tests with coverage
pytest --cov=aiologging
# Run type checking
mypy aiologging
Contributing
- Fork the repository
- Create a feature branch
- Add tests for new functionality
- Ensure all tests pass and type checking succeeds
- Submit a pull request
License
MIT License - see LICENSE file for details.
Changelog
0.2.9
Three delivery-guarantee fixes, all found by the new stress harness
(stress/, see docs/stress-testing.md):
- The atexit drain delivered zero file records: aiofiles needs
executors, which are already shut down at interpreter exit —
AsyncFileHandlernow falls back to blocking file I/O there flush_syncafter an event-loop death lost the in-flight records when the queues were full: rescuing a dead queue no longer wakes its dead-loop putters, teardown errors are no longer mistaken for handler failures, and the rescue never evicts records silentlyforce_flush()(and thereforeflush(),flush_sync()and the atexit drain) flushed at most onemax_batch_sizebatch, leaving the rest buffered while reporting success — it now drains the buffers completely
0.2.0
Breaking: the logging pipeline is now queue-based and the API strictly follows the standard logging module.
- Records are enqueued at the call site and dispatched by a background consumer —
await logger.info(...)no longer waits for handler I/O - Configurable
delivery("enqueue"/"await") andoverflow("block"/"drop_new"/"drop_old") viabasicConfigor per logger await aiologging.flush()/await aiologging.shutdown()lifecycle; the consumer starts lazily and survives event loop changes- Stdlib bridge:
captureStdlib()/basicConfig(capture_stdlib=True)routes standardloggingrecords through async handlers - Module-level convenience coroutines (
aiologging.info(...), ...) andaiologging.disable(level) - API aligned with
logging.Logger:getEffectiveLevel,getChild,getChildren,hasHandlers,makeRecord,findCaller,stacklevelsupport,parent/propagate/filtersattributes;getLogger()without arguments returns the root logger - Removed:
Logger.setFormatter/getFormatter(formatters live on handlers),getLevel(usegetEffectiveLevel),disable()/enable()methods (use thedisabledattribute oraiologging.disable),setParent/getParent,getRootLogger,getLoggerContext,log_async async with logger:now flushes instead of closing the shared logger instance- Handlers can be constructed outside a running event loop on Python 3.9
0.1.1
- Python 3.13 and 3.14 support
- Replace deprecated
asyncio.iscoroutinefunctionwithinspect.iscoroutinefunction - Remove dead Python 3.8 compatibility branches
0.1.0
- Initial release
- Full async logging API
- Stream, file, and HTTP handlers
- File rotation support
- Extensible authentication
- Optional dependencies
- Strict type checking
Metadata
Release files for aiologging 0.2.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| aiologging-0.2.9.tar.gz | 123.0 kB | Without provenance | Details for aiologging-0.2.9.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| aiologging-0.2.9-py3-none-any.whl | Python 3 | none | any | Details for aiologging-0.2.9-py3-none-any.whl |
Total release size: 204.4 kB
Release files / aiologging-0.2.9.tar.gz
| Download URL | aiologging-0.2.9.tar.gz |
|---|---|
| Size | 123.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f8147a59648cbafb8fe179cb3405d81d2cc3ac690815f5b0087cdb4f4b040b1c
|
|
BLAKE2b-256 checksum How to use checksums |
7abe0537d5a6966d77b45a192cf38831a2af8d964652eb8dde302597922c955b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Release files / aiologging-0.2.9-py3-none-any.whl
| Download URL | aiologging-0.2.9-py3-none-any.whl |
|---|---|
| Size | 81.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
98468442681d4572548ab709680eca1c468911a9fcce0a62ba0378b7338a4bb3
|
|
BLAKE2b-256 checksum How to use checksums |
33a99e6a3df1648acb5fa704d8b2c209754ab8662936fa6992daaf882e4c6368
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Security for aiologging 0.2.9
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.