advlog

A Python logging library with colored terminal output, file management, progress tracking, and run records for experiments, built on top of Rich.

Personal project notice: This package was written for personal development use. Update frequency and feature scope may be limited. Contributions and suggestions are welcome.

Features

• Rich Console Output — Colored terminal logging via the Rich library
• Flexible Configuration — LoggerConfig dataclass for per-logger overrides; simple initialize() / get_logger() API for common cases
• File Logging — Separate or merged file strategies with size-based rotation
• Structured Output — Optional JSON Lines file with your own fields (extra=, bound context) for parsing and querying
• Progress Tracking — Integrated progress bars via ProgressTracker
• Multi-Module Coordination — LoggerManager for consistent logging across modules
• Run Records — Git commit, host and command line of each run, its configuration, per-step metrics and output files as queryable records
• Smart File Naming — Timestamped, daily, and incremental log file naming

Installation

pip install advlog-core

With all optional features:

pip install "advlog-core[all]"

The PyPI package name is advlog-core; import as import advlog.

Quick Start

One-liner

from advlog import get_logger

log = get_logger(__name__)
log.info("Application started")
log.warning("High memory usage detected")
log.error("Database connection failed")

Initialize once, use everywhere

from advlog import initialize, get_logger, get_progress

initialize(
    output_dir="./logs",
    session_name="myapp",
    log_level="DEBUG",
    show_location=True,
)

log = get_logger(__name__)
log.info("Initialized")

progress = get_progress()
with progress:
    task = progress.add_task("Processing", total=100)
    for i in range(100):
        progress.update(task, advance=1)

Things worth knowing:

• Loggers obtained before initialize() are console-only; they pick up the log file as soon as initialize() runs.
• Calling initialize() again applies the new settings in full to every existing logger. In particular, enable_file_logging=False detaches and closes the current log file.
• use_color=False turns color off for everything drawn on the shared console, progress bars included.
• get_logger(name, log_level="ERROR") sets the level even if that logger already exists.

Libraries: log through the standard library

get_logger() is for application code: it gives the logger its own handlers and stops it from propagating, so the host program's handlers and pytest's caplog never see its records. A package meant to be imported by others should log the standard way and leave output to the application:

# mylib/registry.py
import logging

log = logging.getLogger(__name__)
log.info("registered %d samples", n)

The application then routes those records through advlog with capture:

from advlog import initialize

initialize(output_dir="./logs", session_name="sim", capture=["mylib"])

• Every logger under the captured name (mylib.registry, mylib.io, ...) goes to the advlog console and log file(s), with bound context fields attached.
• Records keep propagating, so handlers the host installed on the root logger and pytest's caplog still receive them. If the host also has a console handler on the root logger (e.g. from logging.basicConfig()), those lines appear twice, and that handler now also gets the package's INFO/DEBUG records, because propagation checks handler levels, not the root logger's level. Capture the package or configure a root console handler, not both.
• The captured logger's level is lowered to the lowest level the advlog handlers accept (DEBUG with file logging) and restored by reset() or by a later initialize() that no longer lists it. A logger obtained with get_logger() under exactly the captured name keeps the level and handlers get_logger() gave it.
• capture=["root"] collects every library, including third-party DEBUG chatter in the log file; prefer naming packages.
• LoggerManager offers the same through LoggerManager(capture=...), capture(name) and release(name).

Per-logger configuration with LoggerConfig

from advlog import LoggerManager, LoggerConfig

manager = LoggerManager(shared_console=True, shared_file="logs/app.log")

# Default registration
api_logger = manager.register_logger("api")

# Override with LoggerConfig
db_logger = manager.register_logger(config=LoggerConfig(
    name="database",
    log_level="DEBUG",
    log_file="logs/db.log",
    show_location=True,
))

api_logger.info("API server started")
db_logger.debug("Executing query: SELECT ...")

LoggerConfig fields and where they apply:

Register a group of loggers at once

from advlog import create_logger_group

loggers = create_logger_group(
    ["api", "database", "auth"],
    shared_console=True,
    file_strategy="merged",
)

loggers["api"].info("Request received")
loggers["database"].debug("Query executed")
loggers["auth"].warning("Invalid login attempt")

Formatters

Four formatters are available:

from advlog import JSONFormatter
import logging

handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logging.getLogger("myapp").addHandler(handler)

File Logging

from advlog import create_file_handler

handler = create_file_handler(
    log_file="logs/app.log",
    mode="a",
    show_location=True,
)

Files hold plain text: paired Rich tags such as [bold red]alert[/bold red] color the console but are removed from file output. Brackets that are not a closed style pair (arr[i], [1/3], [epoch 1]) are left as written. Pass strip_markup=False to create_file_handler() to keep the tags.

Rotation

For long runs, cap the file size. When the file reaches max_file_size bytes it is renamed to app.log.1 (older ones shift to .2, .3, ...) and a fresh file is started; backup_count old files are kept.

from advlog import initialize

initialize(
    output_dir="./logs",
    session_name="train",
    max_file_size=50 * 1024 * 1024,  # 50 MB
    backup_count=3,
)

max_file_size / backup_count are also accepted by LoggerManager, LoggerConfig, create_file_handler() and create_file_handlers(), and apply to the .jsonl file too. A rotating file is always appended to, so file_mode="w" has no effect on it.

Structured (JSON Lines) output

Set file_format to get a machine-readable log next to (or instead of) the text log:

from advlog import initialize, get_logger

initialize(output_dir="./logs", session_name="train", file_format="both")

log = get_logger(__name__)
log.info("Run started", extra={"lr": 1e-3, "batch_size": 32})

The .jsonl file holds one JSON object per line. Anything passed via extra= is stored under the "extra" key:

{"timestamp": "2026-03-06T14:30:22.123456+08:00", "level": "INFO", "logger": "__main__", "message": "Run started", "module": "train", "function": "<module>", "line": 6, "extra": {"lr": 0.001, "batch_size": 32}}

Values JSON cannot represent (paths, sets, NumPy arrays, tensors, arbitrary objects) are converted rather than dropped, and NaN/Infinity are written as null so every line stays valid JSON. Arrays and tensors with more than 1,000 elements are written as a summary instead of in full, so logging a field array cannot put megabytes into one line:

"extra": {"field": {"_type": "array_summary", "shape": [1024, 1024], "dtype": "complex64", "size": 1048576}}

Use JSONFormatter(max_array_size=...) on your own handler to change the limit (None writes every array in full). Complex numbers are written as {"real": 1.0, "imag": 2.0}; for a complex array, real and imag each hold an array of the original shape, so np.array(v["real"]) + 1j * np.array(v["imag"]) restores it. file_format is also accepted by LoggerManager, LoggerConfig, and create_logger_group; create_file_handlers() builds the handlers directly.

import pandas as pd

df = pd.read_json("logs/train.jsonl", lines=True)
errors = df[df["level"] == "ERROR"]

Lines meant only for human eyes (separators, banners) can be kept out of the .jsonl file:

log.info("=" * 60, extra={"text_only": True})  # console and text log only

Context fields on every record

Bind fields once and they are attached to every record from every logger, as if passed via extra= each time. This is what lets you tell runs apart after merging their logs.

from advlog import initialize, bind_context, unbind_context, clear_context

initialize(session_name="train", file_format="both", context={"run_id": "exp-42", "git_sha": "ab6e876"})

bind_context(epoch=3)     # add or update fields later
unbind_context("epoch")   # remove some
clear_context()           # remove all

{"timestamp": "...", "level": "INFO", "logger": "train", "message": "step done", "module": "train", "function": "fit", "line": 88, "extra": {"run_id": "exp-42", "git_sha": "ab6e876", "epoch": 3}}

An explicit extra= on a call wins over a bound field of the same name. Bound fields are also available to custom text formats, e.g. LoggerConfig(log_format="%(asctime)s %(run_id)s %(message)s"). Names of built-in record attributes (message, name, levelname, ...) are rejected.

Distributed and parallel runs

With torchrun, accelerate launch or DeepSpeed every process runs your script. To have only rank 0 write to the console and log files:

initialize(session_name="train", main_process_only=True)

The rank comes from accelerate once Accelerator() / PartialState() exists, and otherwise from the RANK environment variable (or LOCAL_RANK if only that is set), so records are filtered from the first line on. Other ranks create no log file.

Processes started together would otherwise race for the same file: the auto-generated name has one-second resolution. So when initialize() picks the name itself, it appends the Slurm array task and the rank of a multi-process run:

An explicit log_file_path is used exactly as given. For other parallel launches (e.g. a sweep script starting several processes), pass a distinct suffix or log_file_path per process.

Smart log file naming

from advlog import LogNamingStrategy

# Timestamp-based:  logs/2026-03-06/20260306_143022_app_prod.log
log_file = LogNamingStrategy.timestamped("app", suffix="prod")

# Daily directory:  logs/2026-03-06/myapp_errors.log
log_file = LogNamingStrategy.daily("myapp", suffix="errors")

# Incremental (no conflicts):  logs/backup_data.log  →  backup_data_1.log ...
log_file = LogNamingStrategy.incremental("backup", suffix="data")

Progress Tracking

from advlog import ProgressTracker

with ProgressTracker() as progress:
    download = progress.add_task("Downloading", total=100)
    process  = progress.add_task("Processing",  total=50)

    for _ in range(100):
        progress.update(download, advance=1)

    for _ in range(50):
        progress.update(process, advance=1)

Run Records

For simulations, training runs and experiments in general, four functions each write one record, tagged with extra.event:

from advlog import initialize, log_artifact, log_config, log_metrics, log_run_info

initialize(session_name="sim", file_format="both")

info = log_run_info(packages=["numpy", "scipy"])          # once, at the start
log_config({"tol": 1e-10, "grid": {"nx": 128, "ny": 128}})  # dict, OmegaConf config or any object
for it in range(n_iter):
    ...
    log_metrics({"residual": res, "energy": e}, step=it)  # extra fields: log_metrics(..., phase="relax")
log_artifact("field", "out/field.npy")                    # each output file

• git describes the repository containing the running script (or repo_dir=). dirty is true when tracked files have uncommitted changes, which is the case where the commit alone does not reproduce the run. Untracked files are ignored, since results written into the repository would otherwise mark every run dirty. git is null outside a repository or without git.
• All four log through the managed advlog logger unless you pass logger=. collect_run_info() returns the run info without logging it, e.g. to bind_context(git_sha=info["git"]["commit"]).
• The per-key lines and separators of log_config are marked text_only, so the .jsonl file gets the single "config" record rather than one record per printed line.
• For a training loop, tag records with keyword fields and bound context: log_metrics({"loss": loss, "lr": lr}, step=step, phase="train") per batch, log_metrics(val_metrics, step=step, phase="val") per epoch, after bind_context(epoch=epoch).
• The records report the file and line of your call (visible with show_location=True and in the JSON module / function / line fields), not a location inside advlog.

Exception Logging

from advlog import setup_exception_logging

# Log uncaught exceptions, in the main thread and in other threads
setup_exception_logging()

# In a Slurm job (or anything stopped with kill / docker stop): also log SIGTERM
setup_exception_logging(sigterm=True)

The traceback goes wherever your other logs go: the console and the log file(s) configured by initialize(), including the .jsonl file (as an ERROR record with an exception field). Pass your own logger to route it elsewhere. KeyboardInterrupt is left to Python's default handling, as is SystemExit in a thread; threads=False leaves threading.excepthook alone.

SIGTERM is what Slurm sends at the time limit and on scancel. By default Python dies on it at once: no finally blocks, no context managers, no atexit handlers, and nothing in the log. With sigterm=True an ERROR record (extra.event = "terminated") is written and SystemExit(143) is raised, so cleanup code runs and the exit status is the usual one for SIGTERM. A SIGTERM handler you installed earlier is called instead of exiting. Call it from the main thread. Slurm sends SIGKILL after a grace period (KillWait, 30 s by default), so keep the cleanup short.

Real-World Example

See examples/real_world_demo/ for a complete multi-module application demonstrating global initialization, cross-module loggers, progress tracking, error handling, and organized file output.

python -m examples.real_world_demo.main
python -m examples.real_world_demo.structured_demo   # JSON Lines output and querying it
python -m examples.real_world_demo.run_records_demo  # run info, metrics and artifacts of a solver run

Dependencies

Required

• Rich — Terminal formatting and colored output

Optional

pip install "advlog-core[config]"
pip install "advlog-core[distributed]"

Contributing

Contributions are welcome — please feel free to open an issue or submit a pull request.

License

Apache License 2.0 — see LICENSE for details.

Acknowledgments

• Rich for the terminal formatting library
• Python logging for the logging infrastructure