maidkit

Maidkit provides small, typed tools for Python scripts. It keeps each API narrow and avoids application framework features.

• Python 3.12+
• Apache-2.0

Installation

pip install maidkit

With uv:

uv add maidkit

Batch jobs

maidkit.batch processes a finite iterable of items on a fixed number of worker threads. It shows progress with Rich and returns results in input order.

import hashlib
from pathlib import Path

from maidkit.batch import TaskContext, run_batch


def hash_file(path: Path, task: TaskContext) -> str:
    digest = hashlib.sha256()
    total = path.stat().st_size

    task.status("hashing")
    with path.open("rb") as source:
        while chunk := source.read(1024 * 1024):
            task.check_cancelled()
            digest.update(chunk)
            task.advance(len(chunk), total=total, unit="bytes")

    return digest.hexdigest()


files = (path for path in Path("input").iterdir() if path.is_file())
result = run_batch(
    files,
    hash_file,
    title="Hash files",
    workers=4,
    label=lambda path: path.name,
)

for item in result.failed:
    print(f"{item.label}: {item.error}")

raise SystemExit(0 if result.ok else 1)

A worker receives one item and a TaskContext. The context lets the worker:

• Set its status with task.status(...).
• Report absolute or incremental progress.
• Check for cancellation with task.check_cancelled().
• Skip the item with task.skip(...).
• Run a child process with task.run_process(...).

Each item gets one ItemResult. A normal return succeeds. If a worker raises an exception, the item fails. The batch continues by default.

The display supports single-item and multi-item batches. Redirected output uses stable text instead of live progress.

Batch behavior

• run_batch reads all items and labels before it starts a worker.
• It runs at most workers items at the same time.
• It returns results in input order, even when workers finish in a different order.
• It stores worker exceptions in failed results instead of raising them.
• fail_fast=True stops new items after the first failure. Active workers continue.
• The first Ctrl+C stops scheduling new items and requests cancellation from active workers.

TaskContext.run_process captures stdout and stderr and checks the exit status. Batch cancellation also stops the child process. Output callbacks run in the worker thread and receive one line at a time without its newline.