aiojournal

An asynchronous batch execution runner with results persistence and resumability.

Especially useful when making concurrent API calls while directly saving each request's result, for example when benchmarking LLM models, from big providers or inference engines. It is minimal and lightweight, only dependency is tqdm.

pip install aiojournal

Example

import asyncio
from pathlib import Path

from aiojournal import AsyncBatchRunner


async def describe(value: int) -> dict[str, int | str]:
    await asyncio.sleep(0.1)
    return {
        "result": value * 2,
        "input": value,
        "parity": "even" if value % 2 == 0 else "odd",
    }


async def main() -> None:
    runner = AsyncBatchRunner(
        max_concurrency=2,
        results_file_path=Path("tasks.csv"),
    )
    results = await runner.map(
        describe,
        args_list=[{"value": 1}, {"value": 2}, {"value": 3}],
        task_ids=["one", "two", "three"],
    )
    # {"result": 2, "input": 1, "parity": "odd"}
    # {"result": 4, "input": 2, "parity": "even"}
    # {"result": 6, "input": 3, "parity": "odd"}


asyncio.run(main())

Each task appends a row to tasks.csv. The journal can be read without loading the whole file:

for result in runner.read_results_csv():
    print(result["task_id"], result["success"], result["result"])

And will look like this (with different timestamps and durations):

task_id,start_time,end_time,duration_seconds,success,result,error,input,parity
one,2026-08-23T16:00:00+00:00,2026-08-23T16:00:00.100000+00:00,0.1000,True,2,,1,odd
two,2026-08-23T16:00:00+00:00,2026-08-23T16:00:00.100000+00:00,0.1000,True,4,,2,even
three,2026-08-23T16:00:00.100000+00:00,2026-08-23T16:00:00.200000+00:00,0.1000,True,6,,3,odd

Columns names are inferred from the dictionary returned by the function provided to the map method. The function may also return plain text, which in this case will be written in a result column.

By default, if a task fails, map and gather returns/saves it's exception while running the rest of the batch.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for NathanFradet from gravatar.com NathanFradet

Credits

Author: Nathan Fradet

License

Creative Commons Legal Code

Requires

Python >=3.10.0

Provides Extra

tests

Tags

asyncio batch-processing concurrency journaling

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aiojournal-0.0.1.tar.gz (15.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

aiojournal-0.0.1-py3-none-any.whl (14.1 kB view details)

Uploaded Python 3

File details

Details for the file aiojournal-0.0.1.tar.gz.

File metadata

  • Download URL: aiojournal-0.0.1.tar.gz
  • Upload date:
  • Size: 15.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for aiojournal-0.0.1.tar.gz
Algorithm Hash digest
SHA256 ddc7a3332aef1995d6b4c48324465bd688af361722de954d53fd35837729ca3e
MD5 934c894d01ad7ea7a7319872af4b59e2
BLAKE2b-256 9c0fbe7ca61be5047573751d22c628f83236064decb45b0b87e95272fee35dbb

See more details on using hashes here.

Provenance

The following attestation bundles were made for aiojournal-0.0.1.tar.gz:

Publisher: publish-pypi.yml on Natooz/aiojournal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file aiojournal-0.0.1-py3-none-any.whl.

File metadata

  • Download URL: aiojournal-0.0.1-py3-none-any.whl
  • Upload date:
  • Size: 14.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for aiojournal-0.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 94f2cfd67bf342075ef631adc9aa41c8430e2a9a116773206a2e57da12d79e60
MD5 26f637abc2ccf9bec114a2a0c9216baa
BLAKE2b-256 dccf67457e7887e275577e77345199bcf50e09dd5fdd78e394aaa85b8ed734f4

See more details on using hashes here.

Provenance

The following attestation bundles were made for aiojournal-0.0.1-py3-none-any.whl:

Publisher: publish-pypi.yml on Natooz/aiojournal

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.0.1 This release

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page