turbofile

Real async file I/O for Python. A Rust core drives the best completion mechanism each OS has — io_uring on Linux, POSIX AIO on macOS — behind an aiofiles-compatible asyncio API.

Existing Python async-file libraries dispatch every call to a thread pool. turbofile submits the I/O to the kernel and completes your await when the kernel says the data moved:

  • Zero-copy reads and writes. The kernel fills the bytes object your await f.read(n) returns; writes pin your buffer and hand the kernel its pointer. No intermediate copies on the hot paths.
  • Completion batching. A burst of completions costs one event-loop wakeup (one call_soon_threadsafe doorbell per burst, drained entirely in Rust).
  • Parallel large reads. Read-all on a large file is split into chunks filled concurrently into one buffer.
  • Whole-file ops. read_bytes/write_bytes do open+read/write+close as a single submission — one round trip per file.
import turbofile

async def main() -> None:
    async with turbofile.open("data.bin", "rb") as f:
        payload = await f.read()

    async with turbofile.open("log.txt", "a", encoding="utf-8") as f:
        await f.write("one line\n")

    data = await turbofile.read_bytes("data.bin")   # whole file, one op
    text = await turbofile.read_text("notes.md")

turbofile.open mirrors aiofiles.open: binary and text modes, buffering, encoding/errors/newline with full universal-newline semantics, seek/tell (text cookies follow CPython's _pyio scheme), readline, async iteration, readinto, truncate, fsync via sync. Migration is import turbofile as aiofiles for the open API.

Benchmarks

make bench compares against aiofiles on your machine. On an Apple-silicon Mac (macOS 26.4, POSIX AIO backend, page-cache-hot files):

workload vs aiofiles
4 KiB whole-file read (read_bytes) 2.5x
32 concurrent 4 KiB random reads 15x
200 small files read concurrently 3.6x
8 MiB whole-file read (open + read) 1.0x
8 MiB sequential write (1 MiB chunks) 1.0x

Large sequential transfers are memory-bandwidth-bound in the page cache, so every implementation converges there; turbofile wins where per-op overhead and concurrency dominate, which is what an asyncio application actually does. io_uring numbers on Linux come from CI; run make bench there for your hardware.

Backends

OS backend mechanism
Linux compio (fusion driver) io_uring, automatic polling fallback under seccomp/old kernels
macOS darwin-aio POSIX AIO (aio_read/aio_write/aio_fsync in XNU)
macOS compio (opt-in) kqueue polling driver with thread dispatch

TURBOFILE_BACKEND=compio selects the compio driver on macOS (benchmarking, or as an escape hatch). Windows (IOCP via compio) is planned.

Limitations

  • opener= and integer file descriptors are not supported.
  • read_bytes on very large files pays one buffer copy; prefer open(...).read() for multi-megabyte files.
  • Cancelling an await detaches the future; the kernel op still completes (and, for readinto, may still write into the buffer) — standard completion-model semantics.

Development

make develop   # build the extension into the venv (uv + maturin)
make test      # cargo test + pytest
make lint      # clippy -D warnings
make bench     # benchmark against aiofiles

License

MIT OR Apache-2.0

Project links

Data verified by PyPI on Sep 1, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ransomware from gravatar.com ransomware

Credits

Author: 4thel00z

GitHub Statistics

Data verified by PyPI on Sep 1, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT OR Apache-2.0
View SPDX License List

Requires

Python >=3.12

Tags

async asyncio file-io io_uring aio iocp aiofiles rust

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

turbofile-0.1.0.tar.gz (37.8 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

turbofile-0.1.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (474.1 kB view details)

Uploaded CPython 3.12+manylinux: glibc 2.17+ x86-64

turbofile-0.1.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (470.1 kB view details)

Uploaded CPython 3.12+manylinux: glibc 2.17+ ARM64

turbofile-0.1.0-cp312-abi3-macosx_11_0_arm64.whl (423.9 kB view details)

Uploaded CPython 3.12+macOS 11.0+ ARM64

File details

Details for the file turbofile-0.1.0.tar.gz.

File metadata

  • Download URL: turbofile-0.1.0.tar.gz
  • Upload date:
  • Size: 37.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for turbofile-0.1.0.tar.gz
Algorithm Hash digest
SHA256 645030886961151d016ef19e93bcaa4f5fc73b1599c14ff1d2887a4a2474afd9
MD5 7ac61cff57f5ac932a6dac4cae064d3a
BLAKE2b-256 e133f7529cb2a7d9733631dc9354cb5105636e8eef55c6e39b19e8039a08c58c

See more details on using hashes here.

Provenance

The following attestation bundles were made for turbofile-0.1.0.tar.gz:

Publisher: release-please.yaml on 4thel00z/turbofile

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file turbofile-0.1.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for turbofile-0.1.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3a77fcefe4be092b4c17ceccfe99f0805a01775f60e5ad6bae8ea123ae69b258
MD5 b0930c472363110d7208d604547a287c
BLAKE2b-256 a95c32080f46583c05c2bf28acc2a77b6077216a121e637ed9041eb8dc604cd6

See more details on using hashes here.

Provenance

The following attestation bundles were made for turbofile-0.1.0-cp312-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release-please.yaml on 4thel00z/turbofile

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file turbofile-0.1.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for turbofile-0.1.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 125a455753efc14a05f48f569b9be0059bcab91707be358cb93cc8fc118add55
MD5 495291b4d157f9371cf1be0a43068e0c
BLAKE2b-256 7c7dad244dfe800aa5e9e92b288ab1b4c1531b1545a2091b9c54eceff44868eb

See more details on using hashes here.

Provenance

The following attestation bundles were made for turbofile-0.1.0-cp312-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release-please.yaml on 4thel00z/turbofile

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file turbofile-0.1.0-cp312-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for turbofile-0.1.0-cp312-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5a3711976031406b52d63cb25bdc347e855d725bce8dd6ec9e35cdb3248d6490
MD5 eb7298044e9cdcba30300042d18c9939
BLAKE2b-256 7de7310ec59876cff3e11893f4d50e4c03ae0c0eebf9b0597093a14c55272267

See more details on using hashes here.

Provenance

The following attestation bundles were made for turbofile-0.1.0-cp312-abi3-macosx_11_0_arm64.whl:

Publisher: release-please.yaml on 4thel00z/turbofile

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.1.0 This release

4 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page