aiofilepool
Async file-descriptor pool for Python.
FilePool caps how many OS file descriptors are open at once,
while your code can freely hold thousands of
logical async file handles.
- Async-native - built on
asyncio; blocking I/O is offloaded to a thread pool. - Chunked I/O - large reads/writes split via pluggable chunkers; you can open streams with
chunks(). - Zero dependencies - pure standard library.
Install
pip install aiofilepool
# or
uv add aiofilepool
Requires Python 3.12+.
Usage
import asyncio
from aiofilepool import FilePool
async def main():
# at most 128 real OS fds open at once
async with FilePool(descriptor_pool_size=128) as pool:
handle = await pool.open("data.bin", "w+")
await handle.write(b"hello world")
await handle.seek(0)
print(await handle.read()) # b"hello world"
await handle.fsync()
# handles auto-close on exit from pool. Also you can do this manually:
await handle.close()
# or with a manager:
async with pool.open("data.bin", "w+") as fd:
...
asyncio.run(main())
It bounds only the real fds and manages temporary closing and re-opening them, so this is completely fine:
async with FilePool(descriptor_pool_size=64) as pool:
handles = [await pool.open(f"part-{i}.bin", "w+") for i in range(1000)]
await asyncio.gather(*(h.write(payload) for h in handles))
Also you can read a large file in chunks:
handle = await pool.open("big.bin", "r")
async for chunk in handle.chunks():
process(chunk)
Although, the handle.read uses chunking too, so chunks are there just for convenience
API
FilePool- the pool:open(path, mode),manage(binary_io),close(), and an async context manager.FileHandle- an async file handle:read,write,seek,tell,sizeetc. A handler you use to talk to a file.AsyncBinaryIO- the async binary-IO protocolFileHandleimplements.BinaryIOAdapter- allows to wrap an existing synchronousBinaryIOas anAsyncBinaryIO.ModeSpec- a parsed file open-mode.Chunker,FixedChunker,BalancedChunker- chunking strategies. You can use them withAsyncBinaryIO.chunksor as a default for theFilePool.aiofilepool.errors- the exception hierarchy.
License
MIT — see LICENSE.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
aiofilepool-0.1.0.tar.gz
(9.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file aiofilepool-0.1.0.tar.gz.
File metadata
- Download URL: aiofilepool-0.1.0.tar.gz
- Upload date:
- Size: 9.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
5530a5d13461d17d5d23404e53c743d66467412e31b66efc7b491a2bf0c3d987
|
|
| MD5 |
044c759aef1faf879a25b164b453a15b
|
|
| BLAKE2b-256 |
4ec12c95940c8c403055358b6f42f53c4fa8cf90dad51b3fe9c447793c30701e
|
Provenance
The following attestation bundles were made for aiofilepool-0.1.0.tar.gz:
Publisher:
release.yml on MiloLug/aiofilepool
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
aiofilepool-0.1.0.tar.gz -
Subject digest:
5530a5d13461d17d5d23404e53c743d66467412e31b66efc7b491a2bf0c3d987 - Sigstore transparency entry: 2676960811
- Sigstore integration time:
-
Permalink:
MiloLug/aiofilepool@d6873ada5ed65eb261b90b96c8e988cfb47ba8ab -
Branch / Tag:
refs/heads/main - Owner: https://github.com/MiloLug
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@d6873ada5ed65eb261b90b96c8e988cfb47ba8ab -
Trigger Event:
push
-
Statement type:
File details
Details for the file aiofilepool-0.1.0-py3-none-any.whl.
File metadata
- Download URL: aiofilepool-0.1.0-py3-none-any.whl
- Upload date:
- Size: 14.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
dac56765ff57c86de1c00a864789ee116fc8b9e40c5d7b97a423eb82f3d08595
|
|
| MD5 |
b5071df1ce277625c78eccc4f473c36c
|
|
| BLAKE2b-256 |
0d531decffde6e18d4edfc53194cfbda0b8798ba295d2d779f296ce015fe7b4d
|
Provenance
The following attestation bundles were made for aiofilepool-0.1.0-py3-none-any.whl:
Publisher:
release.yml on MiloLug/aiofilepool
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
aiofilepool-0.1.0-py3-none-any.whl -
Subject digest:
dac56765ff57c86de1c00a864789ee116fc8b9e40c5d7b97a423eb82f3d08595 - Sigstore transparency entry: 2676960697
- Sigstore integration time:
-
Permalink:
MiloLug/aiofilepool@d6873ada5ed65eb261b90b96c8e988cfb47ba8ab -
Branch / Tag:
refs/heads/main - Owner: https://github.com/MiloLug
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release.yml@d6873ada5ed65eb261b90b96c8e988cfb47ba8ab -
Trigger Event:
push
-
Statement type:
