Cross-platform async file API for Python. Blazing fast like Shameimaru Aya.
Project description
📝 README.md (中文版)
# ayafileio
[](LICENSE)
[](https://www.python.org/)
[](https://en.wikipedia.org/wiki/Cross-platform)
[]()
> **「幻想郷最速のファイルI/O、風神少女の如く」**
> *—— 射命丸文,今日も全力で翔ける*
**跨平台异步文件 I/O 库,使用原生异步 I/O 机制。**
Windows 上借 **IOCP**(I/O 完成端口)之力,Linux 上用 **io_uring**(内核 5.1+),macOS 上用 **Dispatch I/O (GCD)**,实现真正无阻塞的文件操作。
## 🏆 独一无二的三平台真异步
| 平台 | 后端 | 真异步 | 说明 |
|------|------|--------|------|
| **Windows** | IOCP | ✅ | NT 内核原生 I/O 完成端口 |
| **Linux** | io_uring | ✅ | Linux 5.1+ 下一代异步 I/O |
| **macOS** | Dispatch I/O | ✅ | GCD 内核级异步 I/O |
**ayafileio 是 Python 生态中唯一同时支持三大平台真异步的文件 I/O 库。**
## 📸 核心特性
| 特性 | 说明 |
|------|------|
| 🍃 **零线程开销** | 真异步平台无后台线程,无需 `run_in_executor` |
| 📰 **内核级完成** | IOCP / io_uring / Dispatch I/O 直达内核 |
| ⚡ **高并发友好** | 数千并发文件操作,文文团扇一挥间 |
| 🎴 **标准 API** | 与 aiofiles 兼容,支持 `async/await` |
| 📖 **文本二进制支持** | 文本模式自动编解码 |
| 🔧 **统一配置系统** | 运行时动态调整所有参数 |
| 🌍 **跨平台** | Windows / Linux / macOS 皆可翱翔 |
| 🐍 **最新 Python** | 支持 3.10, 3.11, 3.12, 3.13, 3.14 |
## 🛠️ 安装
```bash
pip install ayafileio
系统要求:
- Python 3.10+
- Windows 7+ / Linux (内核 5.1+ 可启用 io_uring) / macOS 10.10+
- 无其他依赖,预编译 wheel 开箱即用
🚀 快速开始
import asyncio
import ayafileio
async def main():
# 写入文件——像风一样快
async with ayafileio.open("example.txt", "w") as f:
await f.write("Hello, async world!\n")
# 读取并自动解码——文文新闻,一触即达
async with ayafileio.open("example.txt", "r", encoding="utf-8") as f:
content = await f.read()
print(content)
# 二进制操作——数据如风,来去无痕
async with ayafileio.open("data.bin", "rb") as f:
data = await f.read(1024)
await f.seek(0, 0)
asyncio.run(main())
🔍 后端信息
查看当前使用的后端:
import ayafileio
info = ayafileio.get_backend_info()
print(info)
# Windows: {'platform': 'windows', 'backend': 'iocp', 'is_truly_async': True}
# Linux: {'platform': 'linux', 'backend': 'io_uring', 'is_truly_async': True}
# macOS: {'platform': 'macos', 'backend': 'dispatch_io', 'is_truly_async': True}
⚙️ 统一配置
ayafileio 提供统一的配置系统,所有参数可在运行时动态调整:
import ayafileio
# 查看当前配置
config = ayafileio.get_config()
print(config)
# 修改配置——风势加强!
ayafileio.configure({
"io_worker_count": 8,
"buffer_size": 131072, # 128KB 缓冲区
"close_timeout_ms": 2000,
})
# 重置为默认值
ayafileio.reset_config()
配置项说明
| 配置项 | 默认值 | 说明 |
|---|---|---|
handle_pool_max_per_key |
64 | 每个文件最大缓存句柄数 (Windows) |
handle_pool_max_total |
2048 | 全局最大缓存句柄数 (Windows) |
io_worker_count |
0 | I/O 工作线程数,0=自动 |
buffer_pool_max |
512 | 最大缓存缓冲区数 |
buffer_size |
65536 | 单个缓冲区大小 (字节) |
close_timeout_ms |
4000 | 关闭时等待 pending I/O 的超时 (ms) |
io_uring_queue_depth |
256 | io_uring 队列深度 (Linux) |
io_uring_sqpoll |
False | 是否启用 SQPOLL 模式 (Linux) |
enable_debug_log |
False | 是否启用调试日志 |
📚 API 参考
AsyncFile 类
class AsyncFile:
def __init__(self, path: str | Path, mode: str = "rb", encoding: str | None = None): ...
async def read(self, size: int = -1) -> str | bytes: ...
async def write(self, data: str | bytes) -> int: ...
async def seek(self, offset: int, whence: int = 0) -> int: ...
async def flush(self) -> None: ...
async def close(self) -> None: ...
async def readline(self) -> str | bytes: ...
def __aiter__(self) -> AsyncFile: ...
async def __anext__(self) -> str | bytes: ...
支持的模式
| 模式 | 说明 |
|---|---|
"r", "rb" |
读取(文本/二进制) |
"w", "wb" |
写入(文本/二进制) |
"a", "ab" |
追加(文本/二进制) |
"x", "xb" |
独占创建(文本/二进制) |
加上 "+" |
读写组合 |
配置函数
def configure(options: dict) -> None: ... # 统一配置
def get_config() -> dict: ... # 获取当前配置
def reset_config() -> None: ... # 重置为默认值
def get_backend_info() -> dict: ... # 获取后端信息
🧪 性能对比
模拟 Crawlee 爬虫框架的 Dataset 追加写入场景(5000 条记录,50 并发):
| 平台 | ayafileio | aiofiles | 提速 |
|---|---|---|---|
| Windows (NVMe SSD) | 41,336 条/秒 | 9,658 条/秒 | 4.28x |
| Linux (NVMe SSD) | 17,688 条/秒 | 11,455 条/秒 | 1.54x |
| macOS (NVMe SSD) | 29,837 条/秒 | 25,522 条/秒 | 1.17x |
| Windows (6年旧机械盘) | 20,251 条/秒 | 13,011 条/秒 | 1.56x |
关键发现:
- Windows 企业级 SSD 上,ayafileio 的 P99 写入延迟仅 0.044ms,aiofiles 为 1.854ms(低 42 倍)
- aiofiles 在负载下抖动高达 96.7%,ayafileio 仅 16.2%
- 即使在即将报废的机械硬盘上,ayafileio 依然保持稳定性能
测试环境:Windows 10/11, Ubuntu 22.04, macOS 14;GitHub Actions 企业级 NVMe SSD
🤝 贡献
欢迎投稿「文文。新闻」!请:
- Fork 本仓库
- 创建功能分支(
git checkout -b feature/amazing-feature) - 添加测试
- 确保基准测试仍通过
- 提交拉取请求
📄 许可证
MIT 许可证 —— 最速最自由,详见 LICENSE。
「遅いのは罪だぜ?」
—— 射命丸文,『文文。新闻』主编
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file ayafileio-1.0.0-cp314-cp314-win_amd64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp314-cp314-win_amd64.whl
- Upload date:
- Size: 83.5 kB
- Tags: CPython 3.14, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f97cb5fa11376ca7e3cf189329c4c879f86ff12eb8a03beb52a6c16421c93438
|
|
| MD5 |
4c521c0f6dc82510b9ad84e4b250e9c9
|
|
| BLAKE2b-256 |
e57642c3a47160a9e1306b65552e2b8ce3458e613bb400a56837ef2dc1d38af4
|
File details
Details for the file ayafileio-1.0.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 104.4 kB
- Tags: CPython 3.14, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
288a6bf746e06ebfb3b218201dd13761ed3ebc2f92ff08a896012e5b7558df2e
|
|
| MD5 |
9ecad2c84b06480d0a8228dae2b48ea3
|
|
| BLAKE2b-256 |
cbc044a1b36c659785e899360bf20272b4b4f6580432554d6f034a1887ced3b5
|
File details
Details for the file ayafileio-1.0.0-cp314-cp314-macosx_15_0_arm64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp314-cp314-macosx_15_0_arm64.whl
- Upload date:
- Size: 69.9 kB
- Tags: CPython 3.14, macOS 15.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
13391d0d90478bc864d4f04f9e1267b1d6a92abcc89fd0d4baf7673366316be4
|
|
| MD5 |
941ddf259c73c18d1d86e352c74e2cee
|
|
| BLAKE2b-256 |
8ba76698732df69c6b5d97e797a433b8108f41671c68febe8f84407ed8e48278
|
File details
Details for the file ayafileio-1.0.0-cp313-cp313-win_amd64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp313-cp313-win_amd64.whl
- Upload date:
- Size: 81.2 kB
- Tags: CPython 3.13, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
c0a8a917537cb3cae4a1580feb9701cead88cbf10b9b9feb165051549bc0f436
|
|
| MD5 |
deb9c4af9496ee8adb68bfa3646fd2f5
|
|
| BLAKE2b-256 |
f2d117dc145b1cd5724a68c8e939e82f67a8a2f2bcb1bd3a3f0634a69030f86f
|
File details
Details for the file ayafileio-1.0.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 104.5 kB
- Tags: CPython 3.13, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
d7a835ef1305203ae471c68285a8f1d763a5d84151f1b41997cabb091d8a1d3a
|
|
| MD5 |
9722356512ee297e960bbff4d2452ddd
|
|
| BLAKE2b-256 |
5700903fcbd16a346041f4e9391908fa36778ca7ddc0afcd61e918bedbb553f8
|
File details
Details for the file ayafileio-1.0.0-cp313-cp313-macosx_15_0_arm64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp313-cp313-macosx_15_0_arm64.whl
- Upload date:
- Size: 70.0 kB
- Tags: CPython 3.13, macOS 15.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3893239499eb34c20134f7540a4389b27c43039aeb83e7ae5969f6908749bc6b
|
|
| MD5 |
406518aa4a7a6f525fe3db3d15d4078a
|
|
| BLAKE2b-256 |
a4601b029b9c0b92b1c45b4dcbd05b058ecec8e7961437f16b76f57931f5cca4
|
File details
Details for the file ayafileio-1.0.0-cp312-cp312-win_amd64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp312-cp312-win_amd64.whl
- Upload date:
- Size: 81.3 kB
- Tags: CPython 3.12, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
4cf01855e0330d83d6cdbc17284801907ab49ce269048ad8418243e6ad92e5e8
|
|
| MD5 |
130cf827202e34ad739a597d4586645a
|
|
| BLAKE2b-256 |
e2538871075ac5baa0282223e704d35080a2e7b97e2b3d6614fa00cbe10a40ce
|
File details
Details for the file ayafileio-1.0.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 104.6 kB
- Tags: CPython 3.12, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8826d0ae54067989efe0478ff319ff7ebc5f8f7c45baa3d918566a099e9bf8a6
|
|
| MD5 |
ed20c8004029b0a190ecd39c1a758315
|
|
| BLAKE2b-256 |
50f747caa6b829f838fbe3f536339169dda7e52eb57ab9ff6b0812f373aba90a
|
File details
Details for the file ayafileio-1.0.0-cp312-cp312-macosx_15_0_arm64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp312-cp312-macosx_15_0_arm64.whl
- Upload date:
- Size: 70.0 kB
- Tags: CPython 3.12, macOS 15.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9819e22a741d06313f56ab2e97cb9ee53bc05cfe4d8c4bb7dc1da29e3a744493
|
|
| MD5 |
5a730bf92797866a20ff0a3ba6db3140
|
|
| BLAKE2b-256 |
306e77eea4905413365415b4a6a7b3ea6430082b1b4a9661dac3a8c94dc1a287
|
File details
Details for the file ayafileio-1.0.0-cp311-cp311-win_amd64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp311-cp311-win_amd64.whl
- Upload date:
- Size: 82.2 kB
- Tags: CPython 3.11, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
50853e0b107c474df00e57bf162c806e4c65563d3125c94ebc15b7f329dcc596
|
|
| MD5 |
a018e365b3dfe818e8734d70d9835c61
|
|
| BLAKE2b-256 |
4ffcb5172e0733412e4b733caec82af17bff4ef473f9d10a90bbf2337414aa28
|
File details
Details for the file ayafileio-1.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
- Upload date:
- Size: 105.6 kB
- Tags: CPython 3.11, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
47f450e1ff967a87a3d63fd7052d4629f634c44771071246facfa0bab3a5bf79
|
|
| MD5 |
6337f740f95f609116a0323d51f6fd6d
|
|
| BLAKE2b-256 |
4b6ec142c5eb14b18940e0d49897600f3a7e63b748b185838dc8f47c10bf347b
|
File details
Details for the file ayafileio-1.0.0-cp311-cp311-macosx_15_0_arm64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp311-cp311-macosx_15_0_arm64.whl
- Upload date:
- Size: 71.0 kB
- Tags: CPython 3.11, macOS 15.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0031d70bed1e1d1fc938ed65540e30d2f34feca4551d6dc706eb8c4300cada6b
|
|
| MD5 |
7e3087de86e55fc886b903cbb9f2cb57
|
|
| BLAKE2b-256 |
1d55d7da15624527966fc3ad23c906faa3de205692c51e56537ed42f9609ebb9
|
File details
Details for the file ayafileio-1.0.0-cp310-cp310-win_amd64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp310-cp310-win_amd64.whl
- Upload date:
- Size: 82.3 kB
- Tags: CPython 3.10, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
5362108ded2d84b8ff2e0fa0187d1e4e9396fb54511f39ca6f5ab09e89407106
|
|
| MD5 |
566f7a7cd5c4ec31666dd7d66e5e6b8a
|
|
| BLAKE2b-256 |
97dc3fb62b41ac751cfe2dd59fa7846573f4d60e144041aa3d5e3631a5eb607d
|
File details
Details for the file ayafileio-1.0.0-cp310-cp310-manylinux_2_28_x86_64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp310-cp310-manylinux_2_28_x86_64.whl
- Upload date:
- Size: 105.1 kB
- Tags: CPython 3.10, manylinux: glibc 2.28+ x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
aec27c12265e0eb78701175c8e5a86ea6e5d96e2ac3fd98f4fa7b1ad15b7a130
|
|
| MD5 |
7f16336fffd9398aca48aafd954ef227
|
|
| BLAKE2b-256 |
40bbffab80b18d2abab53337754e774d7a16b1e721041217cb2881c2717de8b9
|
File details
Details for the file ayafileio-1.0.0-cp310-cp310-macosx_15_0_arm64.whl.
File metadata
- Download URL: ayafileio-1.0.0-cp310-cp310-macosx_15_0_arm64.whl
- Upload date:
- Size: 71.1 kB
- Tags: CPython 3.10, macOS 15.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1abba7b5f0366636e417dfe02fdb45802ec7f959e4603c4a09b560709af02c76
|
|
| MD5 |
81a707792b2e311f7e77f0d6881dbb20
|
|
| BLAKE2b-256 |
d402a9b4574c336d55ef30ed15ee29ad5e760509ac7bd3814fbe7515b7e05ee4
|