Skip to main content

Cross-platform async file API for Python. Blazing fast like Shameimaru Aya.

Project description


📝 README.md (中文版)

# ayafileio

[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![Python Version](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/)
[![Platform](https://img.shields.io/badge/platform-Cross--platform-blue.svg)](https://en.wikipedia.org/wiki/Cross-platform)
[![Version](https://img.shields.io/badge/version-1.0.0-red.svg)]()

> **「幻想郷最速のファイルI/O、風神少女の如く」**  
> *—— 射命丸文,今日も全力で翔ける*

**跨平台异步文件 I/O 库,使用原生异步 I/O 机制。**  
Windows 上借 **IOCP**(I/O 完成端口)之力,Linux 上用 **io_uring**(内核 5.1+),macOS 上用 **Dispatch I/O (GCD)**,实现真正无阻塞的文件操作。

## 🏆 独一无二的三平台真异步

| 平台 | 后端 | 真异步 | 说明 |
|------|------|--------|------|
| **Windows** | IOCP | ✅ | NT 内核原生 I/O 完成端口 |
| **Linux** | io_uring | ✅ | Linux 5.1+ 下一代异步 I/O |
| **macOS** | Dispatch I/O | ✅ | GCD 内核级异步 I/O |

**ayafileio 是 Python 生态中唯一同时支持三大平台真异步的文件 I/O 库。**

## 📸 核心特性

| 特性 | 说明 |
|------|------|
| 🍃 **零线程开销** | 真异步平台无后台线程,无需 `run_in_executor` |
| 📰 **内核级完成** | IOCP / io_uring / Dispatch I/O 直达内核 |
| ⚡ **高并发友好** | 数千并发文件操作,文文团扇一挥间 |
| 🎴 **标准 API** | 与 aiofiles 兼容,支持 `async/await` |
| 📖 **文本二进制支持** | 文本模式自动编解码 |
| 🔧 **统一配置系统** | 运行时动态调整所有参数 |
| 🌍 **跨平台** | Windows / Linux / macOS 皆可翱翔 |
| 🐍 **最新 Python** | 支持 3.10, 3.11, 3.12, 3.13, 3.14 |

## 🛠️ 安装

```bash
pip install ayafileio

系统要求:

  • Python 3.10+
  • Windows 7+ / Linux (内核 5.1+ 可启用 io_uring) / macOS 10.10+
  • 无其他依赖,预编译 wheel 开箱即用

🚀 快速开始

import asyncio
import ayafileio

async def main():
    # 写入文件——像风一样快
    async with ayafileio.open("example.txt", "w") as f:
        await f.write("Hello, async world!\n")

    # 读取并自动解码——文文新闻,一触即达
    async with ayafileio.open("example.txt", "r", encoding="utf-8") as f:
        content = await f.read()
        print(content)

    # 二进制操作——数据如风,来去无痕
    async with ayafileio.open("data.bin", "rb") as f:
        data = await f.read(1024)
        await f.seek(0, 0)

asyncio.run(main())

🔍 后端信息

查看当前使用的后端:

import ayafileio

info = ayafileio.get_backend_info()
print(info)
# Windows: {'platform': 'windows', 'backend': 'iocp', 'is_truly_async': True}
# Linux:   {'platform': 'linux', 'backend': 'io_uring', 'is_truly_async': True}
# macOS:   {'platform': 'macos', 'backend': 'dispatch_io', 'is_truly_async': True}

⚙️ 统一配置

ayafileio 提供统一的配置系统,所有参数可在运行时动态调整:

import ayafileio

# 查看当前配置
config = ayafileio.get_config()
print(config)

# 修改配置——风势加强!
ayafileio.configure({
    "io_worker_count": 8,
    "buffer_size": 131072,      # 128KB 缓冲区
    "close_timeout_ms": 2000,
})

# 重置为默认值
ayafileio.reset_config()

配置项说明

配置项 默认值 说明
handle_pool_max_per_key 64 每个文件最大缓存句柄数 (Windows)
handle_pool_max_total 2048 全局最大缓存句柄数 (Windows)
io_worker_count 0 I/O 工作线程数,0=自动
buffer_pool_max 512 最大缓存缓冲区数
buffer_size 65536 单个缓冲区大小 (字节)
close_timeout_ms 4000 关闭时等待 pending I/O 的超时 (ms)
io_uring_queue_depth 256 io_uring 队列深度 (Linux)
io_uring_sqpoll False 是否启用 SQPOLL 模式 (Linux)
enable_debug_log False 是否启用调试日志

📚 API 参考

AsyncFile 类

class AsyncFile:
    def __init__(self, path: str | Path, mode: str = "rb", encoding: str | None = None): ...
    async def read(self, size: int = -1) -> str | bytes: ...
    async def write(self, data: str | bytes) -> int: ...
    async def seek(self, offset: int, whence: int = 0) -> int: ...
    async def flush(self) -> None: ...
    async def close(self) -> None: ...
    async def readline(self) -> str | bytes: ...
    def __aiter__(self) -> AsyncFile: ...
    async def __anext__(self) -> str | bytes: ...

支持的模式

模式 说明
"r", "rb" 读取(文本/二进制)
"w", "wb" 写入(文本/二进制)
"a", "ab" 追加(文本/二进制)
"x", "xb" 独占创建(文本/二进制)
加上 "+" 读写组合

配置函数

def configure(options: dict) -> None: ...      # 统一配置
def get_config() -> dict: ...                   # 获取当前配置
def reset_config() -> None: ...                 # 重置为默认值
def get_backend_info() -> dict: ...             # 获取后端信息

🧪 性能对比

模拟 Crawlee 爬虫框架的 Dataset 追加写入场景(5000 条记录,50 并发):

平台 ayafileio aiofiles 提速
Windows (NVMe SSD) 41,336 条/秒 9,658 条/秒 4.28x
Linux (NVMe SSD) 17,688 条/秒 11,455 条/秒 1.54x
macOS (NVMe SSD) 29,837 条/秒 25,522 条/秒 1.17x
Windows (6年旧机械盘) 20,251 条/秒 13,011 条/秒 1.56x

关键发现:

  • Windows 企业级 SSD 上,ayafileio 的 P99 写入延迟仅 0.044ms,aiofiles 为 1.854ms(低 42 倍)
  • aiofiles 在负载下抖动高达 96.7%,ayafileio 仅 16.2%
  • 即使在即将报废的机械硬盘上,ayafileio 依然保持稳定性能

测试环境:Windows 10/11, Ubuntu 22.04, macOS 14;GitHub Actions 企业级 NVMe SSD

🤝 贡献

欢迎投稿「文文。新闻」!请:

  1. Fork 本仓库
  2. 创建功能分支(git checkout -b feature/amazing-feature
  3. 添加测试
  4. 确保基准测试仍通过
  5. 提交拉取请求

📄 许可证

MIT 许可证 —— 最速最自由,详见 LICENSE


「遅いのは罪だぜ?」
—— 射命丸文,『文文。新闻』主编

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ayafileio-1.0.0-cp314-cp314-win_amd64.whl (83.5 kB view details)

Uploaded CPython 3.14Windows x86-64

ayafileio-1.0.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (104.4 kB view details)

Uploaded CPython 3.14manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

ayafileio-1.0.0-cp314-cp314-macosx_15_0_arm64.whl (69.9 kB view details)

Uploaded CPython 3.14macOS 15.0+ ARM64

ayafileio-1.0.0-cp313-cp313-win_amd64.whl (81.2 kB view details)

Uploaded CPython 3.13Windows x86-64

ayafileio-1.0.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (104.5 kB view details)

Uploaded CPython 3.13manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

ayafileio-1.0.0-cp313-cp313-macosx_15_0_arm64.whl (70.0 kB view details)

Uploaded CPython 3.13macOS 15.0+ ARM64

ayafileio-1.0.0-cp312-cp312-win_amd64.whl (81.3 kB view details)

Uploaded CPython 3.12Windows x86-64

ayafileio-1.0.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (104.6 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

ayafileio-1.0.0-cp312-cp312-macosx_15_0_arm64.whl (70.0 kB view details)

Uploaded CPython 3.12macOS 15.0+ ARM64

ayafileio-1.0.0-cp311-cp311-win_amd64.whl (82.2 kB view details)

Uploaded CPython 3.11Windows x86-64

ayafileio-1.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (105.6 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.27+ x86-64manylinux: glibc 2.28+ x86-64

ayafileio-1.0.0-cp311-cp311-macosx_15_0_arm64.whl (71.0 kB view details)

Uploaded CPython 3.11macOS 15.0+ ARM64

ayafileio-1.0.0-cp310-cp310-win_amd64.whl (82.3 kB view details)

Uploaded CPython 3.10Windows x86-64

ayafileio-1.0.0-cp310-cp310-manylinux_2_28_x86_64.whl (105.1 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

ayafileio-1.0.0-cp310-cp310-macosx_15_0_arm64.whl (71.1 kB view details)

Uploaded CPython 3.10macOS 15.0+ ARM64

File details

Details for the file ayafileio-1.0.0-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: ayafileio-1.0.0-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 83.5 kB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for ayafileio-1.0.0-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 f97cb5fa11376ca7e3cf189329c4c879f86ff12eb8a03beb52a6c16421c93438
MD5 4c521c0f6dc82510b9ad84e4b250e9c9
BLAKE2b-256 e57642c3a47160a9e1306b65552e2b8ce3458e613bb400a56837ef2dc1d38af4

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 288a6bf746e06ebfb3b218201dd13761ed3ebc2f92ff08a896012e5b7558df2e
MD5 9ecad2c84b06480d0a8228dae2b48ea3
BLAKE2b-256 cbc044a1b36c659785e899360bf20272b4b4f6580432554d6f034a1887ced3b5

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp314-cp314-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp314-cp314-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 13391d0d90478bc864d4f04f9e1267b1d6a92abcc89fd0d4baf7673366316be4
MD5 941ddf259c73c18d1d86e352c74e2cee
BLAKE2b-256 8ba76698732df69c6b5d97e797a433b8108f41671c68febe8f84407ed8e48278

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: ayafileio-1.0.0-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 81.2 kB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for ayafileio-1.0.0-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 c0a8a917537cb3cae4a1580feb9701cead88cbf10b9b9feb165051549bc0f436
MD5 deb9c4af9496ee8adb68bfa3646fd2f5
BLAKE2b-256 f2d117dc145b1cd5724a68c8e939e82f67a8a2f2bcb1bd3a3f0634a69030f86f

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d7a835ef1305203ae471c68285a8f1d763a5d84151f1b41997cabb091d8a1d3a
MD5 9722356512ee297e960bbff4d2452ddd
BLAKE2b-256 5700903fcbd16a346041f4e9391908fa36778ca7ddc0afcd61e918bedbb553f8

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp313-cp313-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp313-cp313-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 3893239499eb34c20134f7540a4389b27c43039aeb83e7ae5969f6908749bc6b
MD5 406518aa4a7a6f525fe3db3d15d4078a
BLAKE2b-256 a4601b029b9c0b92b1c45b4dcbd05b058ecec8e7961437f16b76f57931f5cca4

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: ayafileio-1.0.0-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 81.3 kB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for ayafileio-1.0.0-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 4cf01855e0330d83d6cdbc17284801907ab49ce269048ad8418243e6ad92e5e8
MD5 130cf827202e34ad739a597d4586645a
BLAKE2b-256 e2538871075ac5baa0282223e704d35080a2e7b97e2b3d6614fa00cbe10a40ce

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8826d0ae54067989efe0478ff319ff7ebc5f8f7c45baa3d918566a099e9bf8a6
MD5 ed20c8004029b0a190ecd39c1a758315
BLAKE2b-256 50f747caa6b829f838fbe3f536339169dda7e52eb57ab9ff6b0812f373aba90a

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp312-cp312-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp312-cp312-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 9819e22a741d06313f56ab2e97cb9ee53bc05cfe4d8c4bb7dc1da29e3a744493
MD5 5a730bf92797866a20ff0a3ba6db3140
BLAKE2b-256 306e77eea4905413365415b4a6a7b3ea6430082b1b4a9661dac3a8c94dc1a287

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp311-cp311-win_amd64.whl.

File metadata

  • Download URL: ayafileio-1.0.0-cp311-cp311-win_amd64.whl
  • Upload date:
  • Size: 82.2 kB
  • Tags: CPython 3.11, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for ayafileio-1.0.0-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 50853e0b107c474df00e57bf162c806e4c65563d3125c94ebc15b7f329dcc596
MD5 a018e365b3dfe818e8734d70d9835c61
BLAKE2b-256 4ffcb5172e0733412e4b733caec82af17bff4ef473f9d10a90bbf2337414aa28

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 47f450e1ff967a87a3d63fd7052d4629f634c44771071246facfa0bab3a5bf79
MD5 6337f740f95f609116a0323d51f6fd6d
BLAKE2b-256 4b6ec142c5eb14b18940e0d49897600f3a7e63b748b185838dc8f47c10bf347b

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp311-cp311-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp311-cp311-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 0031d70bed1e1d1fc938ed65540e30d2f34feca4551d6dc706eb8c4300cada6b
MD5 7e3087de86e55fc886b903cbb9f2cb57
BLAKE2b-256 1d55d7da15624527966fc3ad23c906faa3de205692c51e56537ed42f9609ebb9

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: ayafileio-1.0.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 82.3 kB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.3

File hashes

Hashes for ayafileio-1.0.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 5362108ded2d84b8ff2e0fa0187d1e4e9396fb54511f39ca6f5ab09e89407106
MD5 566f7a7cd5c4ec31666dd7d66e5e6b8a
BLAKE2b-256 97dc3fb62b41ac751cfe2dd59fa7846573f4d60e144041aa3d5e3631a5eb607d

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 aec27c12265e0eb78701175c8e5a86ea6e5d96e2ac3fd98f4fa7b1ad15b7a130
MD5 7f16336fffd9398aca48aafd954ef227
BLAKE2b-256 40bbffab80b18d2abab53337754e774d7a16b1e721041217cb2881c2717de8b9

See more details on using hashes here.

File details

Details for the file ayafileio-1.0.0-cp310-cp310-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for ayafileio-1.0.0-cp310-cp310-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 1abba7b5f0366636e417dfe02fdb45802ec7f959e4603c4a09b560709af02c76
MD5 81a707792b2e311f7e77f0d6881dbb20
BLAKE2b-256 d402a9b4574c336d55ef30ed15ee29ad5e760509ac7bd3814fbe7515b7e05ee4

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page