Skip to main content

Cross-platform async file API for Python. Blazing fast like Shameimaru Aya.

Project description

ayafileio

License Python Version Platform Speed

「幻想郷最速のファイルI/O、風神少女の如く」
—— 射命丸文,今日も全力で翔ける

跨平台异步文件 I/O 库,使用原生异步 I/O 机制。
Windows 上借 IOCP(I/O 完成端口)之力,Linux 上以线程池模拟异步,实现真正无阻塞的文件操作。

📸 核心特性

特性 说明
🍃 零线程开销 无需 run_in_executor,无后台线程 (Windows)
📰 内核级完成 无用户态调度延迟,直达内核
高并发友好 数千并发文件操作,文文团扇一挥间
🎴 标准 API 熟悉的文件接口,支持 async/await
📖 文本二进制支持 文本模式自动编解码
🔧 可配置句柄池 根据工作负载调整性能 (Windows)
🌍 跨平台 Windows / Linux 皆可翱翔

🛠️ 安装

pip install ayafileio

系统要求:

  • Python 3.10+(幻想郷的科技也要与时俱进)
  • Windows 7 / Server 2008 R2 或更高版本,或 Linux
  • 无其他依赖(自带团扇,无需外援)

🚀 快速开始

import asyncio
import ayafileio

async def main():
    # 写入文件——像风一样快
    async with ayafileio.open("example.txt", "w") as f:
        await f.write("Hello, async world!\n")
        await f.flush()

    # 读取并自动解码——文文新闻,一触即达
    async with ayafileio.open("example.txt", "r", encoding="utf-8") as f:
        content = await f.read()
        print(content)  # "Hello, async world!\n"

    # 二进制操作——数据如风,来去无痕
    async with ayafileio.open("data.bin", "rb") as f:
        data = await f.read(1024)
        await f.seek(0, 0)  # 风神少女,回首即原点

asyncio.run(main())

⚙️ 高级配置

句柄池调优(Windows)

高并发工作负载时,调整句柄池大小,让文件句柄如天狗之羽,收放自如:

import ayafileio

# 查看当前限制
max_per_key, max_total = ayafileio.get_handle_pool_limits()
print(f"当前: 每键 {max_per_key},总计 {max_total}")

# 增加以提升多文件性能——风势加强!
ayafileio.set_handle_pool_limits(128, 4096)

这会在打开/关闭周期中重用文件句柄,减少昂贵的 CreateFile 调用——毕竟,天狗从不做无谓的起落。

I/O 工作线程数(跨平台)

# 设置 I/O worker 数量(Windows 上为 IOCP 线程数)
ayafileio.set_io_worker_count(8)  # 0 = 自动,1-128 = 手动
# Linux 上调用此方法也不会报错——优雅降级,是风神的风格

📚 API 参考

AsyncFile 类

class AsyncFile:
    def __init__(self, path: str | Path, mode: str = "rb", encoding: str | None = None): ...
    async def read(self, size: int = -1) -> str | bytes: ...
    async def write(self, data: str | bytes) -> int: ...
    async def seek(self, offset: int, whence: int = 0) -> int: ...
    async def flush(self) -> None: ...
    async def close(self) -> None: ...
    async def readline(self) -> str | bytes: ...
    def __aiter__(self) -> AsyncFile: ...
    async def __anext__(self) -> str | bytes: ...

支持的模式

模式 说明
"r", "rb" 读取(文本/二进制)
"w", "wb" 写入(文本/二进制)
"a", "ab" 追加(文本/二进制)
"x", "xb" 独占创建(文本/二进制)
加上 "+" 读写组合,风神双持

函数

def set_handle_pool_limits(max_per_key: int, max_total: int) -> None: ...
def get_handle_pool_limits() -> tuple[int, int]: ...
def set_io_worker_count(count: int = 0) -> None: ...  # 跨平台通用

🧪 性能对比

在同等环境下(测试时长 10 秒,随机读写混合),ayafileio 展现出了天狗般的速度:

并发数 ayafileio aiofiles 优势
50 2,043 ops/s 1,657 ops/s +23%
100 770 ops/s 405 ops/s +90%
500 1,130 ops/s 1,032 ops/s +9.5%

P99 延迟对比(越低越好):

  • 100 并发:ayafileio 33ms vs aiofiles 562ms
  • 200 并发:ayafileio 35ms vs aiofiles 155ms

数据基于 Windows 10 + 机械硬盘(C5=12, C6=45)实测——即使是即将报废的硬件,文文依然能跑出速度。

运行基准测试:

git clone https://github.com/your-repo/ayafileio.git
cd ayafileio
python run_benchmark.py

🤝 贡献

欢迎投稿「文文。新闻」!请:

  1. Fork 本仓库
  2. 创建功能分支(git checkout -b feature/amazing-feature
  3. 添加测试(天狗也要认真对待)
  4. 确保基准测试仍通过
  5. 提交拉取请求

📄 许可证

MIT 许可证 —— 最速最自由,详见 LICENSE

🙏 致谢

  • 感谢 Windows IOCP 提供的真异步之力
  • 感谢射命丸文带来的「最速」精神
  • 感谢所有在坏盘上测试的勇者们

「遅いのは罪だぜ?」
—— 射命丸文,『文文。新闻』主编

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ayafileio-0.1.7.post1-cp314-cp314t-musllinux_1_2_x86_64.whl (536.8 kB view details)

Uploaded CPython 3.14tmusllinux: musl 1.2+ x86-64

ayafileio-0.1.7.post1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (90.4 kB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.17+ x86-64

ayafileio-0.1.7.post1-cp310-abi3-win_amd64.whl (68.1 kB view details)

Uploaded CPython 3.10+Windows x86-64

ayafileio-0.1.7.post1-cp310-abi3-musllinux_1_2_x86_64.whl (535.8 kB view details)

Uploaded CPython 3.10+musllinux: musl 1.2+ x86-64

ayafileio-0.1.7.post1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (89.6 kB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ayafileio-0.1.7.post1-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89.7 kB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ayafileio-0.1.7.post1-cp310-abi3-macosx_15_0_arm64.whl (55.6 kB view details)

Uploaded CPython 3.10+macOS 15.0+ ARM64

File details

Details for the file ayafileio-0.1.7.post1-cp314-cp314t-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-0.1.7.post1-cp314-cp314t-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 efb85309e62af01e8a1440e3232d7c5609fcbc3c5790a33cdf5be1013b20d1d8
MD5 2dfcea0271beaa622e566e4c3cf684b1
BLAKE2b-256 12f22d06c0f1e3394f72e365e968b5a9c467c655f6ba104988a5f0c782a34df6

See more details on using hashes here.

File details

Details for the file ayafileio-0.1.7.post1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-0.1.7.post1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 f3f053e34057d5c0b484df7ace697564da792a8c02ceae84f7d9a05162731dcf
MD5 487b9536cab9e9c21d2a5c919e3ba16d
BLAKE2b-256 d9f640ff3be4690ee6e7eb3f87420eeef6a050712162570a9fe1980890263a1c

See more details on using hashes here.

File details

Details for the file ayafileio-0.1.7.post1-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for ayafileio-0.1.7.post1-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 727946d38434f9aeb34df01740da8b12d9d03789560d0ca071bcb75f4f8e7e08
MD5 9ae7c203ea486d311b7cfba756a32ed4
BLAKE2b-256 4e131d7bf996c8b092f9b893cc304feae6a08694f9a96ba47ea9cf0932751a3e

See more details on using hashes here.

File details

Details for the file ayafileio-0.1.7.post1-cp310-abi3-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-0.1.7.post1-cp310-abi3-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 a7617c7571168883759fcabbbb6af3b0d5b9e65ae605d092235041f861ce4232
MD5 16c3a747994d4c47f7314b12275941da
BLAKE2b-256 6e7ef28440bf4e12782239f96b748d7e146cb721ee2f8a40e0974850e6443828

See more details on using hashes here.

File details

Details for the file ayafileio-0.1.7.post1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-0.1.7.post1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b0780056b728ca237c74d0b31ec6deb2e9221915b2fe7655c41017086a1e9231
MD5 b387709860ef5358c71adcddb7178a45
BLAKE2b-256 c51bdc5b14345bea5aa53ea9b46cb1171c9a7eb375ff4fb08446ffff90b8ca0c

See more details on using hashes here.

File details

Details for the file ayafileio-0.1.7.post1-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for ayafileio-0.1.7.post1-cp310-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 3212ab9534e25d2b2be5ed863e0116308ade5fab8a9fe46c87c41f9e574d276f
MD5 6aa910e6673e8180fab946c1a3bad7a7
BLAKE2b-256 549e22136bbd4fd8e6192d18eb96211d808b5bd7ec3a480d50f497f88f31b744

See more details on using hashes here.

File details

Details for the file ayafileio-0.1.7.post1-cp310-abi3-macosx_15_0_arm64.whl.

File metadata

File hashes

Hashes for ayafileio-0.1.7.post1-cp310-abi3-macosx_15_0_arm64.whl
Algorithm Hash digest
SHA256 942b163ac75737afff23db4855059cb231cf4748841c508aec8a3ebfa704af0a
MD5 265ad5e5af0f31a9bb85305143f56167
BLAKE2b-256 c344f5c17c7c15d846884f568536dcd7eaaa8c88c2fdc6e3ea39654347634593

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page