py-asyncfiles 1.2.0

High-performance async file I/O library built on libuv

asyncfiles

asyncfiles is a high-performance async file I/O library for Python, built on top of libuv using Cython. It provides true asynchronous file operations without blocking the event loop, offering significant performance improvements over thread-based alternatives.

🚀 Features

• True Async I/O: Built directly on libuv's async filesystem operations
• High Performance: Cython-optimized with zero-copy buffer operations
• Non-blocking: No thread pools - pure async I/O all the way down
• Simple API: Familiar async context manager interface
• Type Hints: Full typing support for better IDE integration
• Memory Efficient: Configurable buffer sizes for optimal memory usage

📦 Installation

From PyPI

pip3 install py-asyncfiles

Requirements

• Python 3.8 or higher
• Cython ~3.0
• A C compiler (gcc, clang, or MSVC)
• libuv (included as submodule)

🔥 Quick Start

import asyncio
from asyncfiles import open

async def main():
    # Read a file
    async with open('example.txt', 'r') as f:
        content = await f.read()
        print(content)
    
    # Write to a file
    async with open('output.txt', 'w') as f:
        await f.write('Hello, async world!')

asyncio.run(main())

📖 Usage Examples

Reading Files

import asyncio
from asyncfiles import open

async def read_file():
    # Read entire file
    async with open('data.txt', 'r') as f:
        content = await f.read()
        return content

asyncio.run(read_file())

Writing Files

import asyncio
from asyncfiles import open

async def write_file():
    async with open('output.txt', 'w') as f:
        await f.write('Line 1\n')
        await f.write('Line 2\n')
        await f.write('Line 3\n')

asyncio.run(write_file())

Custom Buffer Size

import asyncio
from asyncfiles import open

async def large_file():
    # Use 1MB buffer for large files
    async with open('large_file.bin', 'r', buffer_size=1024*1024) as f:
        data = await f.read()
        return data

asyncio.run(large_file())

Multiple Concurrent Operations

import asyncio
from asyncfiles import open

async def process_files():
    # Read multiple files concurrently
    async def read_file(path):
        async with open(path, 'r') as f:
            return await f.read()
    
    results = await asyncio.gather(
        read_file('file1.txt'),
        read_file('file2.txt'),
        read_file('file3.txt'),
    )
    
    return results

asyncio.run(process_files())

🎯 Supported File Modes

Mode	Description
'r'	Read text (default)
'w'	Write text (truncate)
'a'	Append text
'x'	Exclusive creation
'r+'	Read and write
'w+'	Write and read (truncate)
'a+'	Append and read

Additional modifiers:

• 'b': Binary mode
• 't': Text mode (default)

⚡ Performance

asyncfiles significantly outperforms thread-based async file libraries:

Benchmark Results (10MB file):
├── asyncfiles:    45.2 ms  ← 🏆 Fastest
├── aiofile:       78.4 ms
├── aiofiles:      112.6 ms
└── stdlib+thread: 156.3 ms

Run benchmarks yourself:

python -m benchmark.benchmark_read
python -m benchmark.benchmark_write

🛠️ Development

Building from Source

# Clean build
make clean

# Compile with optimizations
make compile

# Debug build with line tracing
make debug

# Run tests
make test

Running Tests

# Run all tests
make test

# Run with async debug mode
PYTHONASYNCIODEBUG=1 python -m unittest discover -v tests

# Test installed package
make testinstalled

Code Style

• Follow PEP 8 for Python code
• Use type hints where applicable
• Add docstrings for public APIs
• Keep Cython code clean and well-commented

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

This project is heavily inspired by uvloop, which pioneered the use of libuv in Python's async ecosystem. The build system, Cython integration patterns, and overall architecture follow uvloop's excellent design.

Special thanks to:

• uvloop team for showing how to properly integrate libuv with Python
• libuv project for the excellent async I/O library
• Cython team for making high-performance Python extensions possible

🔗 Related Projects

• uvloop - Ultra fast asyncio event loop
• aiofiles - Thread-based async file I/O
• aiofile - Another async file library
• libuv - Cross-platform async I/O

📊 Status

Current Status: Experimental / In Development

This project is currently in active development. The API may change. Production use is not recommended yet.

Roadmap

• Iteration support
• Comprehensive test suite
• Complete documentation
• Binary mode support
• Seek/tell operations
• File metadata operations
• PyPI package release
• CI/CD pipeline
• Performance optimizations
• More comprehensive benchmarks

💬 Support

• Issues: GitHub Issues

📈 Benchmarks

For detailed benchmark results and methodology, see benchmark/README.md.

---

Made with ❤️ and ⚡ by Zh00