rapfiles 0.2.0

True async filesystem I/O — no fake async, no GIL stalls.

rapfiles

True async filesystem I/O — no fake async, no GIL stalls.

Overview

rapfiles provides true async filesystem I/O operations for Python, backed by Rust and Tokio. Unlike libraries that wrap blocking I/O in async syntax, rapfiles guarantees that all I/O work executes outside the Python GIL, ensuring event loops never stall under load.

Roadmap Goal: Achieve drop-in replacement compatibility with aiofiles, enabling seamless migration with true async performance. See docs/ROADMAP.md for details.

Why rap*?

Packages prefixed with rap stand for Real Async Python. Unlike many libraries that merely wrap blocking I/O in async syntax, rap* packages guarantee that all I/O work is executed outside the Python GIL using native runtimes (primarily Rust). This means event loops are never stalled by hidden thread pools, blocking syscalls, or cooperative yielding tricks. If a rap* API is async, it is structurally non-blocking by design, not by convention. The rap prefix is a contract: measurable concurrency, real parallelism, and verifiable async behavior under load.

See the rap-manifesto for philosophy and guarantees.

Features

• ✅ True async file reads and writes
• ✅ Native Rust-backed execution (Tokio)
• ✅ Zero Python thread pools
• ✅ Event-loop-safe concurrency under load
• ✅ GIL-independent I/O operations
• ✅ Verified by Fake Async Detector
• ✅ File handles with async context managers (async with)
• ✅ Directory operations (create, remove, list, walk)
• ✅ File metadata (stat, size, timestamps)
• ✅ Path operations (rapfiles.ospath module)
• ✅ File manipulation (copy, move, rename, remove, links, canonicalize)
• ✅ Atomic operations (atomic writes and moves)
• ✅ File locking (shared/exclusive advisory locks)
• ✅ Batch operations (concurrent read/write/copy of multiple files)
• ✅ aiofiles compatibility (Phase 1 complete)

Requirements

• Python 3.8+ (including Python 3.13 and 3.14)
• Rust 1.70+ (for building from source)

Installation

pip install rapfiles

Building from Source

git clone https://github.com/eddiethedean/rapfiles.git
cd rapfiles
pip install maturin
maturin develop

---

Usage

Basic File Operations

import asyncio
from rapfiles import read_file, write_file

async def main():
    # Write file asynchronously (true async, GIL-independent)
    await write_file("example.txt", "Hello from rapfiles!")
    
    # Read file asynchronously (true async, GIL-independent)
    content = await read_file("example.txt")
    print(content)  # Output: Hello from rapfiles!
    
    # Write another file
    await write_file("output.txt", content)

asyncio.run(main())

File Handles (aiofiles compatible)

import asyncio
from rapfiles import open

async def main():
    # Open file with async context manager
    async with open("file.txt", "r") as f:
        content = await f.read()
        print(content)
    
    # Write mode
    async with open("output.txt", "w") as f:
        await f.write("Hello, world!")
    
    # Binary mode
    async with open("image.png", "rb") as f:
        data = await f.read()
    
    # Read lines
    async with open("file.txt", "r") as f:
        line = await f.readline()
        lines = await f.readlines()

asyncio.run(main())

Directory Operations

import asyncio
from rapfiles import create_dir, list_dir, exists, is_file, is_dir, walk_dir

async def main():
    # Create directories
    await create_dir("new_dir")
    await create_dir_all("path/to/nested/dir")
    
    # Check if path exists
    if await exists("file.txt"):
        print("File exists!")
    
    # Check file/directory type
    if await is_file("file.txt"):
        print("It's a file")
    if await is_dir("directory"):
        print("It's a directory")
    
    # List directory contents
    files = await list_dir(".")
    print(files)
    
    # Recursively walk directory
    for path, is_file in await walk_dir("."):
        print(f"{path}: {'file' if is_file else 'dir'}")

asyncio.run(main())

File Metadata

import asyncio
from rapfiles import stat, FileMetadata

async def main():
    # Get file statistics
    metadata: FileMetadata = await stat("file.txt")
    print(f"Size: {metadata.size} bytes")
    print(f"Is file: {metadata.is_file}")
    print(f"Modified: {metadata.modified}")
    print(f"Created: {metadata.created}")

asyncio.run(main())

Concurrent File Operations

import asyncio
from rapfiles import read_file, write_file

async def main():
    # Process multiple files concurrently
    tasks = [
        write_file("file1.txt", "Content 1"),
        write_file("file2.txt", "Content 2"),
        write_file("file3.txt", "Content 3"),
    ]
    await asyncio.gather(*tasks)
    
    # Read all files concurrently
    contents = await asyncio.gather(
        read_file("file1.txt"),
        read_file("file2.txt"),
        read_file("file3.txt"),
    )
    print(contents)  # ['Content 1', 'Content 2', 'Content 3']

asyncio.run(main())

File Manipulation Operations

import asyncio
from rapfiles import copy_file, move_file, remove_file, symlink, canonicalize

async def main():
    # Copy files
    await copy_file("source.txt", "destination.txt")
    
    # Move/rename files
    await move_file("old_name.txt", "new_name.txt")
    await rename("old.txt", "new.txt")  # Alias for move_file
    
    # Remove files
    await remove_file("unwanted.txt")
    
    # Create symbolic links
    await symlink("/path/to/original", "/path/to/link")
    
    # Resolve paths (canonicalize)
    abs_path = await canonicalize("./relative/path/../file.txt")
    print(abs_path)  # /absolute/path/to/file.txt

asyncio.run(main())

Atomic File Operations

import asyncio
from rapfiles import atomic_write_file, atomic_move_file

async def main():
    # Atomic write ensures file is never partially written
    await atomic_write_file("important.txt", "Critical data")
    
    # Atomic move ensures destination is never in partial state
    await atomic_move_file("source.txt", "destination.txt")

asyncio.run(main())

File Locking

import asyncio
from rapfiles import lock_file, lock_file_shared

async def main():
    # Exclusive lock for writing
    async with lock_file("data.txt", exclusive=True) as lock:
        await write_file("data.txt", "Exclusive access")
    # Lock automatically released
    
    # Shared lock for reading
    async with lock_file_shared("data.txt") as lock:
        content = await read_file("data.txt")
        # Multiple readers can hold shared locks simultaneously

asyncio.run(main())

Batch Operations

import asyncio
from rapfiles import read_files, write_files, copy_files

async def main():
    # Write multiple files concurrently
    files = {
        "file1.txt": b"Content 1",
        "file2.txt": b"Content 2",
        "file3.txt": b"Content 3",
    }
    await write_files(files)
    
    # Read multiple files concurrently
    paths = ["file1.txt", "file2.txt", "file3.txt"]
    results = await read_files(paths)  # Returns [(path, bytes), ...]
    
    # Or as a dictionary
    content_dict = await read_files_dict(paths)  # Returns {path: bytes}
    
    # Copy multiple files concurrently
    copy_pairs = [
        ("src1.txt", "dst1.txt"),
        ("src2.txt", "dst2.txt"),
    ]
    await copy_files(copy_pairs)

asyncio.run(main())

API Reference

File Operations

read_file(path: str) -> str

Read a file asynchronously and return its contents as a string.

Parameters:

• path (str): Path to the file to read

Returns:

• str: File contents as UTF-8 decoded string

Raises:

• FileNotFoundError: If the file does not exist
• IOError: If the file cannot be read
• ValueError: If the path is invalid

write_file(path: str, contents: str) -> None

Write content to a file asynchronously.

Parameters:

• path (str): Path to the file to write
• contents (str): Content to write to the file

Raises:

• IOError: If the file cannot be written
• PermissionError: If write permission is denied
• ValueError: If the path is invalid

read_file_bytes(path: str) -> bytes

Read a file asynchronously and return its contents as bytes.

Parameters:

• path (str): Path to the file to read

Returns:

• bytes: File contents as raw bytes

Raises:

• FileNotFoundError: If the file does not exist
• IOError: If the file cannot be read

write_file_bytes(path: str, contents: bytes) -> None

Write bytes to a file asynchronously.

Parameters:

• path (str): Path to the file to write
• contents (bytes): Bytes to write to the file

Raises:

• IOError: If the file cannot be written
• PermissionError: If write permission is denied

append_file(path: str, contents: str) -> None

Append content to a file asynchronously.

Parameters:

• path (str): Path to the file to append to
• contents (str): Content to append to the file

Raises:

• IOError: If the file cannot be written
• PermissionError: If write permission is denied

File Handles

open(file: Union[str, bytes], mode: str = "r", ...) -> AsyncFile

Open a file asynchronously (aiofiles.open() compatible).

Parameters:

• file (Union[str, bytes]): Path to the file
• mode (str): File mode (r, r+, w, w+, a, a+, rb, rb+, wb, wb+, ab, ab+)
• buffering (int): Buffer size (accepted for compatibility, not yet implemented)
• encoding (Optional[str]): Text encoding (accepted for compatibility, not yet implemented)
• errors (Optional[str]): Error handling (accepted for compatibility, not yet implemented)
• newline (Optional[str]): Newline handling (accepted for compatibility, not yet implemented)
• closefd (bool): Close file descriptor (accepted for compatibility, not yet implemented)
• opener (Optional[Any]): Custom opener (accepted for compatibility, not yet implemented)

Returns:

• Async context manager that yields an AsyncFile instance

Example:

async with open("file.txt", "r") as f:
    content = await f.read()

AsyncFile Class

An async file handle for true async I/O operations.

Methods:

• read(size: int = -1) -> Union[str, bytes]: Read from file (returns str for text mode, bytes for binary)
• write(data: Union[str, bytes]) -> int: Write to file, returns number of bytes written
• readline(size: int = -1) -> Union[str, bytes]: Read a single line
• readlines(hint: int = -1) -> List[Union[str, bytes]]: Read all lines
• seek(offset: int, whence: int = 0) -> int: Seek to position (0=start, 1=current, 2=end)
• tell() -> int: Get current file position
• close() -> None: Close the file (automatic on context exit)

Directory Operations

create_dir(path: str) -> None

Create a directory asynchronously. Parent directories must exist.

Raises:

• FileExistsError: If the directory already exists
• IOError: If the directory cannot be created

create_dir_all(path: str) -> None

Create a directory and all parent directories asynchronously.

Raises:

• IOError: If the directory cannot be created

remove_dir(path: str) -> None

Remove an empty directory asynchronously.

Raises:

• FileNotFoundError: If the directory does not exist
• IOError: If the directory is not empty or cannot be removed

remove_dir_all(path: str) -> None

Remove a directory and all its contents asynchronously.

Raises:

• FileNotFoundError: If the directory does not exist
• IOError: If the directory cannot be removed

list_dir(path: str) -> List[str]

List directory contents asynchronously.

Returns:

• List[str]: List of file and directory names

Raises:

• FileNotFoundError: If the directory does not exist
• IOError: If the directory cannot be read

exists(path: str) -> bool

Check if a path exists asynchronously.

Returns:

• bool: True if path exists, False otherwise

is_file(path: str) -> bool

Check if a path is a file asynchronously.

Returns:

• bool: True if path is a file, False otherwise

Raises:

• IOError: If the path does not exist

is_dir(path: str) -> bool

Check if a path is a directory asynchronously.

Returns:

• bool: True if path is a directory, False otherwise

Raises:

• IOError: If the path does not exist

walk_dir(path: str) -> List[Tuple[str, bool]]

Recursively walk a directory asynchronously.

Parameters:

• path (str): Directory path to walk

Returns:

• List[Tuple[str, bool]]: List of (path, is_file) tuples for all files and directories found

Raises:

• FileNotFoundError: If the directory does not exist
• IOError: If the directory cannot be read

File Metadata

stat(path: str) -> FileMetadata

Get file statistics asynchronously.

Returns:

• FileMetadata: File metadata object with size, timestamps, and type information

Raises:

• FileNotFoundError: If the path does not exist
• IOError: If metadata cannot be retrieved

metadata(path: str) -> FileMetadata

Get file metadata asynchronously (alias for stat).

FileMetadata Class

File metadata structure (aiofiles.stat_result compatible).

Properties:

• size (int): File size in bytes
• is_file (bool): True if path is a file
• is_dir (bool): True if path is a directory
• modified (float): Modification time as Unix timestamp
• accessed (float): Access time as Unix timestamp
• created (float): Creation time as Unix timestamp

File Manipulation Operations

copy_file(src: str, dst: str) -> None

Copy a file asynchronously from source to destination.

move_file(src: str, dst: str) -> None

Move or rename a file asynchronously (atomic within same filesystem).

rename(src: str, dst: str) -> None

Rename a file asynchronously (alias for move_file).

remove_file(path: str) -> None

Remove a file asynchronously.

hard_link(src: str, dst: str) -> None

Create a hard link asynchronously.

symlink(src: str, dst: str) -> None

Create a symbolic link asynchronously.

canonicalize(path: str) -> str

Resolve all symbolic links and return the canonical absolute path.

Atomic Operations

atomic_write_file(path: str, contents: str) -> None

Write a file atomically using a temporary file, ensuring the target is never partially written.

atomic_write_file_bytes(path: str, contents: bytes) -> None

Write bytes to a file atomically.

atomic_move_file(src: str, dst: str) -> None

Move a file atomically.

File Locking

lock_file(path: str, exclusive: bool = True) -> FileLock

Acquire an advisory file lock (exclusive or shared) as an async context manager.

lock_file_shared(path: str) -> FileLock

Acquire a shared (read) file lock as an async context manager.

FileLock Class

File lock object returned by lock_file() and lock_file_shared(). Use with async with syntax.

Methods:

• release() -> None: Manually release the lock

Batch Operations

read_files(paths: List[str]) -> List[Tuple[str, bytes]]

Read multiple files concurrently. Returns list of (path, bytes) tuples.

read_files_dict(paths: List[str]) -> Dict[str, bytes]

Read multiple files concurrently. Returns dictionary mapping paths to file contents.

write_files(files: Dict[str, bytes]) -> None

Write multiple files concurrently. Accepts dictionary of {path: bytes}.

copy_files(files: List[Tuple[str, str]]) -> None

Copy multiple files concurrently. Accepts list of (src_path, dst_path) tuples.

Path Operations

The rapfiles.ospath module provides synchronous path operations compatible with aiofiles.ospath:

• exists(path) -> bool
• isfile(path) -> bool
• isdir(path) -> bool
• getsize(path) -> int
• join(*paths) -> str
• normpath(path) -> str
• abspath(path) -> str
• dirname(path) -> str
• basename(path) -> str
• splitext(path) -> Tuple[str, str]
• split(path) -> Tuple[str, str]

Benchmarks

This package passes the Fake Async Detector. Benchmarks are available in the rap-bench repository.

Run the detector yourself:

pip install rap-bench
rap-bench detect rapfiles

Documentation

Comprehensive documentation is available in the docs/ directory:

• Roadmap - Detailed development plans
• Build and Test - Local development setup
• Release Notes - Version history and changes
• Security - Security policy and reporting

Roadmap

See docs/ROADMAP.md for detailed development plans. Key goals include:

• Drop-in replacement for aiofiles (Phase 1)
• Comprehensive filesystem operations (directories, metadata, permissions)
• Advanced I/O patterns and zero-copy optimizations
• Filesystem traversal and watching capabilities

Related Projects

• rap-manifesto - Philosophy and guarantees
• rap-bench - Fake Async Detector CLI
• rapsqlite - True async SQLite
• rapcsv - Streaming async CSV

Changelog

See CHANGELOG.md for detailed version history.

v0.2.0 (2026-01-17)

Phase 2 Complete - Advanced Filesystem Operations:

• ✅ File manipulation operations (copy, move, rename, remove, links, canonicalize)
• ✅ Atomic operations (atomic writes and moves)
• ✅ File locking (shared/exclusive advisory locks)
• ✅ Batch operations (concurrent read/write/copy of multiple files)
• ✅ 188+ comprehensive tests
• ✅ Full documentation and type stubs

v0.1.1 (2026-01-16)

Python 3.14 Support:

• ✅ Added Python 3.14 classifier to pyproject.toml
• ✅ Updated CI/CD workflows to test for Python 3.14
• ✅ Added ABI3 forward compatibility for Python 3.14

Python 3.13 Support:

• ✅ Added Python 3.13 classifier to pyproject.toml
• ✅ Updated CI/CD workflows to test for Python 3.13

Compatibility:

• Python 3.8 through 3.14 supported
• All platforms: Ubuntu (x86-64, aarch64), macOS (aarch64, x86-64), Windows (x86-64, aarch64)

v0.1.0 (2025-01-12)

Current Status (v0.1.2)

Phase 1 Complete ✅:

• ✅ File handle operations (AsyncFile class with async with support)
• ✅ File operations: read(), write(), readline(), readlines(), seek(), tell()
• ✅ Binary file operations: read_file_bytes(), write_file_bytes()
• ✅ Append operations: append_file()
• ✅ Directory operations: create_dir(), create_dir_all(), remove_dir(), remove_dir_all(), list_dir()
• ✅ Path checking: exists(), is_file(), is_dir()
• ✅ Directory traversal: walk_dir() for recursive directory walking
• ✅ File metadata: stat(), metadata(), FileMetadata class
• ✅ Path operations: rapfiles.ospath module (aiofiles.ospath compatible)
• ✅ aiofiles compatibility: Drop-in replacement for basic aiofiles operations

Phase 2 Complete ✅:

• ✅ File manipulation: copy_file(), move_file(), rename(), remove_file()
• ✅ Link operations: hard_link(), symlink(), canonicalize()
• ✅ Atomic operations: atomic_write_file(), atomic_write_file_bytes(), atomic_move_file()
• ✅ File locking: lock_file(), lock_file_shared() with FileLock class
• ✅ Batch operations: read_files(), write_files(), copy_files() with concurrent execution
• ✅ Comprehensive test suite: 188+ tests covering all Phase 1 and Phase 2 features
• ✅ Type stubs: Complete .pyi files for IDE support
• ✅ Type checking: Full mypy support with Python 3.8+ compatibility
• ✅ Code quality: Ruff formatted and linted, clippy checked

Known Limitations:

• buffering, encoding, errors, newline, closefd, opener parameters accepted for API compatibility but not yet fully implemented
• No streaming operations for large files (planned for Phase 3)
• No file watching capabilities (planned for future phases)
• No advanced I/O patterns like zero-copy (planned for future phases)

Roadmap: See docs/ROADMAP.md for planned improvements. Phase 1 (aiofiles compatibility) and Phase 2 (advanced operations) are complete. Future phases will add streaming operations and advanced optimizations.

Contributing

Contributions are welcome! Please see our contributing guidelines (coming soon).

License

MIT

For detailed release notes, see docs/PYPI_RELEASE_NOTES.md.