fsspec-rs

fsspec-compatible filesystem backends with Rust acceleration.

Overview

fsspec-rs provides Python filesystem classes that inherit from fsspec.spec.AbstractFileSystem while delegating core operations to Rust. They can be used anywhere fsspec filesystems are accepted, including pandas, dask, xarray, and direct fsspec.open() calls.

The package currently includes:

Install

pip install fsspec-rs

Quick start

import fsspec
from fsspec_rs import LocalFileSystem
from fsspec_rs import S3FileSystem

fs = LocalFileSystem()
fs.pipe_file("/tmp/example.txt", b"hello")
print(fs.cat_file("/tmp/example.txt"))

s3 = S3FileSystem(bucket="my-bucket", region="us-east-1")
s3.pipe_file("path/to/output.txt", b"hello")

with fsspec.open("file-rs:///tmp/example.txt", "rb") as f:
    print(f.read())

Documentation

• Filesystems: local and S3 usage, configuration, caching, and path conventions.
• API reference: Python API generated with yardang/Sphinx.
• Rust traits: Rust FileSystem, experimental AsyncFileSystem, FsFile, and cache traits.

Build docs locally with:

yardang build