Skip to main content

FlatFS

Minimal filesystem abstraction with pluggable backends, treating paths as keys and file contents as values.

About

FlatFS provides minimalistic protocol-based API for accessing files. The API allows you to:

  • create files using generator of binary chunks as source of data
  • read files in chunks
  • checking if file exists
  • removing file from the filesystem
  • scanning filesystem to retrieve all the files that are available

The API from above is supplied with a rich set of helpers that can simplify common actions, like writing/reading whole files in text or binary mode, reading files line-by-line, reading exact amount of bytes etc.

FlatFS does not use directories; those are just used to organize paths (aka keys) and some implementations may use real directories, and the other may not.

The library is implemented in a way allowing it to be used with large files thanks to the generator-based core API.

Installation

The project can be installed directly from PyPI:

$ pip install flatfs

You can alternatively use any of the existing Python package management tools (e.g. Poetry or uv) to install the library directly to your project.

Examples

Here are some practical examples of how this library can be used:

  1. Access /tmp directory:

    import pathlib
    
    from flatfs.api import LocalFlatFs, FlatFsReaderWriter, write_text, read_text
    
    def make_temp_fs() -> FlatFsReaderWriter:
        return LocalFlatFs(pathlib.Path("/tmp"))
    
    temp_fs = make_temp_fs()
    write_text(temp_fs, "/foo/bar.txt", "content of file")  # Creates text file at /tmp/foo/bar.txt
    content = read_text(temp_fs, "/foo/bar.txt")  # Read the content back
    
  2. Create in-memory flat filesystem (this implementation can be used during testing):

    from flatfs.api import InMemoryFlatFs
    
    fs = InMemoryFlatFs()
    fs.write_chunks("foo.txt", [b"spam"])
    paths = list(fs.scan())  # Will return: ["/foo.txt"]
    
  3. Wrap flat filesystem created earlier with async interface:

    from flatfs.api import AsyncFlatFsAdapter, async_read_text
    
    async_temp_fs = AsyncFlatFsAdapter(temp_fs)
    content = await async_read_text(async_temp_fs, "foo/bar.txt")  # Paths are normalized and leading / is automatically added
    print(content)  # Would print: content of file
    

Rationale

During developing of various systems and tools that follow clean architecture and layered design (business logic, use case interactors, data ports and gateways etc.) I found myself writing similar tool as one of the port/gateway pair over and over again. This library was created to have such toolkit a bit more advanced, based on chunk generators and producers, backed up with comprehensive set of helpers, just to avoid writing it again for the (N+1)th time.

Author

Maciej Wiatrzyk maciej.wiatrzyk@gmail.com

License

This project is released under the terms of the MIT license.

See LICENSE.txt for more details.

Metadata

Release files for flatfs 0.7.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for flatfs 0.7.1
File Size Uploaded Provenance Actions
flatfs-0.7.1.tar.gz 11.8 kB Without provenance Details for flatfs-0.7.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for flatfs 0.7.1
File Interpreter ABI Platform Actions
flatfs-0.7.1-py3-none-any.whl Python 3 none any Details for flatfs-0.7.1-py3-none-any.whl

Total release size: 27.8 kB

Release files / flatfs-0.7.1.tar.gz

Download URL flatfs-0.7.1.tar.gz
Size 11.8 kB
Tags Source
SHA-256 checksum
How to use checksums
7cf649849bd02fff0f26ea70eb95420f369fdab7eb4b4eb3ff59b73844e9c7b6
BLAKE2b-256 checksum
How to use checksums
e22a8b2a45e8eb6bb1d977347994efadd65722be9409130a57f7addf3aa4d624
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.5.1 CPython/3.13.18 Linux/7.0.0-1016-aws

Release files / flatfs-0.7.1-py3-none-any.whl

Download URL flatfs-0.7.1-py3-none-any.whl
Size 15.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d9ffe73f45a3b4e71c83ee9c957c3d1d4ae067965f8f8c076418e287327f2ef5
BLAKE2b-256 checksum
How to use checksums
706564c464fbe15f24dd8323c4713fac7e1c068ad8b305019481a862a437e811
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.5.1 CPython/3.13.18 Linux/7.0.0-1016-aws

Security for flatfs 0.7.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page