Skip to main content

FlatFS

Minimal filesystem abstraction with pluggable backends, treating paths as keys and file contents as values.

About

FlatFS provides minimalistic protocol-based API for accessing files. The API allows you to:

  • create files using generator of binary chunks as source of data
  • read files in chunks
  • checking if file exists
  • removing file from the filesystem
  • scanning filesystem to retrieve all the files that are available

The API from above is supplied with a rich set of helpers that can simplify common actions, like writing/reading whole files in text or binary mode, reading files line-by-line, reading exact amount of bytes etc.

FlatFS does not use directories; those are just used to organize paths (aka keys) and some implementations may use real directories, and the other may not.

The library is implemented in a way allowing it to be used with large files thanks to the generator-based core API.

Installation

The project can be installed directly from PyPI:

$ pip install flatfs

You can alternatively use any of the existing Python package management tools (e.g. Poetry or uv) to install the library directly to your project.

Examples

Here are some practical examples of how this library can be used:

  1. Access /tmp directory:

    import pathlib
    
    from flatfs.api import LocalFlatFs, FlatFsReaderWriter, write_text, read_text
    
    def make_temp_fs() -> FlatFsReaderWriter:
        return LocalFlatFs(pathlib.Path("/tmp"))
    
    temp_fs = make_temp_fs()
    write_text(temp_fs, "/foo/bar.txt", "content of file")  # Creates text file at /tmp/foo/bar.txt
    content = read_text(temp_fs, "/foo/bar.txt")  # Read the content back
    
  2. Create in-memory flat filesystem (this implementation can be used during testing):

    from flatfs.api import InMemoryFlatFs
    
    fs = InMemoryFlatFs()
    fs.write_chunks("foo.txt", [b"spam"])
    paths = list(fs.scan())  # Will return: ["/foo.txt"]
    
  3. Wrap flat filesystem created earlier with async interface:

    from flatfs.api import AsyncFlatFsAdapter, async_read_text
    
    async_temp_fs = AsyncFlatFsAdapter(temp_fs)
    content = await async_read_text(async_temp_fs, "foo/bar.txt")  # Paths are normalized and leading / is automatically added
    print(content)  # Would print: content of file
    

Rationale

During developing of various systems and tools that follow clean architecture and layered design (business logic, use case interactors, data ports and gateways etc.) I found myself writing similar tool as one of the port/gateway pair over and over again. This library was created to have such toolkit a bit more advanced, based on chunk generators and producers, backed up with comprehensive set of helpers, just to avoid writing it again for the (N+1)th time.

Author

Maciej Wiatrzyk maciej.wiatrzyk@gmail.com

License

This project is released under the terms of the MIT license.

See LICENSE.txt for more details.

Metadata

Release files for flatfs 0.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for flatfs 0.6.0
File Size Uploaded Provenance Actions
flatfs-0.6.0.tar.gz 11.7 kB Without provenance Details for flatfs-0.6.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for flatfs 0.6.0
File Interpreter ABI Platform Actions
flatfs-0.6.0-py3-none-any.whl Python 3 none any Details for flatfs-0.6.0-py3-none-any.whl

Total release size: 27.5 kB

Release files / flatfs-0.6.0.tar.gz

Download URL flatfs-0.6.0.tar.gz
Size 11.7 kB
Tags Source
SHA-256 checksum
How to use checksums
6948df53719521f8369c996634170414e1eede2d750bc8d4bb66be44a5990309
BLAKE2b-256 checksum
How to use checksums
98604e9b0601770b5f2fb70457f57c501ea9446d0d712c639e5e30f6ff7d199e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.4.1 CPython/3.13.15 Linux/7.0.0-1012-aws

Release files / flatfs-0.6.0-py3-none-any.whl

Download URL flatfs-0.6.0-py3-none-any.whl
Size 15.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0cc3ee19ac2605272232124116ecad856571470e3ffa745d8393fe0c6b082aa6
BLAKE2b-256 checksum
How to use checksums
6d45684ab127b2525953e725bdda43ce69707a7f7a1c500032b8abcbc251f47e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.4.1 CPython/3.13.15 Linux/7.0.0-1012-aws

Security for flatfs 0.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page