kadi-fsspec

Expose a kadi4mat collection as fsspec filesystem. Collections and records are represented as directories. Files attached to records are represented as files.

The kadi-apy library is used, in particular all API calls go through a KadiManager instance.

Usage

General

Basic usage, assuming you have a .kadiconfig file (e.g. because you have previously used kadi-apy):

from kadi_fsspec import KadiFileSystem

root_collection_identifier = "@root-collection"

fs = KadiFileSystem(root_collection_identifier)

Alternatively, pass an instance of KadiManager or options to create one, for example:

from kadi_fsspec import KadiFileSystem
from kadi_apy import KadiManager

root_collection_identifier = "@root-collection"

manager = KadiManager(
    host="https://kadi4mat.iam.kit.edu/",
    pat="<your-kadi-access-token>",
)

fs = KadiFileSystem(root_collection_identifier, kadi_manager=manager)

Inside marimo

The original motivation for this package was to allow users of marimo to browse a kadi4mat collection from inside a notebook. If you add a cell with the code shown above, marimo should autodetect the instance of fsspec.AbstractFileSystem and a file tree should appear in the "Remote Storage" section of the "Files" tab of the left sidebar.

Other usage

The package registers the kadi:// protocol in fsspec, so you can also get an instance of a KadiFileSystem filesystem or open a file via the fsspec API:

# Instance of KadiFileSystem
fs = fsspec.filesystem("kadi", root_collection_identifier="@root-collection")

# Open files directly
with fsspec.open("kadi:///some-collection/some-record/some-file.txt", "rb", root_collection_identifier="@root-collection") as f:
    print(f.read().decode())

Status

• Listing records and collections as directories
• Read and download files
• Create new records (parent collection must exist)
• Write to existing files
• Create new files (parent record must exist)
• Partial reads, streaming

License and Copyright

Copyright GFZ Helmholtz Centre for Geosciences. kadi-fsspec is licensed under EUPL-1.2.