Skip to main content

DataLad FUSE extension package

Project description

DataLad FUSE extension package

codecov.io tests docs

datalad-fuse provides commands for reading files in a DataLad dataset from their remote web URLs without having to download them in their entirety first. Instead, fsspec is used to sparsely download and locally cache the files as needed.

Installation

Current version of datalad-fuse requires Python 3.7 or higher. Just use pip for Python 3 (You have pip, right?) to install it:

python3 -m pip install datalad-fuse

In addition, use of the datalad fusefs command requires FUSE to be installed; on Debian-based systems, this can be done with:

sudo apt-get install fuse

Commands

datalad fsspec-cache-clear [<options>]

Clears the local download cache for a dataset.

Options

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -r, --recursive — Clear the caches of subdatasets as well.

datalad fsspec-head [<options>] <path>

Shows leading lines/bytes of an annexed file by fetching its data from a remote URL.

Options

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -n <INT>, --lines <INT> — How many lines to show (default: 10)

  • -c <INT>, --bytes <INT> — How many bytes to show

datalad fusefs [<options>] <mount-path>

Create a read-only FUSE mount at <mount-path> that exposes the files in the given dataset. Opening a file under the mount that is not locally present in the dataset will cause its contents to be downloaded from the file's web URL as needed.

When the command finishes, fsspec-cache-clear may be run depending on the value of the datalad.fusefs.cache-clear configuration option. If it is set to "visited", then any (sub)datasets that were accessed in the FUSE mount will have their caches cleared; if it is instead set to "recursive", then all (sub)datasets in the dataset being operated on will have their caches cleared.

Options

  • --allow-other — Allow all users to access files in the mount. This requires setting user_allow_other in /etc/fuse.conf.

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -f, --foreground — Run the FUSE process in the foreground; use Ctrl-C to exit. This option is currently required.

  • --mode-transparent — Expose the dataset's .git directory in the mount

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datalad-fuse-0.5.1.tar.gz (57.7 kB view details)

Uploaded Source

Built Distribution

datalad_fuse-0.5.1-py3-none-any.whl (26.7 kB view details)

Uploaded Python 3

File details

Details for the file datalad-fuse-0.5.1.tar.gz.

File metadata

  • Download URL: datalad-fuse-0.5.1.tar.gz
  • Upload date:
  • Size: 57.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for datalad-fuse-0.5.1.tar.gz
Algorithm Hash digest
SHA256 ea46178e37e0b981db780a0288c829ac5099473ba36e422f2c3662a6860d4881
MD5 fe829912df06b67ab597261c2db02dc0
BLAKE2b-256 baab1f1fc336a0df1320ee047644729a209a9315d05af46940fba074d21e0927

See more details on using hashes here.

Provenance

File details

Details for the file datalad_fuse-0.5.1-py3-none-any.whl.

File metadata

  • Download URL: datalad_fuse-0.5.1-py3-none-any.whl
  • Upload date:
  • Size: 26.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for datalad_fuse-0.5.1-py3-none-any.whl
Algorithm Hash digest
SHA256 a2bd234646a665fd54211eaf7125b7934005ef95aec882da3a8aa9cbe74c1faa
MD5 9cbe0b23b0a14355fdda424418c23999
BLAKE2b-256 8c736749bde486cf7450a82b0f97d1fa9a840da407550c8b0529c53dc7d0724d

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page