Skip to main content

DataLad FUSE extension package

Project description

DataLad FUSE extension package

codecov.io tests docs

datalad-fuse provides commands for reading files in a DataLad dataset from their remote web URLs without having to download them in their entirety first. Instead, fsspec is used to sparsely download and locally cache the files as needed.

Installation

Current version of datalad-fuse requires Python 3.7 or higher. Just use pip for Python 3 (You have pip, right?) to install it:

python3 -m pip install datalad-fuse

In addition, use of the datalad fusefs command requires FUSE to be installed; on Debian-based systems, this can be done with:

sudo apt-get install fuse

Commands

datalad fsspec-cache-clear [<options>]

Clears the local download cache for a dataset.

Options

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -r, --recursive — Clear the caches of subdatasets as well.

datalad fsspec-head [<options>] <path>

Shows leading lines/bytes of an annexed file by fetching its data from a remote URL.

Options

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -n <INT>, --lines <INT> — How many lines to show (default: 10)

  • -c <INT>, --bytes <INT> — How many bytes to show

datalad fusefs [<options>] <mount-path>

Create a read-only FUSE mount at <mount-path> that exposes the files in the given dataset. Opening a file under the mount that is not locally present in the dataset will cause its contents to be downloaded from the file's web URL as needed.

When the command finishes, fsspec-cache-clear may be run depending on the value of the datalad.fusefs.cache-clear configuration option. If it is set to "visited", then any (sub)datasets that were accessed in the FUSE mount will have their caches cleared; if it is instead set to "recursive", then all (sub)datasets in the dataset being operated on will have their caches cleared.

Options

  • --allow-other — Allow all users to access files in the mount. This requires setting user_allow_other in /etc/fuse.conf.

  • -d <DATASET>, --dataset <DATASET> — Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory.

  • -f, --foreground — Run the FUSE process in the foreground; use Ctrl-C to exit. This option is currently required.

  • --mode-transparent — Expose the dataset's .git directory in the mount

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

datalad-fuse-0.4.2.tar.gz (53.8 kB view details)

Uploaded Source

Built Distribution

datalad_fuse-0.4.2-py3-none-any.whl (26.4 kB view details)

Uploaded Python 3

File details

Details for the file datalad-fuse-0.4.2.tar.gz.

File metadata

  • Download URL: datalad-fuse-0.4.2.tar.gz
  • Upload date:
  • Size: 53.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.2

File hashes

Hashes for datalad-fuse-0.4.2.tar.gz
Algorithm Hash digest
SHA256 ab6dc76ff66491c4d74a84cca19b99e6efae2c793e6f2dfaaa594f6d04add878
MD5 686e42d0544b98b98181bb2a4248ef1f
BLAKE2b-256 1548a10c34dd5053f2b059fa0ed301962833e7f4a3784b507c5d5940d7c5b0c7

See more details on using hashes here.

File details

Details for the file datalad_fuse-0.4.2-py3-none-any.whl.

File metadata

  • Download URL: datalad_fuse-0.4.2-py3-none-any.whl
  • Upload date:
  • Size: 26.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.2

File hashes

Hashes for datalad_fuse-0.4.2-py3-none-any.whl
Algorithm Hash digest
SHA256 988ccf6d9a74eaeaf39cf1ed29d403ad7a1540daf1f0df1bb5a18597858dfafe
MD5 cd91bc87b568468ede318307ed4220c2
BLAKE2b-256 2d501ad7761f078a819a69e1d03e87282832f34a150170415240388a6dafbd17

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page