DataLad FUSE extension package
Project description
DataLad FUSE extension package
datalad-fuse
provides commands for reading files in a
DataLad dataset from their remote web URLs without having
to download them in their entirety first. Instead,
fsspec is used to sparsely download
and locally cache the files as needed.
Installation
Current version of datalad-fuse
requires Python 3.7 or higher. Just use
pip for Python 3 (You have pip, right?) to install it:
python3 -m pip install datalad-fuse
In addition, use of the datalad fusefs
command requires FUSE to be installed;
on Debian-based systems, this can be done with:
sudo apt-get install fuse
Commands
datalad fsspec-cache-clear [<options>]
Clears the local download cache for a dataset.
Options
-
-d <DATASET>
,--dataset <DATASET>
— Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory. -
-r
,--recursive
— Clear the caches of subdatasets as well.
datalad fsspec-head [<options>] <path>
Shows leading lines/bytes of an annexed file by fetching its data from a remote URL.
Options
-
-d <DATASET>
,--dataset <DATASET>
— Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory. -
-n <INT>
,--lines <INT>
— How many lines to show (default: 10) -
-c <INT>
,--bytes <INT>
— How many bytes to show
datalad fusefs [<options>] <mount-path>
Create a read-only FUSE mount at <mount-path>
that exposes the files in the
given dataset. Opening a file under the mount that is not locally present in
the dataset will cause its contents to be downloaded from the file's web URL as
needed.
When the command finishes, fsspec-cache-clear
may be run depending on the
value of the datalad.fusefs.cache-clear
configuration option. If it is set
to "visited
", then any (sub)datasets that were accessed in the FUSE mount
will have their caches cleared; if it is instead set to "recursive
", then all
(sub)datasets in the dataset being operated on will have their caches cleared.
Options
-
--allow-other
— Allow all users to access files in the mount. This requires settinguser_allow_other
in/etc/fuse.conf
. -
-d <DATASET>
,--dataset <DATASET>
— Specify the dataset to operate on. If no dataset is given, an attempt is made to identify the dataset based on the current working directory. -
-f
,--foreground
— Run the FUSE process in the foreground; use Ctrl-C to exit. This option is currently required. -
--mode-transparent
— Expose the dataset's.git
directory in the mount
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file datalad-fuse-0.4.2.tar.gz
.
File metadata
- Download URL: datalad-fuse-0.4.2.tar.gz
- Upload date:
- Size: 53.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ab6dc76ff66491c4d74a84cca19b99e6efae2c793e6f2dfaaa594f6d04add878 |
|
MD5 | 686e42d0544b98b98181bb2a4248ef1f |
|
BLAKE2b-256 | 1548a10c34dd5053f2b059fa0ed301962833e7f4a3784b507c5d5940d7c5b0c7 |
File details
Details for the file datalad_fuse-0.4.2-py3-none-any.whl
.
File metadata
- Download URL: datalad_fuse-0.4.2-py3-none-any.whl
- Upload date:
- Size: 26.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.11.2
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 988ccf6d9a74eaeaf39cf1ed29d403ad7a1540daf1f0df1bb5a18597858dfafe |
|
MD5 | cd91bc87b568468ede318307ed4220c2 |
|
BLAKE2b-256 | 2d501ad7761f078a819a69e1d03e87282832f34a150170415240388a6dafbd17 |