A Snakemake storage plugin that reads and writes from a locally mounted filesystem using rsync
Project description
snakemake-storage-plugin-fs
A Snakemake storage plugin that reads and writes from a locally mounted filesystem using rsync. This is particularly useful when running Snakemake on an NFS. Complex parallel IO patterns can slow down NFS quite substantially. The following Snakemake CLI flags allow to avoid such patterns by instructing Snakemake to copy any input to a fast local scatch disk and copying output files back to NFS at the end of a job.
snakemake --default-storage-provider fs --no-shared-fs --local-storage-prefix /local/work/$USER
with /local/work/$USER being the path to the local (non NFS) scratch dir.
Alternatively, these options can be persisted in a profile:
default-storage-provider: fs
no-shared-fs: true
local-storage-prefix: /local/work/$USER
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file snakemake_storage_plugin_fs-0.1.3.tar.gz.
File metadata
- Download URL: snakemake_storage_plugin_fs-0.1.3.tar.gz
- Upload date:
- Size: 3.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.1 CPython/3.11.6 Linux/6.2.0-1016-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2b4073e9f7c3ee826ea57aab6f600dd481ebab76ed46f538d04ea9e15682bedc
|
|
| MD5 |
d8d8f159d529e37026146ac941d6512d
|
|
| BLAKE2b-256 |
b874a9870aee3c8a8da05b9f9d6e5bd84e0cdf046daab8d996515b86e19b4f9b
|
File details
Details for the file snakemake_storage_plugin_fs-0.1.3-py3-none-any.whl.
File metadata
- Download URL: snakemake_storage_plugin_fs-0.1.3-py3-none-any.whl
- Upload date:
- Size: 4.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.7.1 CPython/3.11.6 Linux/6.2.0-1016-azure
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6ee82649e47ce85664b177b2c51e4ae0a6a66578e4d42c6f680fd0ec0142d4da
|
|
| MD5 |
ca303f4205a4ad86303a147be053f3db
|
|
| BLAKE2b-256 |
dc9b6aa4e4fbe65949ebd794926238c8ce481500c2d0bc2fee74b9d50f5d1b76
|
