git annex support for Ebrains Data Proxy
Project description
annex_dataproxy
This git annex external remote extension uses AnnexRemote to talk to EBRAINS Data Proxy so that you can use EBRAINS Collaboratory buckets as Datalad siblings.
Install
pip install annex_dataproxy
.
Usage
create a dataset
$ datalad create pdfdata
[INFO ] Creating a new annex repo at /tmp/pdfdata
[INFO ] scanning for unlocked files (this may take some time)
create(ok): /tmp/pdfdata (dataset)
$ cd pdfdata/
$ rsync -ra $HOME/PDFs/ ./
$ datalad save
add(ok): 1107.0903 Montbrio Pazo - Stuart Landau.pdf (file)
[199 similar messages have been suppressed; disable with datalad.ui.suppress-similar-results=off]
save(ok): . (dataset)
action summary:
add (ok: 209)
save (ok: 1)
tell the dataproxy remote our token and what bucket & prefix to use
$ export EBRAINS_TOKEN=$EBRAINS_TOKEN
$ export DATAPROXY_PATH=insference/pdfs
create the annex remote and datalad push --to
it
$ git annex initremote pdfs type=external externaltype=dataproxy encryption=none
initremote pdfs ok
(recording state in git...)
$ datalad push --to pdfs
copy(ok): 15009.full.pdf (file) [to pdfs...]
[193 similar messages have been suppressed; disable with datalad.ui.suppress-similar-results=off]
action summary:
copy (notneeded: 6, ok: 203)
Status
- sloppy proof of concept
-
git annex testremote
passes - PyPI package for easier install
- better mechanism for specifying bucket & prefix
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for annex_dataproxy-1.0.0rc3-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1eece3506d5a5d69a1c6cbd07c6de2f05d4c97eb098d044d1328d45d9779a009 |
|
MD5 | 115ac6b2430c9fe511f478db5da4ce98 |
|
BLAKE2b-256 | 5420a166f3036ff467ad7a76cdd9d032d2237eae6af375edd6f76283aaa80077 |