Cli toolbelt for Datashare.
Project description
Datashare Tarentula
Cli toolbelt for Datashare.
/ \
\ \ ,, / /
'-.`\()/`.-'
.--_'( )'_--.
/ /` /`""`\ `\ \
| | >< | |
\ \ / /
'.__.'
Usage: tarentula [OPTIONS] COMMAND [ARGS]...
Options:
--syslog-address TEXT Syslog address
--syslog-port INTEGER Syslog port
--syslog-facility TEXT Syslog facility
--stdout-loglevel TEXT Change the default log level for stdout error
handler
--help Show this message and exit.
Commands:
download
tagging
Tagging
A command to batch tag documents with a CSV file.
Usage: tarentula tagging [OPTIONS] CSV_PATH
Options:
--datashare-url TEXT Datashare URL
--datashare-project TEXT Datashare project
--throttle INTEGER Request throttling (in ms)
--cookies TEXT Key/value pair to add a cookie to each
request to the API. You can separate
semicolons: key1=val1;key2=val2;...
--traceback / --no-traceback Display a traceback in case of error
--progressbar / --no-progressbar
Display a progressbar
--help Show this message and exit.
CSV formats
Tagging with a documentId
and routing
:
tag,documentId,routing
Actinopodidae,l7VnZZEzg2fr960NWWEG,l7VnZZEzg2fr960NWWEG
Antrodiaetidae,DWLOskax28jPQ2CjFrCo
Atracidae,6VE7cVlWszkUd94XeuSd,vZJQpKQYhcI577gJR0aN
Atypidae,DbhveTJEwQfJL5Gn3Zgi,DbhveTJEwQfJL5Gn3Zgi
Barychelidae,DbhveTJEwQfJL5Gn3Zgi,DbhveTJEwQfJL5Gn3Zgi
Tagging with a documentUrl
:
tag,documentUrl
Mecicobothriidae,http://localhost:8080/#/d/local-datashare/DbhveTJEwQfJL5Gn3Zgi/DbhveTJEwQfJL5Gn3Zgi
Microstigmatidae,http://localhost:8080/#/d/local-datashare/iuL6GUBpO7nKyfSSFaS0/iuL6GUBpO7nKyfSSFaS0
Migidae,http://localhost:8080/#/d/local-datashare/BmovvXBisWtyyx6o9cuG/BmovvXBisWtyyx6o9cuG
Nemesiidae,http://localhost:8080/#/d/local-datashare/vZJQpKQYhcI577gJR0aN/vZJQpKQYhcI577gJR0aN
Paratropididae,http://localhost:8080/#/d/local-datashare/vYl1C4bsWphUKvXEBDhM/vYl1C4bsWphUKvXEBDhM
Porrhothelidae,http://localhost:8080/#/d/local-datashare/fgCt6JLfHSl160fnsjRp/fgCt6JLfHSl160fnsjRp
Theraphosidae,http://localhost:8080/#/d/local-datashare/WvwVvNjEDQJXkwHISQIu/WvwVvNjEDQJXkwHISQIu
Download
A command to download all files matching a query.
Usage: tarentula download [OPTIONS]
Options:
--datashare-url TEXT Datashare URL
--datashare-project TEXT Datashare project
--elasticsearch-url TEXT You can additionally pass the Elasticsearch
URL in order to use scrolling capabilities
of Elasticsearch (useful when dealing with a
lot of results)
--query TEXT The query string to filter documents
--destination-directory TEXT Directory documents will be downloaded
--throttle INTEGER Request throttling (in ms)
--cookies TEXT Key/value pair to add a cookie to each
request to the API. You can separate
semicolons: key1=val1;key2=val2;...
--path-format TEXT Downloaded document path template
--scroll TEXT Scroll duration
--once / --not-once Download file only once
--traceback / --no-traceback Display a traceback in case of error
--progressbar / --no-progressbar
Display a progressbar
--help Show this message and exit.
Testing
To test this tool, you must have Datashare and Elasticsearch running on your development machine. We provide a Docker Compose file to simplify the installation.
docker-compose -p tarentula -f tests/docker-compose.yml up
Install the development dependencies:
sudo apt install pipenv
make install
Then in a separated terminal, just run:
make test
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tarentula-1.4.2.tar.gz
(10.8 kB
view hashes)
Built Distributions
tarentula-1.4.2-py3.6.egg
(13.3 kB
view hashes)
tarentula-1.4.2-py3-none-any.whl
(25.7 kB
view hashes)
Close
Hashes for tarentula-1.4.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 453c3fa67bcffc1ecf201159b3d582b0d75ed7bf7c8aa8f0bda43a4c9ea8294e |
|
MD5 | e1ccc62f0f02620de2c81f79e685d0d1 |
|
BLAKE2b-256 | 3bb960acce4f982befd162cff8b4ea88d713b5c0a04390442d9c8ea4301ec4b4 |