Skip to main content

Utility for downloading large datasets from Synapse.

Project description

Synapse Downloader

Utility for downloading large datasets from Synapse.

Dependencies

  • Python3.7
  • A Synapse account with a username/password. Authentication through a 3rd party (.e.g., Google) will not work, you must have a Synapse user/pass for the API to authenticate.

Install

pip install synapse-downloader

Configuration

Your Synapse credential can be provided on the command line (--username, --password) or via environment variables.

SYNAPSE_USERNAME=your-synapse-username
SYNAPSE_PASSWORD=your-synapse-password

Usage

usage: synapse-downloader [-h] [-u USERNAME] [-p PASSWORD] [-ll LOG_LEVEL]
                          [-ld LOG_DIR] [-dt DOWNLOAD_TIMEOUT] [-w] [-wc] [-c]
                          [-ci [COMPARE_IGNORE]]
                          entity-id download-path

positional arguments:
  entity-id             The ID of the Synapse entity to download or compare
                        (Project, Folder or File).
  download-path         The local path to save the files to or to compare.

optional arguments:
  -h, --help            show this help message and exit
  -u USERNAME, --username USERNAME
                        Synapse username.
  -p PASSWORD, --password PASSWORD
                        Synapse password.
  -ll LOG_LEVEL, --log-level LOG_LEVEL
                        Set the logging level.
  -ld LOG_DIR, --log-dir LOG_DIR
                        Set the directory where the log file will be written.
  -dt DOWNLOAD_TIMEOUT, --download-timeout DOWNLOAD_TIMEOUT
                        Set the maximum time (in seconds) a file can download
                        before it is canceled.
  -w, --with-view       Use an entity view for loading file info. Fastest for
                        large projects.
  -wc, --with-compare   Run the comparison after downloading everything.
  -c, --compare         Compare a local directory against a remote project or
                        folder.
  -ci [COMPARE_IGNORE], --compare-ignore [COMPARE_IGNORE]
                        Path to directories or files to ignore when comparing.

Development Setup

pipenv --three
pipenv shell
make pip_install
make build
make install_local

See Makefile for all commands.

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for synapse-downloader, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size synapse_downloader-0.0.1-py2.py3-none-any.whl (19.8 kB) File type Wheel Python version py2.py3 Upload date Hashes View hashes
Filename, size synapse-downloader-0.0.1.tar.gz (16.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page