Skip to main content

A Python module for downloading ESA satellite data

Project description

Build Status

esahub: Downloading data from ESA scihub

Author: Johannes Hansen (johannes.hansen@ed.ac.uk)

Setup

Install esahub:

$ pip install esahub

This will also create a YAML configuration file in ~/.esahub.conf (unless that file exists) by copying config.yaml. The configuration settings in ~/.esahub.conf have precedence over the settings in config.yaml.

You should overwrite the required settings in your ~/.esahub.conf, especially:

  • GENERAL.DATA_DIR
  • GENERAL.LOG_FILE
  • GENERAL.EMAIL_REPORT_RECIPIENTS

The full range of Copernicus data servers are supported, including:

  • https://scihub.copernicus.eu/dhus
  • https://scihub.copernicus.eu/apihub
  • https://scihub.copernicus.eu/s3 (guest access)
  • https://s5phub.copernicus.eu/dhus (guest access)
  • https://tmphub.copernicus.eu/dhus
  • https://colhub.copernicus.eu/dhus
  • https://colhub2.copernicus.eu/dhus

For the majority of these you will need to provide your own authentication details in SERVERS.

Testing the installation

The recommended way of running tests is:

$ python setup.py test

Note: Running the tests may take a while as it includes testing live downloads from SciHub (although with very small files).

Command Line Usage

$ esahub [cmd] [args] ...

Available commands:

Command Description
ls Queries SciHub for archives matching the specified query parameters. Prints the total number of files and data size.
get Queries SciHub like ls, but then downloads the files.
doctor Checks local .zip archives for consistency, either by validating the zip format or by comparing to the MD5 checksum from SciHub. Allows to either delete or repair broken files.

Options

Option Available for Description
<SAT> all Satellite to query, e.g. S1A, S1B, S2A, S2B, S3A
-N | --nproc <N> all number of parallel processes/downloads (defaults to config GENERAL.N_PROC and GENERAL.N_DOWNLOADS)
--log all write log file
-d | --dir <DIR> all raw data directory (defaults to config GENERAL.DATA_DIR)
--out <FILE> ls write files to JSON
--in <FILE> get read files from JSON
--mission <MISSION> ls|get Sentinel-1|Sentinel-2|Sentinel-3 (default: Sentinel-3)
-g | --geo <WKT> ls|get geospatial location in WKT format
--location <LOCATION> ls|get location as defined in config LOCATIONS
-A | --from_time <TIME> ls|get start time in format %Y-%m-%dT%H:%M:%S.000Z
-B | --to_time <TIME> ls|get end time in format %Y-%m-%dT%H:%M:%S.000Z
-t | --time <ARG> ls|get Convenience wrapper for --from_time and --to_time
today|yesterday|24h|midnight
--type <TYPE> ls|get e.g. GRD
-q | --query <QUERY> ls|get custom query for SciHub, e.g. for single archive: identifier:...
-m | --mode <MODE> doctor zip|file
--delete doctor delete corrupt files
--repair doctor redownload corrupt files
--email all send email report
--gui all use the GUI (by default runs in background)

Examples

Ex 1. Retrieve the number of archives and total file size of Sentinel-3 archives uploaded to SciHub during the past midnight-to-midnight period intersecting Ireland (only works if Ireland is defined in the config item LOCATIONS):

$ esahub ls -t yesterday --location=Ireland

Ex 2. Download the archives uploaded yesterday for four locations.

$ esahub get -t yesterday --location=Ireland_Mace_Head --location=Namibia_Gobabeb --location=Italy_Rome_Tor_Vergata --location=France_La_Crau

Ex 3. Query SciHub for all available Sentinel-2 data for Ireland and write the result to a JSON file. Then read that JSON file by the get command, thus downloading the specified files. Note: Since the JSON file may be edited manually, this approach offers the most flexibility.

$ esahub ls --location=Ireland --mission=Sentinel-2 --out=Sen2_IE.json
$ esahub get --in=Sen2_IE.json --log

Ex 4. Check all zip archives in a custom directory for MD5 consistency and generate a log file.

$ esahub doctor --dir=/path/to/dir/ --mode=md5 --log

Python API

from esahub import scihub
query = {'mission': 'Sentinel-1',
         'geo': 'POINT(-9.0 53.0)',
         'time': 'today'}
files = scihub.search(query)
scihub.download_many(files)

Dependencies

Python packages

  • pyyaml
  • numpy
  • lxml
  • pyproj
  • shapely
  • netCDF4
  • python-dateutil
  • pytz
  • tqdm

Libraries

  • libgeos_c

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for esahub, version 0.0.1
Filename, size File type Python version Upload date Hashes
Filename, size esahub-0.0.1-py3-none-any.whl (35.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size esahub-0.0.1.tar.gz (31.9 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page