bcdata 0.2.0

Data BC Distribution Service, automated

British Columbia’s data distribution service, automated.

There is a wealth of British Columbia geographic information available as open data, but access to much of it is available only via a ‘Custom Download’ through the Data Distribution Service application - direct download urls are not available. Data analysis tasks requiring many inputs can be tedious to set up or replicate.

This Python module and CLI script enables relatively quick, scriptable downloads of BC geographic data.

Note

• this tool is for my convenience, it is in no way endorsed by the Province of Britsh Columbia or DataBC

• use with care, please don’t overload the service

• the download service seems to be ok with many download requests but failures may be unpredictable

• data are generally licensed as OGL-BC, but it is up to the user to check the licensing for any data downloaded

Installation

Install via pip:

$ pip install bcdata

Usage

The most basic usage requires:

• a valid email address (a required Distribution Service form input, the address is not otherwise used)

• name of any DataBC Catalog package/dataset that is available via the Download Service

For example, to order and download airport data, use bc-airports - the final segment of the url string is the package/dataset name. For Grizzly Bear Population Units use grizzly-bear-population-units.

Python module

>>> import bcdata
>>> dl = bcdata.download('bc-airports', '[e-mail]')
>>> dl
/tmp/bcdata/unzipped_download_folder/GSR_AIRPORTS_SVW.gdb

Download times will vary based mainly on the size of your requested data. Expect about a minute for the smallest requests to complete. Individual downloads will time out after 2 hours - this is long enough for datasets larger than the max size permitted on the web form.

CLI

The CLI uses the $BCDATA_EMAIL environment variable if available, otherwise an email address must be provided as an option.

$ bcdata --help
Usage: bcdata [OPTIONS] DATASET

  Download a dataset from BC Data Distribution Service

Options:
  --email TEXT       Email address. Default: $BCDATA_EMAIL
  -d, --driver TEXT  Output file format. Default: FileGDB
  -o, --output TEXT  Output folder/gdb
  -i, --info         Display info about dataset
  --help             Show this message and exit.

Common uses might look something like this:

$ bcdata --email [e-mail] bc-airports  # basic usage
$ export BCDATA_EMAIL=[e-mail]         # set a default email
$ bcdata bc-airports                                 # use default email
$ bcdata -o my_spots.gdb bc-airports                 # download to specified output location
$ bcdata bc-airports \                               # get airports as shapefile
    -f shp \
    -o bc_airports
$ bcdata -i bc-airports                              # what are the source schema and table names?
{"table": "gsr_airports_svw", "schema": "whse_imagery_and_base_maps"}

Note that data are downloaded to specified folder. For above example, a bc_airports folder would be created in the current working directory and the individual shp, prj etc files would be found within.

Projections / CRS

Several projections are available on request from the Download Service, but this tool does not support this option. All data are downloaded as the default CRS (BC Albers, which should be EPSG:3005).

Use some other tool to reproject your downloads.

Development and testing

Note that tests require Fiona (and thus GDAL) to verify downloads. Using a virtualenv is probably a good idea.

macOS/Linux/etc

$ mkdir bcdata_env
$ virtualenv bcdata_env
$ source bcdata_env/bin/activate
(bcdata_env)$ git clone [e-mail]:smnorris/bcdata.git
(bcdata_env)$ cd bcdata
(bcdata_env)$ pip install -e .[test]
(bcdata_env)$ export BCDATA_EMAIL=[e-mail]
(bcdata_env)$ py.test

Windows

Development setup on Windows should be quite similar but installing Fiona on Windows can be more challenging.

Credits

• @ateucher for the correct POST url and syntax

• pyskel for the python packaging template