skinnywms 0.13.2

The skinny WMS is a small WMS server that will help you to visualise your NetCDF and Grib Data. The principle is simple: skinny will browse the directory, or the single file passed as argument, and try to interpret each NetCDF or GRIB files. From the metadata, it will be built the getCapabilities document, and find a relevant style to plot the data.

Features:

SkinnyWMS implements 3 of the WMS endpoints:

• getCapabilities: Discover the data, build an XML Document presenting each identified parameter in the file(s) as a layer with the list of their predefined styles. (There is always a default style)
• getMap : Return the selected layer suing the selected style.
• getLegendGraphic: Return the legend.

Usage:

There are 2 ways to start using it. By default the command line starts a gunicorn server. Once running, a small leaflet client is accessible [http://127.0.0.1:5000/]

• The demo:

python demo.py --path /path/to/mydata

• The command line:

skinny-wms --path /path/to/mydata

Use Flask explicitly (for development/debugging):

skinny-wms --path /path/to/mydata --server flask

You can also choose the backend through environment variable:

export SKINNYWMS_SERVER=gunicorn  # or flask

Running behind a reverse proxy

URLs advertised in responses (e.g. the OnlineResource entries in GetCapabilities) are, by default, built from the request as this server sees it. Behind a reverse proxy this yields internal addresses. Set the externally visible base URL of the server, including the path prefix if the proxy mounts skinnywms under a sub-path (the proxy is expected to strip that prefix before forwarding):

export SKINNYWMS_PUBLIC_URL=https://example.com/skinny  # or --public-url

The demo page and all URLs in responses then use this location, regardless of the Host of the incoming request.

To explicitly serve static assets from a custom location (mounted volume, local checkout, etc.):

skinny-wms --path /path/to/mydata --static-path /path/to/static

To check the installed version:

skinny-wms --version

• Or with gunicorn explicitly:

gunicorn --bind localhost:5000 --workers 4 --worker-class gthread --threads 8 skinnywms.wmssvr:application

Run using Docker

By default the docker image starts the application using gunicorn and loads demo data.

• Run the demo:

docker run --rm -p 5000:5000 -it ecmwf/skinnywms 

Now you can try the leaflet demo at http://localhost:5000/

• Run using data on your machine:

docker run --rm -p 5000:5000 -it \
    --volume=/path/to/my/data:/path/inside/the/container \
    --env SKINNYWMS_DATA_PATH=/path/inside/the/container \
      ecmwf/skinnywms

Now you can access the leaflet demo with your data at http://localhost:5000/

• Configure different options by setting environment variables accordingly:

docker run --rm -p 5000:5000 -it \
    --volume=/path/to/my/data:/path/inside/the/container \
    --env SKINNYWMS_DATA_PATH=/path/inside/the/container \
    --env SKINNYWMS_HOST=0.0.0.0 \
    --env SKINNYWMS_PORT=5000 \
    --env SKINNYWMS_GUNICORN_WORKERS=4 \
    --env SKINNYWMS_GUNICORN_THREADS=8 \
      ecmwf/skinnywms

Now you can access the GetCapabilities document for your data at http://localhost:5000/wms?request=GetCapabilities

Additional Configuration

You can provide additional configuration, such as CORS configration, through environment variables. See .env file for more details.

For static assets, set SKINNYWMS_STATIC_PATH to a directory that contains css, js and related static files. This is equivalent to passing --static-path on the command line.

Installation

SkinnyWMS depends on the ECMWF Magics library.

If you do not have Magics installed on your platform, skinnywms is available on conda forge https://conda-forge.org/

conda config --add channels conda-forge
conda install skinnywms

If you have Magics already installed you can use pip:

pip install skinnywms

Limitations:

• SkinnyWMS will perform better on well formatted and documented NetCDF and GRIB.

• grib fields containing corresponding wind components u,v need to be placed together in a single grib file in order to be displayed as vectors/wind barbs in SkinnyWMS. You can combine multiple grib files into a single file using ecCodes grib_copy (included in the docker image), e.g.:

grib_copy input_wind_u_component.grb2 input_wind_v_component.grib2 output_wind_u_v_combined.grb2

• The time and elevation dimension implementations follow OGC Met Ocean DWG WMS 1.3 Best Practice for using Web Map Services (WMS) with Time-Dependent or Elevation-Dependent Data. To enable dimension grouping (disabled by default) set the environment variable SKINNYWMS_ENABLE_DIMENSION_GROUPING=1

• By default the forecast reference time (the forecast run) is not a dimension: fields of different runs that share a validity time and elevation overwrite one another, so only one run can be served at a time. To serve several runs from the same layer, pass --enable-reference-time-dimension or set SKINNYWMS_ENABLE_REFERENCE_TIME_DIMENSION=1. Layers then holding more than one run advertise a reference_time dimension, which clients select with the DIM_REFERENCE_TIME parameter, e.g.:

...&LAYERS=2t&DIM_REFERENCE_TIME=2019-01-01T00:00:00Z&TIME=2019-01-01T12:00:00Z

The run defaults to the most recent one available. Note that WMS declares dimensions independently of one another, so the advertised time extent is the union over all runs; a validity time that a given run does not reach resolves to the nearest earlier time within that run.

• development stage: Alpha,

Add your own styles

Multi-process

Cache

How to install Magics

that must be installed on the system and accessible as a shared library. Some Linux distributions ship a binary version that may be installed with the standard package manager.

As an alternative you may install the official source distribution by following the instructions at https://software.ecmwf.int/magics/Installation+Guide Magics is available on github https://github.com/ecmwf/magics

Note that Magics support for the Windows operating system is experimental.

Alternatively you can use the ecmwflibs package (currently in Alpha development stage) to install magics and eccodes libraries:

pip install ecmwflibs

Start up a local development environment (Docker)

Make sure you have Docker and docker-compose installed. Then run:

docker-compose up

This will build a dev image and start up a local gunicorn server at http://localhost:5000 based on the configuration stored in docker-compose.yml and .env and by default try to load all GRIB and NetCDF data stored in skinnywms/testdata.

Benchmark WMS performance (GetMap + non-blocking GetCapabilities)

Use the benchmark script to generate concurrent GetMap load while polling GetCapabilities asynchronously in parallel:

python scripts/benchmark_wms.py --base-url http://127.0.0.1:5000/wms --duration 30 --concurrency 8 --cap-interval 0.5

The script will:

• discover a valid GetMap request from GetCapabilities
• report cold-start timings for the first GetCapabilities and first GetMap requests (useful for lazy-loading/indexing cost)
• run concurrent GetMap requests for the specified duration
• poll GetCapabilities in a non-blocking loop while load is running
• report throughput and latency percentiles for both request types

Export machine-readable results:

python scripts/benchmark_wms.py --base-url http://127.0.0.1:5000/wms --duration 30 --concurrency 8 --cap-interval 0.5 --output-json benchmark.json --output-csv benchmark.csv

To measure true lazy-loading startup cost, restart the service immediately before running the benchmark so the first GetCapabilities request is genuinely cold.

You can also let the script do this automatically:

python scripts/benchmark_wms.py --base-url http://127.0.0.1:5000/wms --restart-command "docker compose restart skinnywms" --duration 30 --concurrency 8 --cap-interval 0.5 --output-json benchmark-cold.json --output-csv benchmark-cold.csv

By default, readiness is checked on the site root (/) to avoid touching GetCapabilities before cold-start timing is measured. You can override this with --ready-url if needed.

Example with higher parallelism:

python scripts/benchmark_wms.py --base-url http://127.0.0.1:5000/wms --duration 60 --concurrency 16 --cap-interval 0.2

Contributing

The main repository, as well as related projects are hosted on GitHub. Testing, bug reports and contributions to all our projects are highly welcomed and appreciated:

• https://github.com/ecmwf/skinnywms
• https://github.com/ecmwf/magics-python
• https://github.com/ecmwf/magics
• https://github.com/ecmwf/eccodes
• https://github.com/ecmwf/ecmwflibs

Lead developers:

• Sylvie Lamy-Thepaut <https://github.com/sylvielamythepaut>_ - ECMWF
• Baudouin Raoult <https://github.com/b8raoult> - ECMWF
• Eduard Rosert <https://github.com/EduardRosert> - ECMWF

Main contributors:

• Stephan Siemen <https://github.com/stephansiemen>_ - ECMWF
• Milana Vuckovic <https://github.com/milanavuckovic> - ECMWF

License

Copyright 2017-2019 European Centre for Medium-Range Weather Forecasts (ECMWF).

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.