birdhousebuilder.recipe.pywps 0.10.0

A Buildout recipe to install and configure PyWPS Web Processing Service with conda.

*****************************
birdhousebuilder.recipe.pywps
*****************************

.. image:: https://travis-ci.org/bird-house/birdhousebuilder.recipe.pywps.svg?branch=master
:target: https://travis-ci.org/bird-house/birdhousebuilder.recipe.pywps
:alt: Travis Build

.. image:: https://img.shields.io/github/license/bird-house/birdhousebuilder.recipe.pywps.svg
:target: https://github.com/bird-house/birdhousebuilder.recipe.pywps/blob/master/LICENSE.txt
:alt: GitHub license

.. image:: https://badges.gitter.im/bird-house/birdhouse.svg
:target: https://gitter.im/bird-house/birdhouse?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
:alt: Join the chat at https://gitter.im/bird-house/birdhouse

Introduction
************

``birdhousebuilder.recipe.pywps`` is a `Buildout`_ recipe to install and configure `PyWPS`_ with `Anaconda`_. `PyWPS`_ is a Python implementation of a `Web Processing Service`_ (WPS). ``PyWPS`` will be deployed as a `Supervisor`_ service and is available behind a `Nginx`_ web server.
This recipe is used by the `Birdhouse`_ project.

.. _`Buildout`: http://buildout.org/
.. _`Anaconda`: http://continuum.io/
.. _`Supervisor`: http://supervisord.org/
.. _`Nginx`: http://nginx.org/
.. _`PyWPS`: http://pywps.org/
.. _`PyWPS documentation`: http://pywps.readthedocs.io/en/latest/configuration.html
.. _`Web Processing Service`: https://en.wikipedia.org/wiki/Web_Processing_Service
.. _`Birdhouse`: http://bird-house.github.io/


Usage
*****

The recipe requires that Anaconda is already installed. You can use the buildout option ``anaconda-home`` to set the prefix for the anaconda installation. Otherwise the environment variable ``CONDA_PREFIX`` (variable is set when activating a conda environment) is used as conda prefix.

It installs the ``pywps`` package from a conda channel in a conda environment defined by ``CONDA_PREFIX``. The location of the intallation is given by the ``prefix`` buildout option. It setups a `PyWPS`_ output folder in ``${prefix}/var/lib/pywps``. It deploys a `Supervisor`_ configuration for ``PyWPS`` in ``${prefix}/etc/supervisor/conf.d/pywps.conf``. Supervisor can be started with ``${prefix}/etc/init.d/supervisor start``.

The recipe will install the ``nginx`` package from a conda channel and deploy a Nginx site configuration for ``PyWPS``. The configuration will be deployed in ``${prefix}/etc/nginx/conf.d/pywps.conf``. Nginx will be started by supervisor.

By default ``PyWPS`` will be available on http://localhost:8091/wps?service=WPS&version=1.0.0&request=GetCapabilities.

The recipe depends on:

* ``birdhousebuilder.recipe.conda``,
* ``birdhousebuilder.recipe.supervisor``,
* ``birdhousebuilder.recipe.nginx`` and
* ``zc.recipe.deployment``.

Supported options
=================

The PyWPS options which are configured by buildout are explained in the `PyWPS documentation`_.

The recipe supports the following buildout options:

**anaconda-home**
Buildout option pointing to the root folder of the Anaconda installation. Default: ``$HOME/anaconda``.

Buildout options for ``pywps``:

**prefix**
Deployment option to set the prefix of the installation folder. Default: ``/``

**user**
Deployment option to set the run user.

**etc-user**
Deployment option to set the user of the ``/etc`` directory. Default: ``root``

**name**
The name of your WPS project (used for config names and folder path).

**hostname**
The hostname of the ``PyWPS`` service (nginx). Default: ``localhost``

**http-port**
The http port of the ``PyWPS`` service (nginx). Default: ``8091``

**https-port**
The https port of the ``PyWPS`` service (nginx). Default: ``28091``

**http-output-port**, **output-port**
The http port of the ``PyWPS`` output file service (nginx). Default: ``8090``

**https-output-port**
The https port of the ``PyWPS`` output file service (nginx). Default: ``28090``

**ssl-verify-client**
Nginx option to verify SSL client certificates. Possible values: ``off`` (default), ``on``, ``optional``.
`Nginx ssl_verify_client option`_

**ssl-client-certificate**
Nginx option with the name of the bundle of CA certificates for the client. Default: ``esgf-ca-bundle.crt``.
`Nginx ssl_client_certificate`_

**ssl-client-certificate-url**
Optional URL to download a bundle of CA certificates for ``ssl-client-certificate``. Default:
https://github.com/ESGF/esgf-dist/raw/master/installer/certs/esgf-ca-bundle.crt

**workers**
The number of gunicorn workers for handling requests. Default: 1

**application**
PyWPS WSGI Application. Default: ``${name}:application``.

**title**
Title used for your WPS service.

**abstract**
Description of your WPS service.

**loglevel**
Logging level for ``PyWPS``. Default: ``WARN``

**logformat**
Logging string format for ``PyWPS``. Default: ``%(asctime)s] [%(levelname)s] line=%(lineno)s module=%(module)s %(message)s``

**database**
Database where the logs about requests/responses is to be stored. Allowed values are ``memory`` or ``sqlite``.
Default: ``sqlite``.

**parallelprocesses**
Maximum number of parallel running processes.
The effective number of parallel running processes is limited by the number of cores
in the hosting machine. Default: 2

**maxprocesses**
Maximum number of processes which are accepted in the queue. Default: 30

**maxrequestsize**
Maximal request size accepted in WPS process. Default: 30mb

**allowedinputpaths**
List of server paths which are allowed to be accessed by file URL complex input parameters.

**mode**
Processing mode to run jobs. Allowed values are ``default`` (multiprocessing) and ``slurm``.
Default: default



Example usage
=============

The following example ``buildout.cfg`` installs ``PyWPS`` with Anaconda::

[buildout]
parts = pywps

[pywps]
recipe = birdhousebuilder.recipe.pywps
name = myproject
prefix = /
user = www-data
hostname = localhost
http-port = 8091
https-port = 28091

# pywps options
processes-import = myproject.processes
processes-path = ${buildout:directory}/myproject/processes
title = MyProject ...
abstract = MyProject does ...

After installing with Buildout start the ``PyWPS`` service with::

$ cd ${prefix}
$ etc/init.d/supervisord start # start|stop|restart
$ etc/init.d/nginx start # start|stop|restart
$ bin/supervisorctl status # check that pycsw is running
$ less var/log/pywps/myproject.log # check log file

Open your browser with the following URL::

http://localhost:8091/wps?service=WPS&version=1.0.0&request=GetCapabilities

Authors
*******

Carsten Ehbrecht ehbrecht at dkrz.de

Change History
**************

0.10.0 (2018-03-20)
===================

* cleaned up recipe (#15).
* update nginx option with ssl-verify option (#13).

0.9.3 (2018-01-24)
==================

* added outputurl option to overwrite defaults.

0.9.2 (2017-07-28)
==================

* removed unused ``remotehost`` option.
* updated ``database`` option.

0.9.1 (2017-06-28)
==================

* added allowedinputpaths option.

0.9.0 (2017-05-18)
==================

* cleaned up buildout build ... fixed travis.
* removed unused options processes-path and processes-import.
* does not generate etc/pywps/app.py anymore.
* added application option.

0.8.9 (2017-05-11)
=================

* added processing options ``mode`` and ``remotehost``.

0.8.8 (2017-04-26)
==================

* added ``database`` option.

0.8.7 (2017-03-30)
==================

* added ``logformat`` option.

0.8.6 (2017-02-09)
==================

* added ``extra-options`` which replaces also the ``archive-root`` option.
* added test module test_unit.py for unit-testing.
* updated versions.cfg.


0.8.5 (2017-02-01)
==================

* setting NCARG_ROOT in gunicorn config.

0.8.4 (2017-01-31)
==================

* added options ``sethomedir`` and ``setworkdir``.

0.8.3 (2017-01-16)
==================

* added ``archive-root`` cache option.

0.8.2 (2016-12-09)
==================

* set ``HOME`` in gunicorn config to var/lib/pywps/tmp/${name}.
* added ``parallelprocesses`` option.
* added ``https-output-port`` and ``http-output-port`` option.
* added ``enable-https`` option.
* updated Readme.

0.8.1 (2016-11-10)
==================

* fixed wspapp.py template: using processes-import option
* using proccess-path option.
* updated Readme.

0.8.0 (2016-10-17)
==================

* updated to pywps 4.x (new config files)

0.5.1 (2016-07-06)
==================

* added client_body_max_size to nginx config for uploads.

0.5.0 (2016-06-30)
==================

* using zc.recipe.deployment.
* changed cache path to ``var/lib/pywps/cache``.
* changed tmp path to ``var/lib/pywps/tmp``.

0.4.0 (2016-03-03)
==================

* update to pywps 3.2.5.
* fixed wpsapp.py wsgi script.
* added missing server/debug parameter to pywps.cfg.

0.3.10 (2016-02-12)
===================

* added ``maxoperations`` and ``maxfilesize`` to options.

0.3.9 (2016-02-08)
==================

* updated pywps conda dependency.

0.3.8 (2016-02-04)
==================

* configure pywps logFile in ${prefix}/var/log/pywps/.

0.3.7 (2016-01-22)
==================

* added environment variables PATH and GDAL_DATA to bin/runwps script.

0.3.6 (2016-01-22)
==================

* generates bin/runwps script to test pywps service.

0.3.5 (2016-01-21)
==================

* updated pywps conda dependency.

0.3.4 (2016-01-19)
==================

* cleaned up templates.
* added eventlet to the conda dependencies.

0.3.3 (2016-01-18)
==================

* renamed gunicorn template.
* updated pywps.cfg for gunicron keywords template.

0.3.2 (2016-01-15)
==================

* added gunicorn workers parameter.
* using gevent worker_class.
* using gunicorn config folder etc/gunicorn/.

0.3.1 (2016-01-05)
==================

* using cache path var/lib/cache/.

0.3.0 (2015-12-01)
==================

* updated to latest pywps wsgi app.

0.2.6 (2015-06-25)
==================

* added user option for supervisor and nginx.

0.2.5 (2015-06-24)
==================

* enabled https access.

0.2.4 (2015-06-23)
==================

* removed unused proxyEnabled option.
* cleaned up templates.

0.2.3 (2015-05-18)
==================

* updated supervisor config.
* log pywps to stderr/supervisor.

0.2.2 (2015-04-21)
==================

* do not set ``HOME`` environment variable in gunicorn.

0.2.1 (2015-03-24)
==================

* added mako_cache to pywps config.

0.2.0 (2015-02-24)
==================

* installing in conda enviroment ``birdhouse``.
* using ``$ANACONDA_HOME`` environment variable.
* separation of anaconda-home and installation prefix.

0.1.11 (2014-12-08)
===================

* changed default log level.

0.1.10 (2014-12-06)
===================

* Don't update conda on buildout update.
* Sets PYTHONPATH in gunicon.conf.py. Used in PyWPS async processes.

0.1.9 (2014-11-26)
==================

* Added cache section to pywps.cfg template.

0.1.8 (2014-11-03)
==================

* GDAL_DATA added to environment in gunicorn.conf.py template.

0.1.7 (2014-08-27)
==================

* phoenix option added for wpsoutputs.

0.1.6 (2014-08-26)
==================

* Fixed proxy config for wpsoutputs.

0.1.5 (2014-08-23)
==================

added cache path to nginx configuration.

0.1.4 (2014-08-17)
==================

added /usr/local/bin to gunicorn path (needed for brew on macosx)

0.1.3 (2014-08-01)
==================

Updated documentation.

0.1.2 (2014-07-24)
==================

Fixed hostname in nginx template.

0.1.1 (2014-07-11)
==================

Fixed HOME env in gunicorn template.

0.1.0 (2014-07-10)
==================

Initial Release.