borinud 0.7.3

weather data web api

borinud

Contents

• borinud

  • Introduction

  • Dependencies

  • Installation

  • Context url

    • Example of context url

  • Web interface

  • Web API

    • Summaries

    • Time series

      • Example of timeseries

    • Spatial series

    • Station data

  • Simple web server

  • Deployement under Apache with mod_wsgi

  • Database

    • Caching the summary

  • Copyright

• ChangeLog

  • borinud 0.7.3 (2014-10-08)

  • borinud 0.7.2 (2014-09-30)

  • borinud 0.7 (2014-09-24)

  • borinud 0.6 (2014-09-08)

  • borinud 0.5

  • borinud 0.4

  • borinud 0.3

  • borinud 0.2

• Contributors

Introduction

TODO

Dependencies

• python-dballe

• python-bottle

Installation

From source code:

python setup.py install

Context url

/IDENT/COORDINATES/NETWORK/TIMERANGE/LEVEL/BCODE

• IDENT: mobile station identifier or - for fixed stations

• COORDINATES: coordinates in format int(lon*10^5),int(lat*10^5) - e.g.: 1212345,4467890 means longitude 12.12345 and latitude 44.67890

• NETWORK: station network (max 16 characters)

• TIMERANGE: time range in format type,p1,p2

• LEVEL: level in format type1,l1,type2,l2 (- means missing)

• BCODE: local B table descriptor if format BXXYYY

Example of context url

Instantaneous temperature at 2m above ground measured by fixed station at longitude 12.12345, latitude 44.67890 in network “agrmet”:

/-/1212345,4467890/agrmet/254,0,0/103,2000,-,-/B12101

Web interface

A simple web interface is available at the base dir /.

Web API

The response is in GeoJSON format (http://geojson.org/):

{
  "type": "FeatureCollection",
  "features": [
    {
      "geometry": { "type": "Point", "coordinates": [ 11.95861, 44.36013 ] },
      "properties": {
        "ident": null,
        "lon": 1195861,
        "lat": 4436013,
        "network": "locali",
        "trange_pind": 254,
        "trange_p1": 0,
        "trange_p2": 0,
        "level_t1": 103,
        "level_v1": 2000,
        "level_t2": null,
        "level_v2": null,
        "bcode": "B12101"
      }
    },
    {
      ...
    }
  ]
}

The services can extend the property list (e.g. datetime and value for timeseries services).

The services support the JSONP format, using the GET parameter callback=NAME.

Summaries

/*/*/*/*/*/*/summaries
/*/*/NETWORK/*/*/*/summaries
/-/COORDINATES/NETWORK/*/*/*/summaries
/IDENT/*/NETWORK/*/*/*/summaries
/*/*/NETWORK/TIMERANGE/LEVEL/BCODE/summaries/YEAR/MONTH
/*/*/NETWORK/TIMERANGE/LEVEL/BCODE/summaries/YEAR/MONTH/DAY

The features have a datetime property: if the date extremes are available, then is an array with min and max date. Otherwise, is null.

Time series

/IDENT/COORDINATES/NETWORK/TIMERANGE/LEVEL/BCODE/timeseries/YEAR
/IDENT/COORDINATES/NETWORK/TIMERANGE/LEVEL/BCODE/timeseries/YEAR/MONTH
/IDENT/COORDINATES/NETWORK/TIMERANGE/LEVEL/BCODE/timeseries/YEAR/MONTH/DAY

The features have a datetime and value property.

Example of timeseries

$ curl http://host/path/to/-/1200000,4400000/agrmet/254,0,0/103,10000,-,-/B12101/timeseries/2011
$ curl http://host/path/to/-/1200000,4400000/agrmet/254,0,0/103,10000,-,-/B12101/timeseries/2011/01
$ curl http://host/path/to/-/1200000,4400000/agrmet/254,0,0/103,10000,-,-/B12101/timeseries/2011/01/01

Spatial series

/*/*/NETWORK/TIMERANGE/LEVEL/BCODE/spatialseries/YEAR/MONTH/DAY/HOUR

The features have a datetime and value property.

Station data

/-/COORDINATES/NETWORK/-,-,-/257,-,-,-/*/stationdata
/IDENT/*/-,-,-/257,-,-,-/*/stationdata

The features have a value property.

Simple web server

The borinud.ws module has a simple web server tool:

python -m borinud.ws --dsn=sqlite://...

Or run python -m borinud.ws --help for more options.

Deployement under Apache with mod_wsgi

The Apache configuration is something like this (see http://code.google.com/p/modwsgi/ for details):

<VirtualHost *:80>
  WSGIDaemonProcess borinud user=www-data group=www-data processes=10 threads=1
  WSGIScriptAlias /borinud /var/www/borinud/app.wsgi
</VirtualHost>

And you must create the file /var/www/borinud/app.wsgi:

import borinud.ws
import borinud.db

dba_dsn = "sqlite:/dev/shm/borinud.db"

application = borinud.ws.app
application.config.db = borinud.db.DballeDB(dba_dsn)

Database

borinud reads data from a DB-All.e database:

from borinud.db import DballeDB
# Create a DB-All.e database
db = DballeDB("sqlite:mydatabase.sqlite")
# Load the summary
summary = db.query_summary(dballe.Record())
# Load temperature data
data = db.query_data(dballe.Record(var="B12101"))

Caching the summary

If the database has a lot of data, loading the summary could be a bottle neck. You can dump the summary in a file and use it as a preemptive cache:

import dballe
from borinud.db import DballeDB, SummaryCacheDB
# Create a DB-All.e database with cached summary
db = SummaryCacheDB(DballeDB("sqlite:mydatabase.sqlite"), "/var/lib/borinud-summary.json")
# Dump cached summary in /var/lib/borinud-summary.json.
# NOTE: the cached summary must be written before using it
db.write_cached_summary()
# Load the summary
summary = db.query_summary(dballe.Record())
# Load temperature data
data = db.query_data(dballe.Record(var="B12101"))
# Update the summary cache
db.write_cached_summary()

The web service can use a summary cache:

python -m borinud.ws --dsn=sqlite:mydatabase.sqlite --cached-summary=/var/lib/borinud-summary.json

And the cache can be updated by a simple script:

#!/usr/bin/python
# cacheupdater.py
from borinud.db import DballeDB, SummaryCacheDB
db = SummaryCacheDB(DballeDB("sqlite:mydatabase.sqlite"), "/var/lib/borinud-summary.json")
db.write_cached_summary()

This script can be executed by cron to update periodically the summary:

*/10 * * * * python cacheupdater.py

Note

When the web service starts, the summary must be already created:

python cacheupdater.py && python -m borinud.ws --dsn=sqlite:mydatabase.sqlite --cached-summary=/var/lib/borinud-summary.json

Copyright

Copyright (C) 2013 ARPA-SIM <[e-mail]>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

ChangeLog

Copyright (C) 2013 ARPA-SIM <[e-mail]>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

borinud 0.7.3 (2014-10-08)

• Fixed web api /stationdata

borinud 0.7.2 (2014-09-30)

• Patch for compatibility with ECMAScript-ed6 (Date.parse())

borinud 0.7 (2014-09-24)

• Daily summary

• Daily timeseries in web interface

borinud 0.6 (2014-09-08)

• Removed autotools

• Sphinx documentation

borinud 0.5

• Summary cache filter

• (Optional) versioned web api

borinud 0.4

• Summary cache wrapper

borinud 0.3

• Summary response contains items description.

• Example web client has a dynamic variable list.

borinud 0.2

• Example web client plots monthly timeseries

Contributors

• Emanuele Di Giacomo <[e-mail]>