Skip to main content

Outdoor air quality data

Project description

Testing Status

Fetch outdoor air quality data from multiple resources, the system is made of the following components:

  • fetching: fetches resources to create the raw material for air quality data. That material is saved to create a history and possibility to fix errors after the fact.
  • data extraction: extract the data from the raw material to create data feeds.
  • data reliability: determine which data source is the most reliable at any given time.

Architecture

The data flow from online resource to reliable data point is broken down as follow:

  • sources: online resources that provide the data, polled at regular interval and cached for archive purpose
  • feeds: extracted data from sources, feeds are saved as time series in a database
  • datapoint: extracted from feeds using an election algorithm to provide the most reliable data

Configuration

The default local configuration file is okqconfig.py, this can be changed using the environment variable OKQ_CONFMODULE. The okqconfig.py file should define a dict called settings, this dict can be used to overwrite any configuration option defined in the global_settings of openkongqi.conf.

Station UUID

Each station needs to be attached to a unique identier, a station identifier is of the following formats:

CC/STATION
CC/CITY/STATION
CC/STATE/CITY/STATION

The following fragments compose the UUID:

  • CC: country code - a two letter country code as defined by ISO 3166-1 alpha-2, mandatory.
  • STATE: state name - state name
  • CITY: city name - city name
  • STATION: stations name - station name, mandatory.

All names have to be expressed in Uppercase or lowercase or numeric latin letters.

Install

Make sure the lxml package is installed, on debian:

$ apt-get install python-lxml

Install the code:

$ pip install -U pip
$ pip install pip-tools
$ pip-sync requirements.txt

Development

Install development environment:

$ pip install -r requirements-dev.txt

Create necessary database & tables:

(openkongqi)$ python -c "import openkongqi.bin; openkongqi.bin.okq_init()"

Run celery:

(openkongqi)$ python openkongqi/bin.py worker --loglevel=debug --concurrency=1 --autoreload -B

Test

Test the package:

$ python -m unittest discover

Automatic testing in various environments:

$ tox

Versioning

Openkongqi tries to follow the PEP 440 / Semver conventions as defined by PyPA

To update the version of the package, use the following command:

$ bumpr -b -m

Packaging

Create packages:

$ python setup.py sdist bdist_wheel

Push package:

$ twine upload dist/*
$ twine upload -r pypi dist/*

Name origin

Kōngqì (空气) is the Chinese word for air/atmosphere.

License

This software is licensed under the Apache License 2.0. See the LICENSE file in the top distribution directory for the full license text.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
openkongqi-0.1.8-py2.py3-none-any.whl (30.3 kB) Copy SHA256 hash SHA256 Wheel py2.py3 Sep 21, 2018
openkongqi-0.1.8.tar.gz (19.6 kB) Copy SHA256 hash SHA256 Source None Sep 21, 2018

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page