Outdoor air quality data
Project description
Fetch outdoor air quality data from multiple resources, the system is made of the following components:
fetching: fetches resources to create the raw material for air quality data. That material is saved to create a history and possibility to fix errors after the fact.
data extraction: extract the data from the raw material to create data feeds.
data reliability: determine which data source is the most reliable at any given time.
Architecture
The data flow from online resource to reliable data point is broken down as follow:
sources: online resources that provide the data, polled at regular interval and cached for archive purpose
feeds: extracted data from sources, feeds are saved as time series in a database
datapoint: extracted from feeds using an election algorithm to provide the most reliable data
Configuration
The default local configuration file is okqconfig.py, this can be changed using the environment variable OKQ_CONFMODULE. The okqconfig.py file should define a dict called settings, this dict can be used to overwrite any configuration option defined in the global_settings of openkongqi.conf.
Station UUID
Each station needs to be attached to a unique identier, a station identifier is of the following formats:
CC/STATION CC/CITY/STATION CC/STATE/CITY/STATION
The following fragments compose the UUID:
CC: country code - a two letter country code as defined by ISO 3166-1 alpha-2, mandatory.
STATE: state name - state name
CITY: city name - city name
STATION: stations name - station name, mandatory.
All names have to be expressed in Uppercase or lowercase or numeric latin letters.
Install
Make sure the lxml package is installed, on debian:
$ apt-get install python-lxml
Install the code:
$ pip install -U pip
$ pip install pip-tools
$ pip-sync requirements.txt
Development
Install development environment:
$ pip install -r requirements-dev.txt
Create necessary database & tables:
(openkongqi)$ python -c "import openkongqi.bin; openkongqi.bin.okq_init()"
Run celery:
(openkongqi)$ python openkongqi/bin.py worker --loglevel=debug --concurrency=1 --autoreload -B
Test
Test the package:
$ python -m unittest discover
Automatic testing in various environments:
$ tox
Versioning
Openkongqi tries to follow the PEP 440 / Semver conventions as defined by PyPA
To update the version of the package, use the following command:
$ bumpr -b -m
Packaging
Create packages:
$ python setup.py sdist bdist_wheel
Push package:
$ twine upload dist/*
$ twine upload -r pypi dist/*
Name origin
Kōngqì (空气) is the Chinese word for air/atmosphere.
License
This software is licensed under the Apache License 2.0. See the LICENSE file in the top distribution directory for the full license text.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openkongqi-0.1.8-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c63f9a031fea0bb553702239ce324972c432f3709f097aeb76315b91691e5b25 |
|
MD5 | 78bfca01e93a8b7f88fd90550d10616d |
|
BLAKE2b-256 | 64995a7fbe7848dc8b551d78a2463b44d4395e8900f21a2551a37c04831b2b08 |