This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Fetch outdoor air quality data from multiple resources, the system is made of the following components:

  • fetching: fetches resources to create the raw material for air quality data. That material is saved to create a history and possibility to fix errors after the fact.
  • data extraction: extract the data from the raw material to create data feeds.
  • data reliability: determine which data source is the most reliable at any given time.

Architecture

The data flow from online resource to reliable data point is broken down as follow:

  • sources: online resources that provide the data, polled at regular interval and cached for archive purpose
  • feeds: extracted data from sources, feeds are saved as time series in a database
  • datapoint: extracted from feeds using an election algorithm to provide the most reliable data

Configuration

The default local configuration file is okqconfig.py, this can be changed using the environment variable OKQ_CONFMODULE. The okqconfig.py file should define a dict called settings, this dict can be used to overwrite any configuration option defined in the global_settings of openkongqi.conf.

Station UUID

Each station needs to be attached to a unique identier, a station identifier is of the following formats:

CC/STATION
CC/CITY/STATION
CC/STATE/CITY/STATION

The following fragments compose the UUID:

  • CC: country code - a two letter country code as defined by ISO 3166-1 alpha-2, mandatory.
  • STATE: state name - state name
  • CITY: city name - city name
  • STATION: stations name - station name, mandatory.

All names have to be expressed in Uppercase or lowercase or numeric latin letters.

Install

Make sure the lxml package is installed, on debian:

$ apt-get install python-lxml

Install the code:

$ pip install -U pip
$ pip install pip-tools
$ pip-sync requirements.txt

Development

Install development environment:

$ pip install -r requirements-dev.txt

Create necessary database & tables:

(openkongqi)$ python -c "import openkongqi.bin; openkongqi.bin.okq_init()"

Run celery:

(openkongqi)$ python openkongqi/bin.py worker --loglevel=debug --concurrency=1 --autoreload -B

Test

Test the package:

$ python -m unittest discover

Automatic testing in various environments:

$ tox

Versioning

Openkongqi tries to follow the PEP 440 / Semver conventions as defined by PyPA

To update the version of the package, use the following command:

$ bumpr -b -m

Packaging

Create packages:

$ python setup.py sdist bdist_wheel

Name origin

Kōngqì (空气) is the Chinese word for air/atmosphere.

License

This software is licensed under the Apache License 2.0. See the LICENSE file in the top distribution directory for the full license text.

Release History

Release History

0.1.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.8

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.7

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.6

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
openkongqi-0.1.2-py2.py3-none-any.whl (29.5 kB) Copy SHA256 Checksum SHA256 py2.py3 Wheel Nov 27, 2016
openkongqi-0.1.2.tar.gz (20.4 kB) Copy SHA256 Checksum SHA256 Source Nov 27, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting