No project description provided
Project description
Purple Air Data Logger(s) (PADLs)
A set of data logger(s) that will query purple air sensor(s) for data. That data will then be ingested into a TimeScaleDB PostGreSQL database, CSV files, or a SQLite3 database. To use these tools a PurpleAPI key is required. You can get API keys by sending an email to contact@purpleair.com
with a first and last name to assign them to.
How to Support This Project
Installation
You can install the PurpleAir Data Logger via pip.
python3 -m pip install purpleair_data_logger
You can install PurpleAir Data Logger by cloning down this repo.
git clone https://github.com/carlkidcrypto/purpleair_data_logger.git
cd purpleair_data_logger
python3 setup.py install
Usage PurpleAirPSQLDataLogger.py
usage: PurpleAirPSQLDataLogger.py [-h] -paa_read_key PAA_READ_KEY -paa_write_key PAA_WRITE_KEY
[-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE]
[-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE]
[-paa_group_sensor_request_json_file PAA_GROUP_SENSOR_REQUEST_JSON_FILE]
[-db_drop_all_tables] -db_usr DB_USR [-db_host DB_HOST] -db DB [-db_port DB_PORT]
[-db_pwd DB_PWD]
Collect data from PurpleAir sensors and insert into a database!
options:
-h, --help show this help message and exit
-paa_read_key PAA_READ_KEY
The PurpleAirAPI Read key
-paa_write_key PAA_WRITE_KEY
The PurpleAirAPI write key
-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a single sensor request.
-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a multiple sensor request.
-paa_group_sensor_request_json_file PAA_GROUP_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a group sensor request.
-db_drop_all_tables Set this flag if you wish to drop all tables before loading in new data. Useful if a database
change has happened. Note: Make sure to provide a db_usr with DROP rights. WARNING: ALL
COLLECTED DATA WILL BE LOST!
-db_usr DB_USR The PSQL database user
-db_host DB_HOST The PSQL database host
-db DB The PSQL database name
-db_port DB_PORT The PSQL database port number
-db_pwd DB_PWD The PSQL database password
Using it with single sensor requests...
python3 -m purpleair_data_logger.PurpleAirPSQLDataLogger -db_usr USER -db_host localhost -db DB_NAME -db_port 5432 -db_pwd PASSWORD -paa_read_key 12345678-1234-1234-1234-123456789123 -paa_write_key 12345678-1234-1234-1234-123456789123 -paa_single_sensor_request_json_file PATH_TO_YOUR_FILE
Using it with multiple sensor requests...
python3 -m purpleair_data_logger.PurpleAirPSQLDataLogger -db_usr USER -db_host localhost -db DB_NAME -db_port 5432 -db_pwd PASSWORD -paa_read_key 12345678-1234-1234-1234-123456789123 -paa_write_key 12345678-1234-1234-1234-123456789123 -paa_multiple_sensor_request_json_file PATH_TO_YOUR_FILE
High Level Design
Getting Started
- Grab and install Postgresql for your platform. https://www.postgresql.org/download/
- Create two database users. One for Grafana with select only privileges. The other for the data logger with only insert/create privileges. https://medium.com/coding-blocks/creating-user-database-and-adding-access-on-postgresql-8bfcd2f4a91e, https://www.techonthenet.com/postgresql/grant_revoke.php
- Install and configure TimescaleDB. https://docs.timescale.com/install/latest/self-hosted/
- Install and configure Grafana. https://grafana.com/docs/grafana/latest/setup-grafana/installation/
- Import into your local Grafana instance the dashboard file found here
Usage PurpleAirCSVDataLogger.py
usage: PurpleAirCSVDataLogger.py [-h] -paa_read_key PAA_READ_KEY -paa_write_key PAA_WRITE_KEY
[-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE]
[-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE]
[-paa_group_sensor_request_json_file PAA_GROUP_SENSOR_REQUEST_JSON_FILE]
-save_file_path SAVE_FILE_PATH
Collect data from PurpleAir sensors and store it in CSV files!
options:
-h, --help show this help message and exit
-paa_read_key PAA_READ_KEY
The PurpleAirAPI Read key
-paa_write_key PAA_WRITE_KEY
The PurpleAirAPI write key
-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a single sensor request.
-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a multiple sensor request.
-paa_group_sensor_request_json_file PAA_GROUP_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a group sensor request.
-save_file_path SAVE_FILE_PATH
The path to save CSV files in.
Using it with single sensor requests...
python3 -m purpleair_data_logger.PurpleAirCSVDataLogger -save_file_path SAVE_FILE_PATH -paa_read_key 12345678-1234-1234-1234-123456789123 -paa_write_key 12345678-1234-1234-1234-123456789123 -paa_single_sensor_request_json_file PATH_TO_YOUR_FILE
Using it with multiple sensor requests...
python3 -m purpleair_data_logger.PurpleAirCSVDataLogger -save_file_path SAVE_FILE_PATH -paa_read_key 12345678-1234-1234-1234-123456789123 -paa_write_key 12345678-1234-1234-1234-123456789123 -paa_multiple_sensor_request_json_file PATH_TO_YOUR_FILE
Usage PurpleAirSQLiteDataLogger.py
usage: PurpleAirSQLiteDataLogger.py [-h] -paa_read_key PAA_READ_KEY -paa_write_key PAA_WRITE_KEY
[-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE]
[-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE]
[-paa_group_sensor_request_json_file PAA_GROUP_SENSOR_REQUEST_JSON_FILE] -db_name
DB_NAME
Collect data from PurpleAir sensors and store it a SQLite3 database file!
options:
-h, --help show this help message and exit
-paa_read_key PAA_READ_KEY
The PurpleAirAPI Read key
-paa_write_key PAA_WRITE_KEY
The PurpleAirAPI write key
-paa_single_sensor_request_json_file PAA_SINGLE_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a single sensor request.
-paa_multiple_sensor_request_json_file PAA_MULTIPLE_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a multiple sensor request.
-paa_group_sensor_request_json_file PAA_GROUP_SENSOR_REQUEST_JSON_FILE
The path to a json file containing the parameters to send a group sensor request.
-db_name DB_NAME The path and name for the SQLite3 database file! i.e database_name.db
Using it with single sensor requests...
python3 -m purpleair_data_logger.PurpleAirSQLiteDataLogger -db_name DB_NAME -paa_read_key 12345678-1234-1234-1234-123456789123 -paa_write_key 12345678-1234-1234-1234-123456789123 -paa_single_sensor_request_json_file PATH_TO_YOUR_FILE
Using it with multiple sensor requests...
python3 -m purpleair_data_logger.PurpleAirSQLiteDataLogger -db_name DB_NAME -paa_read_key 12345678-1234-1234-1234-123456789123 -paa_write_key 12345678-1234-1234-1234-123456789123 -paa_multiple_sensor_request_json_file PATH_TO_YOUR_FILE
Sample JSON Configuration File(s)
The following sample json configuration files can be used with any of the data loggers.
PAA Single Sensor Request Example
Out of the parameters listed below only "sensor_index" is required. The others are all optional according to PurpleAirAPI (PAA) documentation. If a field is not being used, mark it 'null' without the single quotes.
{
"sensor_index": 53,
"read_key": null,
"fields": null
}
Or see this file for another example.
Note: Refer to the PurpleAirAPI (PAA) documentation for more information. https://api.purpleair.com/#api-sensors-get-sensor-data
PAA Multiple Sensor Request Example
Out of the parameters listed below only "fields" is required. The others are all optional according to PurpleAirAPI (PAA) documentation. If a field is not being used, mark it 'null' without the single quotes.
{
"fields": "A string comma delimited string of fields to retrieve. i.e fields,name,etc",
"location_type": null,
"read_keys": null,
"show_only": null,
"modified_since": null,
"max_age": null,
"nwlng" : null,
"nwlat": null,
"selng": null,
"selat": null
}
Or see this file for another example.
Note: Refer to the PurpleAirAPI (PAA) documentation for more information. https://api.purpleair.com/#api-sensors-get-sensors-data
The below snippet is taken From the PurpleAirAPI (PAA) documentation:
Field Type Description
fields String
The 'Fields' parameter specifies which 'sensor data fields' to include in the response. It is a comma separated list with one or more of the following:
Station information and status fields:
name, icon, model, hardware, location_type, private, latitude, longitude, altitude, position_rating, led_brightness, firmware_version, firmware_upgrade, rssi, uptime, pa_latency, memory, last_seen, last_modified, date_created, channel_state, channel_flags, channel_flags_manual, channel_flags_auto, confidence, confidence_manual, confidence_auto
Environmental fields:
humidity, humidity_a, humidity_b, temperature, temperature_a, temperature_b, pressure, pressure_a, pressure_b
Miscellaneous fields:
voc, voc_a, voc_b, ozone1, analog_input
PM1.0 fields:
pm1.0, pm1.0_a, pm1.0_b, pm1.0_atm, pm1.0_atm_a, pm1.0_atm_b, pm1.0_cf_1, pm1.0_cf_1_a, pm1.0_cf_1_b
PM2.5 fields:
pm2.5_alt, pm2.5_alt_a, pm2.5_alt_b, pm2.5, pm2.5_a, pm2.5_b, pm2.5_atm, pm2.5_atm_a, pm2.5_atm_b, pm2.5_cf_1, pm2.5_cf_1_a, pm2.5_cf_1_b
PM2.5 pseudo (simple running) average fields:
pm2.5_10minute, pm2.5_10minute_a, pm2.5_10minute_b, pm2.5_30minute, pm2.5_30minute_a, pm2.5_30minute_b, pm2.5_60minute, pm2.5_60minute_a, pm2.5_60minute_b, pm2.5_6hour, pm2.5_6hour_a, pm2.5_6hour_b, pm2.5_24hour, pm2.5_24hour_a, pm2.5_24hour_b, pm2.5_1week, pm2.5_1week_a, pm2.5_1week_b
PM10.0 fields:
pm10.0, pm10.0_a, pm10.0_b, pm10.0_atm, pm10.0_atm_a, pm10.0_atm_b, pm10.0_cf_1, pm10.0_cf_1_a, pm10.0_cf_1_b
Visibility fields:
scattering_coefficient, scattering_coefficient_a, scattering_coefficient_b, deciviews, deciviews_a, deciviews_b, visual_range, visual_range_a, visual_range_b
Particle count fields:
0.3_um_count, 0.3_um_count_a, 0.3_um_count_b, 0.5_um_count, 0.5_um_count_a, 0.5_um_count_b, 1.0_um_count, 1.0_um_count_a, 1.0_um_count_b, 2.5_um_count, 2.5_um_count_a, 2.5_um_count_b, 5.0_um_count, 5.0_um_count_a, 5.0_um_count_b, 10.0_um_count 10.0_um_count_a, 10.0_um_count_b
ThingSpeak fields, used to retrieve data from api.thingspeak.com:
primary_id_a, primary_key_a, secondary_id_a, secondary_key_a, primary_id_b, primary_key_b, secondary_id_b, secondary_key_b
For field descriptions, please see the 'sensor data fields'. section.
location_type optional Number
The location_type of the sensors.
Possible values are: 0 = Outside or 1 = Inside.
read_keys optional String
A read_key is required for private devices. It is separate to the api_key and each sensor has its own read_key. Submit multiple keys by separating them with a comma (,) character for example: key-one,key-two,key-three.
show_only optional String
A comma (,) separated list of sensor_index values. When provided, the results are limited only to the sensors included in this list.
modified_since optional long
The modified_since parameter causes only sensors modified after the provided time stamp to be included in the results. Using the time_stamp value from a previous call (recommended) will limit results to those with new values since the last request. Using a value of 0 will match sensors modified at any time.
max_age optional Integer
Filter results to only include sensors modified or updated within the last number of seconds. Using a value of 0 will match sensors of any age.
Default value: 604800
nwlng optional Number
A north west longitude for the bounding box.
Use a bounding box to limit the sensors returned to a specific geographic area. The bounding box is defined by two points, a north west latitude/longitude and a south east latitude/longitude.
nwlat optional Number
A north west latitude for the bounding box.
selng optional Number
A south east longitude for the bounding box.
selat optional Number
A south east latitude for the bounding box.
PAA Group Sensor Request Example
Out of the parameters below sensor_group_name
, add_sensors_to_group
, and sensor_index_list
are custom settings not
defined in the official PAA documentation. These three setting help drive the group
request feature.
sensor_group_name
- This will be the name assigned to your group. If it doesn't exist already, it will be created.
Otherwise, the first group matching the name will be used.
add_sensors_to_group
- If true, adds the sensors in the sensor_index_list
. If false, sensor_index_list
is ignored.
sensor_index_list
- A list of sensor indexes that will be added to your group if they don't already exist.
The rest of the settings are official PAA settings. They are the same as the ### PAA Multiple Sensor Request Example. Refer above for details.
{
"sensor_group_name": "A Name Goes Here",
"add_sensors_to_group": true,
"sensor_index_list": [
77,
81,
95079,
167897
],
"fields": "A string comma delimited string of fields to retrieve. i.e fields,name,etc",
"location_type": null,
"read_keys": null,
"show_only": null,
"modified_since": null,
"max_age": null,
"nwlng" : null,
"nwlat": null,
"selng": null,
"selat": null
}
Or see this file for another example.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for purpleair_data_logger-1.3.0a1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 18eded7cfeabc2c9c1757091b085a7d2676dc9cb4eed7cd6cc7823efccf6b434 |
|
MD5 | a53abe4de028b12bf26b9fb652cb2066 |
|
BLAKE2b-256 | dba5ac65a5567b671d36007882f903722e109c8228e7c01b38daa7b93276b1be |
Hashes for purpleair_data_logger-1.3.0a1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 461a08aed94fabc753500099ea20144b07b72c11c03ae8fdcc1693ca7e9a101f |
|
MD5 | 60a98bd521776247699198041a8aeed1 |
|
BLAKE2b-256 | c0ccf8d056300f787c5f870b9e95eb71cdcfcddcda44553b7e04f4436547d8a7 |