Package for easy datarequests from sensortings
Project description
pandassta: combining sensorthings and pandas
pandassta
package allows easy tools to interact with a FROST-Server Sensorthings API, using pandas dataframes.
This package was developed within a quality assurance project, which is reflected in some specific functions.
Installation
pip install pandassta
Basic usage
Building query
Different wrappers are available for some common queries, but custom queries can easily be constructed. The code below builds a query to get the observations per datastream, with the observed properties of thing 1.
obsprop = Entity(Entities.OBSERVEDPROPERTY)
obsprop.selection = [Properties.NAME, Properties.IOT_ID]
obs = Entity(Entities.OBSERVATIONS)
obs.settings = [Settings.COUNT("true"), Settings.TOP(0)]
obs.selection = [Properties.IOT_ID]
ds = Entity(Entities.DATASTREAMS)
ds.settings = [Settings.COUNT("true")]
ds.expand = [obsprop, obs]
ds.selection = [
Properties.NAME,
Properties.IOT_ID,
Properties.DESCRIPTION,
Properties.UNITOFMEASUREMENT,
Entities.OBSERVEDPROPERTY,
]
thing = Entity(Entities.THINGS)
thing.id = 1
thing.selection = [Properties.NAME, Properties.IOT_ID, Entities.DATASTREAMS]
thing.expand = [ds]
query = Query(base_url=config.load_sta_url(), root_entity=thing)
query_http = query.build()
Step by step tutorial
Lets assume you want to obtain the air temperature and water temperature measured between 2023-03-10 00:00 and 2023-03-11 10:00.
- Get list of things
-
Imports
from pandassta.sta_requests import Config, Entity, Entities, Query, Properties from pandassta.sta_requests import set_sta_url, get_request, response_datastreams_to_df
-
Config
config = Config() set_sta_url("https://sensors.naturalsciences.be/sta/v1.1") thing = Entity(Entities.THINGS) #not structly needed in this step, but needed later
-
Get json
# if `thing` is not defined # query = Query(config.load_sta_url(), root_entity=Entities.THINGS) query = Query(config.load_sta_url(), root_entity=thing) q_url = query.build() # if needed response = get_request(query)
-
- Get list of datastreams
-
why not datastreams directly?
-
using application https://sensors.naturalsciences.be/sensorthings-data/
-
Using pandassta
thing.id = 1 ds = Entity(Entities.DATASTREAMS) thing.selection = [ Entities.DATASTREAMS ] thing.expand = [ ds ] response = get_request(query)
-
- Get the relevant data/observations.
In this example, datastreams 7749 and 7767 were selected, but multiple datastreams give the air or water temperature!
-
define the filter
filter_ds = f"{Properties.IOT_ID} in (7749, 7767)" filter_obs = f"overlaps({Properties.PHENOMENONTIME}, 2023-03-10T00:00Z/2023-03-11T10:00Z)" ds.filter = filter_ds obs = Entity(Entities.OBSERVATIONS) obs.filter = filter_obs # # INCLUDING feature of interest! (coordinates) # foi = Entity(Entities.FEATUREOFINTEREST) # foi.selection = [Properties.COORDINATES, Properties.IOT_ID] # obs.expand = [foi] ds.expand = [obs] response = get_request(query)
-
- Data to a pandas dataframe
-
call pandassta method and verify dataframe
df = response_datastreams_to_df(response[1]) df.head()
-
output:
@iot.selfLink @iot.id phenomenonTime resultTime result resultQuality observation_type observed_property_id units feature_id long lat 0 Link 1155244072 2023-03-10 01:04:00 None 9.8811 2 NaN None Degrees Celsius None None None 1 Link 1155246938 2023-03-10 01:10:00 None 9.8618 2 NaN None Degrees Celsius None None None 2 Link 1155251749 2023-03-10 01:20:03 None 9.7390 2 NaN None Degrees Celsius None None None 3 Link 1155256547 2023-03-10 01:30:06 None 9.7692 2 NaN None Degrees Celsius None None None 4 Link 1155261355 2023-03-10 01:40:08 None 9.7360 2 NaN None Degrees Celsius None None None
-
-
Components
General definitions: sta.py
Reflection of the sensorthings structure.
Construction and execution of queries: sta_requests.py
Classes and function that allow or simplify the construction requests.
General function to go from a json response to a pandas dataframe: df.py
Classes and functions to convert observations to a dataframe.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pandassta-0.0.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | db71c8c55ae8a031d002ad76d38d54422fc45487136c038b1e5a9b81dbaaf3e1 |
|
MD5 | 62c39181dcda584ccd5bda7eb4103342 |
|
BLAKE2b-256 | 834b518934842442fb14d575f97f0bee2146bf7228d696d7902fd78fc698f65f |