Skip to main content

Package for easy datarequests from sensortings

Project description

pandassta: combining sensorthings and pandas

pandassta package allows easy tools to interact with a FROST-Server Sensorthings API, using pandas dataframes. This package was developed within a quality assurance project, which is reflected in some specific functions.

Installation

pip install pandassta

Basic usage

Building query

Different wrappers are available for some common queries, but custom queries can easily be constructed. The code below builds a query to get the observations per datastream, with the observed properties of thing 1.

obsprop = Entity(Entities.OBSERVEDPROPERTY)
obsprop.selection = [Properties.NAME, Properties.IOT_ID]

obs = Entity(Entities.OBSERVATIONS)
obs.settings = [Settings.COUNT("true"), Settings.TOP(0)]
obs.selection = [Properties.IOT_ID]

ds = Entity(Entities.DATASTREAMS)
ds.settings = [Settings.COUNT("true")]
ds.expand = [obsprop, obs]
ds.selection = [
    Properties.NAME,
    Properties.IOT_ID,
    Properties.DESCRIPTION,
    Properties.UNITOFMEASUREMENT,
    Entities.OBSERVEDPROPERTY,
]
thing = Entity(Entities.THINGS)
thing.id = 1
thing.selection = [Properties.NAME, Properties.IOT_ID, Entities.DATASTREAMS]
thing.expand = [ds]
query = Query(base_url=config.load_sta_url(), root_entity=thing)
query_http = query.build()

Step by step tutorial

Lets assume you want to obtain the air temperature and water temperature measured between 2023-03-10 00:00 and 2023-03-11 10:00.

  • Get list of things
    • Imports

      from pandassta.sta_requests import Config, Entity, Entities, Query, Properties
      from pandassta.sta_requests import set_sta_url, get_request, response_datastreams_to_df
      
    • Config

      config = Config()
      set_sta_url("https://sensors.naturalsciences.be/sta/v1.1")
      
      thing = Entity(Entities.THINGS) #not structly needed in this step, but needed later
      
    • Get json

      # if `thing` is not defined 
      # query = Query(config.load_sta_url(), root_entity=Entities.THINGS)
      query = Query(config.load_sta_url(), root_entity=thing)
      q_url = query.build() # if needed
      response = get_request(query)
      
  • Get list of datastreams
  • Get the relevant data/observations. In this example, datastreams 7749 and 7767 were selected, but multiple datastreams give the air or water temperature!
    • define the filter

      filter_ds = f"{Properties.IOT_ID} in (7749, 7767)"
      filter_obs = f"overlaps({Properties.PHENOMENONTIME}, 2023-03-10T00:00Z/2023-03-11T10:00Z)"
      ds.filter = filter_ds
      obs = Entity(Entities.OBSERVATIONS)
      
      obs.filter = filter_obs
      
      # # INCLUDING feature of interest! (coordinates)
      # foi = Entity(Entities.FEATUREOFINTEREST)
      # foi.selection = [Properties.COORDINATES, Properties.IOT_ID]
      # obs.expand = [foi]
      
      ds.expand = [obs]
      
      response = get_request(query)
      
  • Data to a pandas dataframe
    • call pandassta method and verify dataframe

      df = response_datastreams_to_df(response[1])
      df.head()
      
      • output:

        @iot.selfLink @iot.id phenomenonTime resultTime result resultQuality observation_type observed_property_id units feature_id long lat
        0 Link 1155244072 2023-03-10 01:04:00 None 9.8811 2 NaN None Degrees Celsius None None None
        1 Link 1155246938 2023-03-10 01:10:00 None 9.8618 2 NaN None Degrees Celsius None None None
        2 Link 1155251749 2023-03-10 01:20:03 None 9.7390 2 NaN None Degrees Celsius None None None
        3 Link 1155256547 2023-03-10 01:30:06 None 9.7692 2 NaN None Degrees Celsius None None None
        4 Link 1155261355 2023-03-10 01:40:08 None 9.7360 2 NaN None Degrees Celsius None None None

Components

General definitions: sta.py

Reflection of the sensorthings structure.

Construction and execution of queries: sta_requests.py

Classes and function that allow or simplify the construction requests.

General function to go from a json response to a pandas dataframe: df.py

Classes and functions to convert observations to a dataframe.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pandassta-0.0.7.tar.gz (34.6 kB view hashes)

Uploaded Source

Built Distribution

pandassta-0.0.7-py3-none-any.whl (28.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page