Skip to main content

Package for easy datarequests from sensortings

Project description

pandassta: combining sensorthings and pandas

pandassta package allows easy tools to interact with a FROST-Server Sensorthings API, using pandas dataframes. This package was developed within a quality assurance project, which is reflected in some specific functions.

Installation

pip install pandassta

Basic usage

Building query

Different wrappers are available for some common queries, but custom queries can easily be constructed. The code below builds a query to get the observations per datastream, with the observed properties of thing 1.

obsprop = Entity(Entities.OBSERVEDPROPERTY)
obsprop.selection = [Properties.NAME, Properties.IOT_ID]

obs = Entity(Entities.OBSERVATIONS)
obs.settings = [Settings.COUNT("true"), Settings.TOP(0)]
obs.selection = [Properties.IOT_ID]

ds = Entity(Entities.DATASTREAMS)
ds.settings = [Settings.COUNT("true")]
ds.expand = [obsprop, obs]
ds.selection = [
    Properties.NAME,
    Properties.IOT_ID,
    Properties.DESCRIPTION,
    Properties.UNITOFMEASUREMENT,
    Entities.OBSERVEDPROPERTY,
]
thing = Entity(Entities.THINGS)
thing.id = 1
thing.selection = [Properties.NAME, Properties.IOT_ID, Entities.DATASTREAMS]
thing.expand = [ds]
query = Query(base_url=config.load_sta_url(), root_entity=thing)
query_http = query.build()

Step by step tutorial

Lets assume you want to obtain the air temperature and water temperature measured between 2023-03-10 00:00 and 2023-03-11 10:00.

  • Get list of things
    • Imports

      from pandassta.sta_requests import Config, Entity, Entities, Query, Properties
      from pandassta.sta_requests import set_sta_url, get_request, response_datastreams_to_df
      
    • Config

      config = Config()
      set_sta_url("https://sensors.naturalsciences.be/sta/v1.1")
      
      thing = Entity(Entities.THINGS) #not structly needed in this step, but needed later
      
    • Get json

      # if `thing` is not defined 
      # query = Query(config.load_sta_url(), root_entity=Entities.THINGS)
      query = Query(config.load_sta_url(), root_entity=thing)
      q_url = query.build() # if needed
      response = get_request(query)
      
  • Get list of datastreams
  • Get the relevant data/observations. In this example, datastreams 7749 and 7767 were selected, but multiple datastreams give the air or water temperature!
    • define the filter

      filter_ds = f"{Properties.IOT_ID} in (7749, 7767)"
      filter_obs = f"overlaps({Properties.PHENOMENONTIME}, 2023-03-10T00:00Z/2023-03-11T10:00Z)"
      ds.filter = filter_ds
      obs = Entity(Entities.OBSERVATIONS)
      
      obs.filter = filter_obs
      
      # # INCLUDING feature of interest! (coordinates)
      # foi = Entity(Entities.FEATUREOFINTEREST)
      # foi.selection = [Properties.COORDINATES, Properties.IOT_ID]
      # obs.expand = [foi]
      
      ds.expand = [obs]
      
      response = get_request(query)
      
  • Data to a pandas dataframe
    • call pandassta method and verify dataframe

      df = response_datastreams_to_df(response[1])
      df.head()
      
      • output:

        @iot.selfLink @iot.id phenomenonTime resultTime result resultQuality observation_type observed_property_id units feature_id long lat
        0 Link 1155244072 2023-03-10 01:04:00 None 9.8811 2 NaN None Degrees Celsius None None None
        1 Link 1155246938 2023-03-10 01:10:00 None 9.8618 2 NaN None Degrees Celsius None None None
        2 Link 1155251749 2023-03-10 01:20:03 None 9.7390 2 NaN None Degrees Celsius None None None
        3 Link 1155256547 2023-03-10 01:30:06 None 9.7692 2 NaN None Degrees Celsius None None None
        4 Link 1155261355 2023-03-10 01:40:08 None 9.7360 2 NaN None Degrees Celsius None None None

Components

General definitions: sta.py

Reflection of the sensorthings structure.

Construction and execution of queries: sta_requests.py

Classes and function that allow or simplify the construction requests.

General function to go from a json response to a pandas dataframe: df.py

Classes and functions to convert observations to a dataframe.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pandassta-0.0.10.tar.gz (34.8 kB view details)

Uploaded Source

Built Distribution

pandassta-0.0.10-py3-none-any.whl (28.6 kB view details)

Uploaded Python 3

File details

Details for the file pandassta-0.0.10.tar.gz.

File metadata

  • Download URL: pandassta-0.0.10.tar.gz
  • Upload date:
  • Size: 34.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.3

File hashes

Hashes for pandassta-0.0.10.tar.gz
Algorithm Hash digest
SHA256 30782a3461a7bed9d286f03e6a5253d8af2476b5e775a675e93ec817081113dd
MD5 1a4a94c63fbe37de9032821ed8227b72
BLAKE2b-256 0340b3311ea8b9bc0dc7d88ebf1dbd8865e0f9d78ffc6508846233ccf393cf27

See more details on using hashes here.

File details

Details for the file pandassta-0.0.10-py3-none-any.whl.

File metadata

  • Download URL: pandassta-0.0.10-py3-none-any.whl
  • Upload date:
  • Size: 28.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.3

File hashes

Hashes for pandassta-0.0.10-py3-none-any.whl
Algorithm Hash digest
SHA256 eb82fcaf44e2f8fb4019426476cb80196b85e7b23d85031a3c16efa9f64a06bd
MD5 1776f62e46545c73654a3be55372d1d7
BLAKE2b-256 fc5ceb4f423ad733ae83878f2c3f3a410c600cfdf1af1c47f510dc11417cfd68

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page