tum_scout_metadata 0.8.4

Single source of truth for ESM's SCOUT measurement logistics

SCOUT Metadata

This repository handles the metadata around the SCOUT (Street-level Carbon Observatory for Urban Territory) sensor network in Munich.
We selected this format over putting it in a database due to various reasons:

• Easy to read, modify and extend by selective group members using GitHub permissions
• Changes to this are more obvious here than in database logs
• Versioning (easy to revert mistakes)
• Automatic testing of the files integrities
• Easy import as a statically typed Python library

What does this data look like?

There is a set of JSON files in the data folder holding the follwing information:

• SENSORS.json
  This files contains basic information about the sensors in use in the monitoring network.

{
    "13077": {
        "sensor_type": "LP8",
        "sensor_make": "Decentlab",
        "sensor_model": "DL-LP8",
        "start_up_date": "2022-08-01T08:00:00+00:00",
        "shut_down_date": null,
        "comment": ""
    },
}

• SITES.json
  This file contains basic information about the sites/locations where sensors have been installed.

{
    "FREV": {
        "site_type": "individual",
        "site_lat": 48.1615591,
        "site_lon": 11.5860387,
        "elevation": 514,
        "comment": "Lamp post ids:55.0"
    },
}

• SAMPLING.json
  This file contains basic information on which site, at which time, which sensors measured at which configuration. A new key "radiation_shield" was added with version 0.7 as all sensors will be deployed within radiation shieling in near future.

[
    {
        "site_id": "HANV",
        "sensor_ids": [
            13171,
            13147
        ],
        "sampling_start": "2023-05-02T12:30+02:00",
        "sampling_end": null,
        "orientation": 0,
        "elevation_ag": 3,
        "comment": "", 
        "radiation_shield": false
    },
]

How to add new measurment metadata?

1. Possibly add new sensor in data/SENSORS.json
2. Possibly add new site in data/SITES.json
3. Add a new sampling event to data/SAMPLING.json

If necessary, use this tool to find the elevation of a certain location: https://www.calcmaps.com/map-elevation/

How to use it in your codebase?

1. Install python library

poetry add tum_scout_metadata
# or
pip install tum_scout_metadata

2. Create a personal access token for a GitHub account that has read access to the metadata repository: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token

3. Use the metadata anywhere

import tum_scout_metadata

metadata_interface = tum_scout_metadata.load_from_github(
    github_repository = "org-name/repo-name",
    access_token = "ghp_..."
)

metadata = metadata_interface.get(
    sensor_id = "13077", date = pendulum.datetime(2023, 6, 6)
)  # is of type list[tum_scout_metadata.types.SensorDataContext]

metadata = interface.get(sensor_id = '13155', timestamp=pendulum.datetime(2023, 6, 6))
interfaces.print_beautiful(metadata))

... prints out:

Metadata for Sensor 13155, located at MOSV.
---
Sensor type:            Decentlab DL-LP8
Site coordinates:       48.1870436 lat
                        11.5622708 lon
                        508.0 m a.s.l.
Orientation             0.0°
Elevation above ground: 3.0 m
Comment:                Lamp post ids:32.0

---