lucihub-common 0.1.1

Common data models and storage components for LuciHub

Models

This component contains data models that are used to communicate between at least two different components of the project.

This includes the following models:

• database: models/classes for interfacing with hot and cold storage
  • hot: use plain Python classes that abstract away the path variables and include static types
  • cold: data models in SQL alchemy

Note that the hot storage essentially caches parts of the data structures of the cold storage. This should be reflected in the SQL alchemy model:

Principles

• make sure to use object-oriented expressons and techniques, for example abstract models and classes which have different implementations to account for versioning.
• consider the fact that the API itself is mostly consisting of CRUD-type of endpoints
• for hot storage, favor small field with primitive data types
• for cold storage, prefer large objects that enable fast data read without too many joins when reading/writing through the API endpoints

Modelled data

This is a very rough first draft which does not correspond to an 1:1 ORM:

Cold storage

• run
  • id
  • type of job (compile, run, verify...)
  • state changes (array of timestamp/status)
  • logs
  • number of channels
  • measurements (per channel) for run/verify jobs
  • device id
  • input type
  • input (JSON data)
  • output (JSON data)
• device
  • id
  • type
  • input type
  • parameter schema (as a data type that will simplify the model checking later)

Hot storage

• run (by UUID)
  • state changes (array of pairs timestamp/status)
  • logs (array of lines, lines are appended)
  • measurements (on different channels, each channel is a time series of floating point values)
• list of active devices (device IDs), regularly updated from scanner