Python utilities for handling ESS streamed data
Project description
Python Streaming Data Types
Utilities for working with the FlatBuffers schemas used at the European Spallation Source ERIC for data transport.
https://github.com/ess-dmsc/streaming-data-types
FlatBuffer Schemas
name | description | verifiable* |
---|---|---|
hs00 | Histogram schema | Y |
ns10 | NICOS cache entry schema | Y |
pl72 | Run start | N |
6s4t | Run stop | N |
f142 | Log Data | Y |
ev42 | Event Data | Y |
x5f2 | Status messages | N |
tdct | Timestamps | Y |
* whether it passes verification via the C++ FlatBuffers library.
hs00
Schema for histogram data. It is one of the more complicated to use schemas. It takes a Python dictionary as its input; this dictionary needs to have correctly named fields.
The input histogram data for serialisation and the output deserialisation data have the same dictionary "layout". Example for a 2-D histogram:
hist = {
"source": "some_source",
"timestamp": 123456,
"current_shape": [2, 5],
"dim_metadata": [
{
"length": 2,
"unit": "a",
"label": "x",
"bin_boundaries": np.array([10, 11, 12]),
},
{
"length": 5,
"unit": "b",
"label": "y",
"bin_boundaries": np.array([0, 1, 2, 3, 4, 5]),
},
],
"last_metadata_timestamp": 123456,
"data": np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]),
"errors": np.array([[5, 4, 3, 2, 1], [10, 9, 8, 7, 6]]),
"info": "info_string",
}
The arrays passed in for data
, errors
and bin_boundaries
can be NumPy arrays
or regular lists, but on deserialisation they will be NumPy arrays.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ess_streaming_data_types-0.7.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6e34f02d2db915f83cf2e02fc17755a7ecf661904c149210a605654946f80b9 |
|
MD5 | 3ff7065e2765c6389bb3aee1164d0cbe |
|
BLAKE2b-256 | 64b6c4bf836e949cad872e5348cd15b6b75ec4700d459e1c9d8572cbef74b089 |
Hashes for ess_streaming_data_types-0.7.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 51045036be31c551432cb23979f15a066f08603ba6e8277e6ac6643f0efe3bba |
|
MD5 | da6a0c900bc09794e22e2cac8ac9f4d6 |
|
BLAKE2b-256 | 597c302273c3a687fc74fc1262df38a081c30345741ca0d91614b57d7b2f533a |