Python utilities for handling ESS streamed data
Project description
Python Streaming Data Types
Utilities for working with the FlatBuffers schemas used at the European Spallation Source ERIC for data transport.
https://github.com/ess-dmsc/streaming-data-types
FlatBuffer Schemas
name | description |
---|---|
hs00 | Histogram schema (deprecated in favour of hs01) |
hs01 | Histogram schema |
ns10 | NICOS cache entry schema |
pl72 | Run start |
6s4t | Run stop |
f142 | Log data (deprecated in favour of f144) |
f144 | Log data |
ev42 | Event data (deprecated in favour of ev44) |
ev43 | Event data from multiple pulses |
ev44 | Event data with signed data types |
x5f2 | Status messages |
tdct | Timestamps |
ep00 | EPICS connection info (deprecated in favour of ep01) |
ep01 | EPICS connection info |
rf5k | Forwarder configuration update (deprecated in favour of fc00) |
fc00 | Forwarder configuration update |
answ | File-writer command response |
wrdn | File-writer finished writing |
NDAr | Deprecated |
ADAr | EPICS areaDetector data |
al00 | Alarm/status messages used by the Forwarder and NICOS |
senv | Deprecated |
json | Generic JSON data |
se00 | Arrays with optional timestamps, for example waveform data. Replaces senv. |
da00 | Scipp-like data arrays, for histograms, etc. |
hs00 and hs01
Schema for histogram data. It is one of the more complicated to use schemas. It takes a Python dictionary as its input; this dictionary needs to have correctly named fields.
The input histogram data for serialisation and the output deserialisation data have the same dictionary "layout". Example for a 2-D histogram:
hist = {
"source": "some_source",
"timestamp": 123456,
"current_shape": [2, 5],
"dim_metadata": [
{
"length": 2,
"unit": "a",
"label": "x",
"bin_boundaries": np.array([10, 11, 12]),
},
{
"length": 5,
"unit": "b",
"label": "y",
"bin_boundaries": np.array([0, 1, 2, 3, 4, 5]),
},
],
"last_metadata_timestamp": 123456,
"data": np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]),
"errors": np.array([[5, 4, 3, 2, 1], [10, 9, 8, 7, 6]]),
"info": "info_string",
}
The arrays passed in for data
, errors
and bin_boundaries
can be NumPy arrays
or regular lists, but on deserialisation they will be NumPy arrays.
Developer documentation
See README_DEV.md
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ess_streaming_data_types-0.26.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 09d4acdc245847f23a6010a8f8294efb5055959822ad5e6425ced510c34be96b |
|
MD5 | a1d03c1a3d89c493cf3c5fcf5fd5102a |
|
BLAKE2b-256 | 5e5c2ac3a11d4278d4b6376f4ff68f9c30d0b8983e51f6b3adbf498c143f85fb |
Hashes for ess_streaming_data_types-0.26.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef718958ab9f8e9342c4df19e3235f34463daec0c64d102deaf3d298ef99f967 |
|
MD5 | 0011cf7357e5b9394b1fc4cbbf007857 |
|
BLAKE2b-256 | ed7803e9600f9de6b2564fb45e678e6650f4717fb67c7d307c9d1d9ce0aeb49d |