Skip to main content

Python Streaming Data Types

Utilities for working with the FlatBuffers schemas used at the European Spallation Source ERIC for data transport.

https://github.com/ess-dmsc/streaming-data-types

FlatBuffer Schemas

name description
hs00 Histogram schema (deprecated in favour of hs01)
hs01 Histogram schema
ns10 NICOS cache entry schema
pl72 Run start
6s4t Run stop
f142 Log data (deprecated in favour of f144)
f144 Log data
ev42 Event data (deprecated in favour of ev44)
ev43 Event data from multiple pulses
ev44 Event data with signed data types
an44 ev44 with ANSTO specific changes
x5f2 Status messages
tdct Timestamps
ep00 EPICS connection info (deprecated in favour of ep01)
ep01 EPICS connection info
rf5k Forwarder configuration update (deprecated in favour of fc00)
fc00 Forwarder configuration update
answ File-writer command response
wrdn File-writer finished writing
NDAr Deprecated
ADAr EPICS areaDetector data
al00 Alarm/status messages used by the Forwarder and NICOS
senv Deprecated
json Generic JSON data
se00 Arrays with optional timestamps, for example waveform data. Replaces senv.
da00 Scipp-like data arrays, for histograms, etc.
vs00 Log data à la f144, but for strings

hs00 and hs01

Schema for histogram data. It is one of the more complicated to use schemas. It takes a Python dictionary as its input; this dictionary needs to have correctly named fields.

The input histogram data for serialisation and the output deserialisation data have the same dictionary "layout". Example for a 2-D histogram:

hist = {
    "source": "some_source",
    "timestamp": 123456,
    "current_shape": [2, 5],
    "dim_metadata": [
        {
            "length": 2,
            "unit": "a",
            "label": "x",
            "bin_boundaries": np.array([10, 11, 12]),
        },
        {
            "length": 5,
            "unit": "b",
            "label": "y",
            "bin_boundaries": np.array([0, 1, 2, 3, 4, 5]),
        },
    ],
    "last_metadata_timestamp": 123456,
    "data": np.array([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]]),
    "errors": np.array([[5, 4, 3, 2, 1], [10, 9, 8, 7, 6]]),
    "info": "info_string",
}

The arrays passed in for data, errors and bin_boundaries can be NumPy arrays or regular lists, but on deserialisation they will be NumPy arrays.

Developer documentation

See README_DEV.md

Metadata

Release files for ess-streaming-data-types 0.28.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ess-streaming-data-types 0.28.0
File Size Uploaded Provenance Actions
ess_streaming_data_types-0.28.0.tar.gz 63.3 kB Without provenance Details for ess_streaming_data_types-0.28.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ess-streaming-data-types 0.28.0
File Interpreter ABI Platform Actions
ess_streaming_data_types-0.28.0-py3-none-any.whl Python 3 none any Details for ess_streaming_data_types-0.28.0-py3-none-any.whl

Total release size: 218.8 kB

Release files / ess_streaming_data_types-0.28.0.tar.gz

Download URL ess_streaming_data_types-0.28.0.tar.gz
Size 63.3 kB
Tags Source
SHA-256 checksum
How to use checksums
7bda2e61023ccdd70e95051115ee3084e36162deceda43a44362c57a5545e2ed
BLAKE2b-256 checksum
How to use checksums
7d5d5953246ea61987b2ef483b865dd38a7a2d2eee2cab01a9e7402f06475cbd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.21

Release files / ess_streaming_data_types-0.28.0-py3-none-any.whl

Download URL ess_streaming_data_types-0.28.0-py3-none-any.whl
Size 155.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b5a13415cd4987ff48b1de56930fbc9cfd486b30c14d6f58bb6539d51c1034f6
BLAKE2b-256 checksum
How to use checksums
769991d57e576d21735172f87b5ba77b851667e318410a926e6af5eae9c23fdf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.21

Security for ess-streaming-data-types 0.28.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.28.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page