No project description provided
Project description
NeXus Streamer
Streams event and metadata from a NeXus file into Kafka, mimicking data acquisition from a live instrument. This facilitates testing software which consume these data.
This Python implementation is intended to replace a C++ implementation (https://github.com/ess-dmsc/NeXus-Streamer) and should be much lower effort to maintain. The C++ implmentation is not yet obsolete; features missing from this implementation are documented in tickets.
Installation
Python 3.7 or higher is required. https://www.python.org/downloads/
To install from PyPi do
pip install nexus-streamer
or to install with conda (does not work on Windows due to confluent-kafka package not supporting Windows)
conda install -c conda-forge -c ess-dmsc nexus-streamer
and check installation was successful by running
nexus_streamer --help
on Windows you may need to add your Python environment's Script
directory to PATH
for the command to work.
Usage
usage: nexus_streamer [-h] [--version]
[--graylog-logger-address GRAYLOG_LOGGER_ADDRESS]
[--log-file LOG_FILE] [-c CONFIG_FILE]
[-v {Trace,Debug,Warning,Error,Critical}] -f
FILENAME [--json-description JSON_DESCRIPTION]
-b BROKER -i INSTRUMENT [-s] [-z]
NeXus Streamer Args that start with '--' (eg. --version) can also be set in a
config file (specified via -c). Config file syntax allows: key=value,
flag=true, stuff=[a,b,c] (for details, see syntax at https://goo.gl/R74nmi).
If an arg is specified in more than one place, then commandline values
override environment variables which override config file values which
override defaults.
optional arguments:
-h, --help show this help message and exit
--version Print application version and exit [env var: VERSION]
--graylog-logger-address GRAYLOG_LOGGER_ADDRESS
<host:port> Log to Graylog [env var:
GRAYLOG_LOGGER_ADDRESS]
--log-file LOG_FILE Log filename [env var: LOG_FILE]
-c CONFIG_FILE, --config-file CONFIG_FILE
Read configuration from an ini file [env var:
CONFIG_FILE]
-v {Trace,Debug,Warning,Error,Critical}, --verbosity {Trace,Debug,Warning,Error,Critical}
Set logging level [env var: VERBOSITY]
-f FILENAME, --filename FILENAME
NeXus file to stream data from [env var: FILENAME]
--json-description JSON_DESCRIPTION
If provided use this JSON template instead of
generating one from the NeXus file [env var:
JSON_FILENAME]
-b BROKER, --broker BROKER
<host[:port]> Kafka broker to forward data into [env
var: BROKER]
-i INSTRUMENT, --instrument INSTRUMENT
Used as prefix for topic names [env var: INSTRUMENT]
-s, --slow Stream data into Kafka at approx realistic rate (uses
timestamps from file) [env var: SLOW]
-z, --single-run Publish only a single run (otherwise repeats until
interrupted) [env var: SINGLE_RUN]
Developer information
See README-dev.md
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for nexus_streamer-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ba955ceee1715c844d8a859578628fdb6e22d19ab34d66eead42c79d5c5bdd92 |
|
MD5 | 90ffd8eadf41457f49292f945d2e49a6 |
|
BLAKE2b-256 | 286ef6aa60b6f42b65e69887d938e32b44cf224a4420e533ab80c160c34d5407 |