A small Python library for controlling the ESS kafka-to-nexus file-writer.
Project description
File-writer-control
This is a small library for controlling the ESS HDF5/NeXus file-writer application. The file-writer is controlled by sending commands to it via an Apache Kafka broker. This library implements the encoding/decoding of commands as well as an abstraction of this interface in order to simplify the commands and control.
Getting started
Documentation on the usage of this library can be found in the examples directory in the root of this repository. For getting started, look no further than the following code snippet:
from file_writer_control.WorkerCommandChannel import WorkerCommandChannel
from file_writer_control.WriteJob import WriteJob
from file_writer_control.JobHandler import JobHandler
from datetime import datetime, timedelta
import time
command_channel = WorkerCommandChannel("dmsc-kafka01:9092/command_topic")
job_handler = JobHandler(worker_finder=command_channel)
write_job = WriteJob(nexus_structure="{...}", "file.nxs", "dmsc-kafka01:9092", datetime.now())
start_handler = job_handler.start_job(write_job)
while not start_handler.is_done():
time.sleep(1)
stop_handler = job_handler.set_stop_time(datetime.now() + timedelta(seconds=60))
while not stop_handler.is_done():
time.sleep(1)
while not write_job.is_done():
time.sleep(1)
print("Write job is done")
Installing dependencies
This library uses the kafka-python library for the communication with the Kafka broker and the python-streaming-data-types for serialising and de-serialising messages to and from the filewriter. These dependencies can be installed by executing the following command in the root of the repository:
pip install -r requirements.txt
Alternatively, to install the dependencies such that they are only available to the current user, execute the following command:
pip install --user -r requirements.txt
Note: At the time of this writing, the python-streaming-data-types_ library version required by this library has not been released. If you want to use this library, you will have to manually (and locally) install the filewriter_ctrl branch of that library.
Also note: This library was developed using Python 3.8 but it is likely that it will work with Python 3.6 and up.
Running the unit tests
For running the unit tests, execute the following command in the root of this repository:
python -m pytest -s .
Installing the development version locally
First, uninstall any existing versions of this library:
pip uninstall file-writer-control
Then, from the file-writer-control root directory, run the following command:
pip install --user -e ./
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for file_writer_control-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 11914504988e08049732d753b28d565110fffdf42d491f7d9cd9f0b0a228a76b |
|
MD5 | 2c60726502c4038e23353a712bde4e68 |
|
BLAKE2b-256 | 9b5f903e141c3bd7c9a866e9a38bf75452ba99a2f9c1577b2c50a21b9310a470 |
Hashes for file_writer_control-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee8c3031cd3426418babe050420c325a2833c2ecb948ab1e8545116fcfb1325d |
|
MD5 | e72a20972c366cb25a6d28a560513ce3 |
|
BLAKE2b-256 | 557b7c49001af1ad625226ec170402529e1e46f7a8db1ca362cba3b93cd8f047 |