Welcome to Assonant 🎶 𝄇

A beamline-agnostic event processing engine for data collection and organization

Description

A Beamline-agnostic event processing engine for data collection and organization (Assonant).

This package provides to beamline groups from the Brazilian Synchrotron Light Laboratory (LNLS) a python API to send metadata, produced during an experiment, to a data management system, responsible to mantain and organize the metadata produced at Sirius.

The Assonant package is composed by public and private modules. Private modules are used by Assonant developers to implemented features that is furtherly exposed through public modules to abstract from final user how to handle specific platforms behaviors. In other words, if you are a beamline developer, don't care about what is inside any private module. Public modules are designed for being used by beamline developers in order to allow easy usage of Assonant resources. To differ private and public modules, private modules names are preceded by a '_':

• 🔒 Private Modules 🔒

  • _kafka

    Assonant tools for dealing with anything related to Kafka.

  • _nexus

    Assonant tools for dealing with anything related to NeXus format.

• 🔓 Public Modules 🔓

  • data_classes

    A group of data classes that defines data schemas to be used all over Assonant modules in order to standardize data acquisiton, transference, manipulation and storage.

  • file_writer

    Assonant tools for writing data from within Assonant Data Classes into specific file format

  • data_logger

    Assonant tools for logging collect data.

  • data_retriever

    Assonant tools for retrieving data from different sources.

  • data_sender

    Assonant tools for dealing with Assonant Data Classes transference over different types of communication method (e.g: Kafka topics).

Code development standards

1. When importing anything from within a module to use internally on that module use relative paths.

2. When importing anything from a module, do not use an absolute path, instead use what is being made public by the module.

3. Classes, methods and settings that will be usable from outside the module, should be exposed on __init__.py files

4. Private methods, files and modules should always have their name preceded by '_'.

Users

Assonant is currently published at the public PyPi server meaning it can be simply installed using pip. It has also been modularized to allow installing the specific dependecies for its features. that said, depending on your use case you will run different command lines:

• To install Assonant DataLogger module dependencies, run the following command line:

python -m pip install assonant[data-logger]

• To install Assonant to access the naming standards defined in it, run the following command line:

python -m pip install assonant[naming-standards]

• To install Assonant to access the path builder submodule, run the following command line:

python -m pip install assonant[path-builder]

• To install all Assonant dependecies, run the following command line:

python -m pip install assonant[full]

Developers

1. First of all, If you want to prepare a development environment, clone the repository into your local machine:

2. Secondly install assonant by executing pip install . inside the cloned repository

python -m pip install -e .[dev]

3. After that install the pre-commit modules. duf

pre-commit install --install-hooks -t pre-commit -t commit-msg

Deploy

1. Firstly, remember to update the version value in this README file badge, and on the pyproject.toml file.

2. Secondly, you will need to build your package:

python -m build

3. After that you need to send the new version to the Global PyPi version by executing the following command line. Exchange the <package_version> place holder by the current version that will be deployed:

twine upload --repository pypi dist/assonant-<package_version>* --verbose

Note: To deploy a new version you will need a PyPi token which only the project maintainers have. That said, if you are a project maintainer and don't have access to it, ask your leader for it.

Mantainers

• 👤 Allan Pinto
• 👤 Paulo B. Mausbach

Credits

• Apache Kafka
• Confluent's Kafka Python Client
• Dockerfile for Apache Kafka
• JSON schema for NeXus files
• NeXus Data Format
• Nexusformat (python package)
• Pydantic

License

TODO: Define a License