Notifier

Notifier is an application that publishes notifications to a communication system such as Matrix or, in the future, to electronic lab notebooks.

The Notifier does not produce notifications itself, but reads messages that have been published by other components of a data processing pipeline such as connectors and workers to a queue. At the moment, the actions stream of a given Asapo data source is used to store the queue of these messages. We recommend the name misc for this Asapo data source.

graph LR
  dps@{ shape: procs, label: "Data Processing Units"}
  actions_queue(["Queue of actions<br/>(Asapo stream)"])
  notifier[Notifier]
  comm_system["Communication system<br/>(e.g. Matrix)"]
  users@{ shape: procs, label: "Users"}

  actions_queue ~~~ notifier
  dps -- publish actions ---> actions_queue
  notifier -- read ---> actions_queue
  notifier -- publish notifications ---> comm_system
  comm_system -- notifies ---> users

The Notifier expects messages that describe notifications to be stored as Asapo messages metadata of the following format:

{
    "version": "0.1.0",
    "from": "<component_description>",
    "action": {"type": "alert", "message": "<message>"},
}

Getting started

To run the Notifier, you will need a device_id and access_token that correspond to a Matrix account. To create a device_id and access_token for a given account, execute the following commands:

MATRIX_USER_ID=""   # Set the corresponding user ID
MATRIX_PASSWORD=""  # Set the corresponding password

AUTH_RESPONSE="$(curl -X POST "https://matrix.org/_matrix/client/v3/login" \
     -H "Content-Type: application/json" \
     -d '{
       "type": "m.login.password",
       "identifier": {
         "type": "m.id.user",
         "user": "'$MATRIX_USER_ID'"
       },
       "password": "'$MATRIX_PASSWORD'",
       "initial_device_display_name": "bot session"
     }')"
# Result:
# {"access_token":"<token>","device_id":"<device_id>","user_id":"<MATRIX_USER_ID>"}
matrix_device_id="$(jq -r .device_id <<<"$AUTH_RESPONSE")"
matrix_access_token="$(jq -r .access_token <<<"$AUTH_RESPONSE")"
# echo "matrix_device_id='$matrix_device_id' && matrix_access_token='$matrix_access_token'"

To run the Notifier, you also need to install its dependencies:

# Create a virtual environment
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install --upgrade pip

# Install dependencies
python3 -m pip install -e .[asapo]

You can now start the Notifier with the following commands:

# The values provided below assume using the Asapo standalone Docker container.
# Edit these values to make the notifier use a different Asapo, beamline, or beamtime.
asapo_endpoint="localhost:8400"
asapo_beamline="test"
asapo_beamtime="asapo_test"
asapo_token="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.e\
yJleHAiOjk1NzE3MTAyMTYsImp0aSI6Ind0ZmlzdGhpcyIsInN\
1YiI6ImJ0X2FzYXBvX3Rlc3QiLCJFeHRyYUNsYWltcyI6eyJBY\
2Nlc3NUeXBlcyI6WyJ3cml0ZSIsIndyaXRlcmF3IiwicmVhZCJ\
dfX0.cz6R_kVf4yh7IJD6bJjDdgTaxPN3txudZx9DE6WaTtk"

# Start the Notifier app:
python3 -m notifier.app \
  --asapo.endpoint "$asapo_endpoint" \
  --beamline "$asapo_beamline" \
  --beamtime "$asapo_beamtime" \
  --asapo.token "$asapo_token" \
  --matrix.device_id "$matrix_device_id" \
  --matrix.access_token "$matrix_access_token" \
  --log_level "debug"

To test that the notifier works correctly, you can publish some notification messages to Asapo:

# The values provided below assume using the Asapo standalone Docker container.
# Edit these values to make the notifier use a different Asapo, beamline, or beamtime.
asapo_endpoint="localhost:8400"
asapo_beamline="test"
asapo_beamtime="asapo_test"
asapo_token="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.e\
yJleHAiOjk1NzE3MTAyMTYsImp0aSI6Ind0ZmlzdGhpcyIsInN\
1YiI6ImJ0X2FzYXBvX3Rlc3QiLCJFeHRyYUNsYWltcyI6eyJBY\
2Nlc3NUeXBlcyI6WyJ3cml0ZSIsIndyaXRlcmF3IiwicmVhZCJ\
dfX0.cz6R_kVf4yh7IJD6bJjDdgTaxPN3txudZx9DE6WaTtk"

# Publish messages to Asapo:
python3 -m tests.notifier.publish_to_asapo \
  --asapo.endpoint "$asapo_endpoint" \
  --beamline "$asapo_beamline" \
  --beamtime "$asapo_beamtime" \
  --asapo.token "$asapo_token" \
  --messages "tests/notifier/messages.json" \
  --log_level "debug"

As a reminder, if you want to use an Asapo standalone Docker container, execute the following command:

docker run --rm --name asapo --network host \
  gitlab.desy.de:5555/asapo/asapo/asapo-standalone:26.02.0.dev01

Contributing

If you want to contribute, please read CONTRIBUTING.md first.

In addition, you can find more information on the organization of the code in the document doc/architecture.md.