A plugin to connect apps via ZeroMQ.
Project description
Threat Bus App Plugin for ZMQ Protocol
A Threat Bus plugin that enables communication with any application that can communicate via [ZeroMQ].
Installation
pip install threatbus-vast
Configuration
The plugin uses ZeroMQ to communicate with applications, like the VAST bridge. The plugin serves three ZeroMQ endpoints to connect with. One endpoint for managing subscriptions (and thus snapshot requests). The other two endpoints exist for pub-sub operations.
...
plugins:
zmq-app:
host: "127.0.0.1"
manage: 13370
pub: 13371
sub: 13372
...
Initially, apps that want to connect with this plugin only need to know the
manage
endpoint. The plugin and the app negotiate all other internals for
pub-sub message exchange at runtime. See the protocol definition below for
details.
Management Protocol
Subscriptions and unsubscriptions are referred to as management
. All
management messages are JSON formatted and exchanged via the manage
ZMQ
endpoint that the plugin exposes.
The manage endpoint uses the
ZeroMQ Request/Reply
pattern for message exchange. That means, all messages get an immediate response
from the server. With each JSON reply, the zmq-app Threat Bus plugin sends a
status
field that indicates the success of the requested operation.
Subscribe at the Plugin
To subscribe to Threat Bus via the zmq-app plugin, an app needs to send a JSON
struct as follows to the manage
endpoint of the plugin:
{
"action": "subscribe",
"topic": <TOPIC>, # either 'threatbus/sighting' or 'threatbus/intel'
"snapshot": <SNAPSHOT> # number of days for a snapshot, 0 for no snapshot
}
In response, the app will either receive a success
or error
response.
-
Error response:
{ "status": "error" }
-
Success response:
{ "topic": <P2P_TOPIC>, "pub_endpoint": "127.0.0.1:13371", "sub_endpoint": "127.0.0.1:13372", "status": "success", }
The
pub_endpoint
andsub_endpoint
of the reply provide the endpoints that an app should connect with to participate in the pub/sub message exchange. The app can access these endpoints following the ZeroMQ Pub/Sub pattern. The plugin will publish messages on thepub_endpoint
and accept messages on thesub_endpoint
.The
topic
field of the response contains a unique topic for the client. That topic must be used to receive messages. The unique topic is a 32 characters wide random string and guarantees that other subscribers won't accidentally see traffic that should only be visible to the new client.For more details see below at
Pub/Sub via ZeroMQ
.
Unsubscribe from the Plugin
To unsubscribe, a connected app has to send a JSON struct to the manage
endpoint of the plugin, as follows:
{
"action": "unsubscribe",
"topic": <P2P_TOPIC> # the 32-characters random topic that the app got during subscription handshake
}
In response, the app will either receive a success
or error
response.
- Error response:
{ "status": "error" }
- Success response:
{ "status": "success" }
Heartbeats
The plugin supports synchronous heartbeats from subscribed apps. Both, Threat Bus and the connected apps benefit from heartbeats, they can mutually ensure that the connected party is still alive.
Subscribed apps can send heartbeat messages with the following JSON format to
the manage
endpoint of this plugin:
{
"action": "heartbeat",
"topic": <P2P_TOPIC> # the 32-characters random topic that the app got during subscription handshake
}
As stated in the beginning of this section, the manage
endpoint implements the
ZeroMQ Request/Reply
pattern. Threat Bus answers immediately to each heartbeat request with either a
success
or error
response.
- Error response:
{ "status": "error" }
- Success response:
{ "status": "success" }
An error
response indicates that either Threat Bus has internal erros or that
it lost track of the app's subscription. Note: This only happens when Threat Bus
is restarted. Apps can then use that information to re-subscribe.
If Threat Bus does not answer a heartbeat message, it is either down or not reachable (e.g., due to network issues). Plugins can use that information to try again later.
Pub/Sub via ZeroMQ
Once an app has subscribed to Threat Bus via using the manage
endpoint of the
zmq-app plugin, it has a unique, random p2p_topic
(see above). Threat Bus
(the zmq-app) uses this topic to publish messages to the plugin. Messages can be
of the types specified in threatbus.data
, i.e. Intel
, Sighting
,
SnapshotRequest
, and SnapshotEnvelope
.
ZeroMQ uses prefix matching for pub/sub connections. The zmq-app plugin leverages this feature to indicate the type of each sent message to the subscriber. Hence, app can simply match the topic suffix to determine the message type.
For example, all Intel
items will always be sent on the topic
p2p_topic + "intel"
, all SnapshotRequest
items on the topic
p2p_topic + "snapshotrequest"
and so forth.
Snapshots
Threat Bus' snapshot
feature allows apps to request a snapshot during subscription. Threat Bus always
forwards snapshot requests to all app plugins. The zmq-app
plugin again
forwards these SnapshotRequests
to all connected apps. Apps, however, can
decide if they want to implement this feature, i.e., whether they respond to
SnapshotRequest
messages or not.
SnapshotRequests
are received like any other messages, via the p2p_topic
. In
case the app wants to provide this feature, it must handle message of this type
(see above for an explanation of topic suffix matching).
Once the snapshot is handled, the app must use the SnapshotEnvelope
message
type to send back messages to the plugin.
License
Threat Bus comes with a 3-clause BSD license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for threatbus-zmq-app-2020.11.26.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b243b82c35928d99a980037181b8792212d45204b2661d455e6fe251f5165e42 |
|
MD5 | 1fd12aa5cd4391b96f6eb11ffb3441ce |
|
BLAKE2b-256 | 5e2a78adc931352cb7ce8714a97d63b7bd0b8c3a5f471e655436fa6036c5f968 |
Hashes for threatbus_zmq_app-2020.11.26-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 79f23204f9476ae2b898270eccb6f6f03058e8cd4b9a0527e5bb82ac72a796f4 |
|
MD5 | 5fc9fb272da50faced278bd1e98e7d68 |
|
BLAKE2b-256 | 11756de736732235a1fac44ffd2d1947ad383b19c44fa57c76e22a362197bac1 |