A plugin to enable threatbus communication with MISP.
Project description
Threat Bus MISP Plugin
A Threat Bus plugin that enables communication to MISP.
The plugin goes against the pub/sub architecture of Threat Bus (for now), because the plugin subscribes a listener to ZeroMQ / Kafka, rather than having MISP subscribe itself to Threat Bus. That will be addressed with a MISP module in the near future.
Installation
pip install threatbus-misp
Prerequisites
Install Kafka on the Threat Bus host
The plugin enables communication either via ZeroMQ or Kafka. When using Kafka,
you have to install librdkafka
for the host system that is running
threatbus
. See also the prerequisites
section of the confluent-kafka
python client.
Configuration
The plugin can either use ZeroMQ or Kafka to retrieve intelligence items from MISP. It uses the MISP REST api to report back sightings of indicators.
ZeroMQ and Kafka are mutually exclusive, such that Threat Bus does not receive all attribute updates twice. See below for an example configuration.
...
plugins:
misp:
api:
host: https://localhost
ssl: false
key: MISP_API_KEY
zmq:
host: localhost
port: 50000
#kafka:
# topics:
# - misp_attribute
# poll_interval: 1.0
# # All config entries are passed as-is to librdkafka
# # https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
# config:
# bootstrap.servers: "localhost:9092"
# group.id: "threatbus"
# auto.offset.reset: "earliest"
...
Development Setup
The following guides describe how to set up local, dockerized instances of MISP and Kafka.
Dockerized Kafka
For a simple, working Kafka Docker setup use the single node example
from confluentinc/cp-docker-images
.
Store the docker-compose.yaml
and modify the Kafka environment variables such
that the Docker host (e.g., 172.17.0.1
) of your Docker machine is advertised
as Kafka listener:
zookeeper:
...
kafka:
...
environment:
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://172.17.0.1:9092 # <-- That is the IP of your Docker host
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
...
For details about Kafka listeners, check out this article.
Then start the compose setup via docker-compose up -d
.
To test the setup, use the tests/utils/kafka_receiver.py
and
tests/utils/kafka_sender.py
scripts.
Dockerized MISP
Use DCSO's dockerized MISP to set up a local testing environment:
Setup a MISP Docker cluster
git clone git@github.com:DCSO/MISP-dockerized.git
cd MISP-dockerized
make install
# follow the dialog...
Edit the docker-compose.yaml
cd current
vim docker-compose.yaml
Find the section misp-server
in the configuration and add the following:
misp-server:
...
ports:
- "50000:50000"
...
Restart MISP to accept the new port
make deploy
Enable the Kafka plugin in the MISP webview
- Visit https://localhost:80
- login with your configured credentials
- Go to
Administration
->Server Settings & Maintenance
->Plugin settings Tab
- Set the following entries
Plugin.Kafka_enable
->true
Plugin.Kafka_brokers
->172.17.0.1:9092
<- In this example, 172.17.0.1 is the Docker host, reachable from other Docker networks. The port is reachable when the Kafka Docker setup binds to it globally.Plugin.Kafka_attribute_notifications_enable
->true
Plugin.Kafka_attribute_notifications_topic
->misp_attribute
<- The topic goes into the threatbusconfig.yaml
Install Kafka inside the misp-server
container
docker exec -ti misp-server bash # pop interactive shell inside the container
apt-get install software-properties-common
apt-get update
# enable stretch-backports to get a recent librdkafka version
add-apt-repository "deb http://deb.debian.org/debian stretch-backports main contrib non-free"
apt-get update
apt-get install librdkafka-dev/stretch-backports
# see https://misp.github.io/MISP/INSTALL.ubuntu1804/#misp-has-a-feature-for-publishing-events-to-kafka-to-enable-it-simply-run-the-following-commands
pecl channel-update pecl.php.net
pecl install rdkafka
echo "extension=rdkafka.so" | tee /etc/php/7.0/mods-available/rdkafka.ini
phpenmod rdkafka
service apache2 restart
exit # leave the Docker container shell
Enable the ZMQ plugin in the MISP webview
- Visit https://localhost:80
- login with your configured credentials
- Go to
Administration
->Server Settings & Maintenance
->Diagnostics Tab
- Find the ZeroMQ plugin section and enable it
- Go to
Administration
->Server Settings & Maintenance
->Plugin settings Tab
- Set the entry
Plugin.ZeroMQ_attribute_notifications_enable
totrue
Restart all MISP services
make restart-all
License
Threat Bus comes with a 3-clause BSD license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for threatbus_misp-2020.7.28-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9f117815234bc3c3d98ec1499e160c236415dd484c4d3254a40ac5ea8dd378bc |
|
MD5 | 9a75927be4486772cde62f96e28b889a |
|
BLAKE2b-256 | ffe13d2a33989b5e2945734de8a353d0d4eaf345a6bd29c45bfe0dc1b8436bec |