Skip to main content

A plugin to enable threatbus communication with MISP.

Project description

Threat Bus MISP Plugin

PyPI Status Build Status License

A Threat Bus plugin that enables communication with MISP.

The plugin goes against the pub/sub architecture of Threat Bus (for now), because it actively binds to a single MISP instance to receive attribute (IoC) updates, and report back sightings via the REST API. Following the strict pub/sub architecture of Threat Bus, it should be the other way around, with MISP binding to Threat Bus. This will eventually be resolved by a MISP module.

For now, the plugin supports two ways to retrieve attribute (IoC) updates from MISP - either via ZeroMQ or via Kafka. Basically, the plugin makes itself a subscriber to MISP events.


Users can specify optional dependencies during installation. The plugin uses either ZeroMQ or Kafka to get IoC updates from MISP. As we don't want to burden the user to install unused dependencies, both options are available as follows:

pip install threatbus-misp[zmq]
pip install threatbus-misp[kafka]

If neither of these dependencies is installed (i.e., you installed threatbus-misp without the [...] suffix for optional deps), the plugin throws an error and exits immediately.

Depending on your setup, you might want to use quotes to avoid shell expansion when using [...]. For example, you can do pip install ".[zmq]" for local development.

Kafka Prerequisites

When you decide to use Kafka to receive IoC updates from MISP, you first need to install Kafka on the Threat Bus host. This plugin uses the confluent-kafka Python package which requires librdkafka. See also the prerequisites section of the confluent-kafka Python client for details about setting it up for your distribution.

Once installed, go ahead and install the Kafka version of this plugin:

pip install threatbus-misp[kafka]


The plugin uses the MISP REST API to report back sightings of IoCs. You need to specify a MISP API key for it to work.

ZeroMQ and Kafka are mutually exclusive, such that Threat Bus does not receive all attribute updates twice. See below for an example configuration.

      host: https://localhost
      ssl: false
      key: MISP_API_KEY
    filter: # filter are optional. you can omit the entire section.
      - orgs: # org IDs must be strings:
          - "1"
          - "25"
          - "TLP:AMBER"
          - "TLP:RED"
        types: # MISP attribute types
          - ip-src
          - ip-dst
          - hostname
          - domain
          - url
      - orgs:
        - "2"
      host: localhost
      port: 50000
    #  topics:
    #  - misp_attribute
    #  poll_interval: 1.0
    #  # All config entries are passed as-is to librdkafka
    #  #
    #  config:
    #    bootstrap.servers: "localhost:9092"
    # "threatbus"
    #    auto.offset.reset: "earliest"

IoC Filter

The plugin can be configured with a list of filters. Every filter describes a whitelist for MISP attributes (IoCs). The MISP plugin will only forward IoCs to Threat Bus if the whitelisted properties are present.

A filter consists of three sub-whitelists for organizations, types, and tags. To pass through the filter, an attribute must provide at least one of the whitelisted properties of each of the whitelists. More precisely, entries of each whitelist are linked by an "or"-function, the whitelists themselves are linked by an "and"-function, as follows: (org_1 OR org_2) AND (type_1 OR type_2) AND (tag_1 OR tag_2).

The MISP plugin always assumes that the absence of a whitelist means that everything is whitelisted. For example, when the entire filter section is omitted from the config, then all attributes are forwarded and nothing is filtered. More examples follow below.


Organizations are whitelisted by their ID, which is a string. Only those MISP attributes that come from any of the whitelisted organizations will be forwarded to Threat Bus.


Types can be whitelisted by specifying MISP attribute types. Only those attributes that are instances of a whitelisted type will be forwarded to Threat Bus.


MISP Attributes can be tagged with arbitrary strings. The tag whitelist respects tag names. Only those attributes that have at least one of the whitelisted tags will be forwarded to Threat Bus.


This section provides some simple configuration examples to illustrate how whitelist filtering works.

  1. Forward all IoCs from the organizations "1" and "25"
- orgs:
  - "1"
  - "25"
  1. Forward only IoCs of the domain, url, or uri type, but only if they come from the organization "1" or "25".
- orgs:
  - "1"
  - "25"
- types:
  - domain
  - url
  - uri
  1. Forward only IoCs that are tagged with TLP:RED or TLP:AMBER, but only of type "src-ip":
- tags:
  - "TLP:RED"
- types:
  - src-ip

Development Setup

The following guides describe how to set up local, dockerized instances of MISP and Kafka.

Dockerized Kafka

For a simple, working Kafka Docker setup use the single node example from confluentinc/cp-docker-images.

Store the docker-compose.yaml and modify the Kafka environment variables such that the Docker host (e.g., of your Docker machine is advertised as Kafka listener:

    KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://   # <-- That is the IP of your Docker host

For details about Kafka listeners, check out this article.

Then start the compose setup via docker-compose up -d.

To test the setup, use the tests/utils/ and tests/utils/ scripts.

Dockerized MISP

Use DCSO's dockerized MISP to set up a local testing environment:

Setup a MISP Docker cluster

git clone
cd MISP-dockerized
make install
# follow the dialog...

Edit the docker-compose.yaml

cd current
vim docker-compose.yaml

Find the section misp-server in the configuration and add the following:

      - "50000:50000"

Restart MISP to accept the new port

make deploy

Enable the Kafka plugin in the MISP web-view

  • Visit https://localhost:80
  • login with your configured credentials
  • Go to Administration -> Server Settings & Maintenance -> Plugin settings Tab
  • Set the following entries
    • Plugin.Kafka_enable -> true
    • Plugin.Kafka_brokers -> <- In this example, is the Docker host, reachable from other Docker networks. The port is reachable when the Kafka Docker setup binds to it globally.
    • Plugin.Kafka_attribute_notifications_enable -> true
    • Plugin.Kafka_attribute_notifications_topic -> misp_attribute <- The topic goes into the threatbus config.yaml

Install Kafka inside the misp-server container

docker exec -ti misp-server bash # pop interactive shell inside the container

apt-get install software-properties-common
apt-get update
# enable stretch-backports to get a recent librdkafka version
add-apt-repository "deb stretch-backports main contrib non-free"
apt-get update
apt-get install librdkafka-dev/stretch-backports
# see
pecl channel-update
pecl install rdkafka
echo "" | tee /etc/php/7.0/mods-available/rdkafka.ini
phpenmod rdkafka
service apache2 restart
exit # leave the Docker container shell

Enable the ZMQ plugin in the MISP web-view

  • Visit https://localhost:80
  • login with your configured credentials
  • Go to Administration -> Server Settings & Maintenance -> Diagnostics Tab
  • Find the ZeroMQ plugin section and enable it
  • Go to Administration -> Server Settings & Maintenance -> Plugin settings Tab
  • Set the entry Plugin.ZeroMQ_attribute_notifications_enable to true

Restart all MISP services

make restart-all


Threat Bus comes with a 3-clause BSD license.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for threatbus-misp, version 2021.2.24
Filename, size File type Python version Upload date Hashes
Filename, size threatbus_misp-2021.2.24-py3-none-any.whl (13.2 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size threatbus-misp-2021.2.24.tar.gz (17.2 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page