pyvast-threatbus 2021.7.29

Connect the open source telemetry engine VAST with Threat Bus, the open source threat intelligence dissemination layer

PyVAST Threat Bus App

Threat Bus is a publish-subscribe broker for threat intelligence. It is expected that applications register themselves at the bus. Since VAST cannot do so on its own (yet), pyvast-threatbus.py implements that functionality in the meantime.

The application provides a thin layer around PyVAST, VAST's Python CLI bindings. It facilitates message exchange between Threat Bus and a VAST instance.

Installation

Install pyvast-threatbus via pip. Optionally, use a virtual environment.

virtualenv venv           # optional
source venv/bin/activate  # optional
python -m pip install pyvast-threatbus

Development

Use the dev-mode command from the Makefile to install the project in development mode. We recommend to use a virtual environment for development.

virtualenv venv
source venv/bin/activate
make dev-mode

Quick Start

You can configure the app via a YAML configuration file. See config.yaml.example for an example config file that uses fever alertify to transform sighting contexts before they get printed to STDOUT. See the section Features for details. Rename the example to config.yaml before starting.

Alternatively, configure the app via environment variables, similarly to Threat Bus, or pass a path to configuration file via -c /path/to/config.yaml.

Start the application:

pyvast-threatbus

Docker

You can also run this app via Docker.

• Build it:

  docker build . -t tenzir/pyvast-threatbus:latest
  

• Run it to print the helptext.

  docker run tenzir/pyvast-threatbus:latest
  

• Run and mount a custom config file into the container:

  docker run --net=host -v /path/to/your/conf.yaml:/opt/tenzir/threatbus/pyvast-threatbus/config.yaml tenzir/pyvast-threatbus:latest -c config.yaml
  

Features

This section explains the most important features of pyvast-threatbus.

IoC Matching

VAST can match IoCs either live or retrospectively via usual queries.

Live Matching

VAST's live matching works as continuous query. pyvast-threatbus subscribes to those continuous query results and reports all new IoC matches from VAST to Threat Bus as Sightings. You can enable live matching in the config file by setting live_match: true.

Retro Matching

pyvast-threatbus supports retro matching. You can enable it in the config file by setting retro_match: true. This instructs the application to translate IoCs from Threat Bus to normal VAST queries instead of feeding the IoCs to a live matcher.

Each result from an IoC query is treated as Sighting of that IoC and reported back to Threat Bus. You can limit the maximum amount of results returned from VAST by setting the config option retro_match_max_events to a positive integer.

Sighting Context Transformation

You can configure pyvast-threatbus to invoke another program for parsing Sighting context data via the config option transform_context.

If set, the app translates the x_threatbus_sighting_context field of a STIX-2 Sighting via the specified utility. For example, configure the app to pass the context object to DCSO/fever alertify:

...
transform_context: fever alertify --alert-prefix 'MY PREFIX' --extra-key my-ioc --ioc %ioc
...

The x_threatbus_sighting_context field can contain arbitrary data. For example, retro matches from VAST contain the full query result in the context field (like a Suricata EVE entry or a Zeek conn.log entry).

Note that the cmd string passed to transform_context is treated as template string. The placeholder %ioc is replaced with the contents of the actually matched IoC.

Custom Sinks for Sightings

pyvast-threatbus offers to send Sighting context to a configurable sink instead of reporting them back to Threat Bus. This can be configured via the sink configuration parameter. The special placeholder STDOUT can be used to print the Sighting context to STDOUT.

A custom sink is useful to forward Sightings to another process, like syslog, or forward STDOUT via a UNIX pipe. Note that it may be desirable to disable logging in that case.

Note that only the x_threatbus_sighting_context field of a STIX-2 Sighting is printed, and not the object structure of the Sighting itself.