Skip to main content

Extract and aggregate IOCs from threat feeds.

Project description

Developed by InQuest Build Status Documentation Status Code Health Test Coverage PyPi Version

An extendable tool to extract and aggregate IOCs from threat feeds.

Integrates out-of-the-box with ThreatKB and MISP, and can fit seamlessly into any existing worflow with SQS, Beanstalk, and custom plugins.

Overview

ThreatIngestor can be configured to watch Twitter, RSS feeds, or other sources, extract meaningful information such as malicious IPs/domains and YARA signatures, and send that information to another system for analysis.

ThreatIngestor flowchart with several sources feeding into multiple operators.

Try it out now with this quick walkthrough, or see more ThreatIngestor walkthroughs on the InQuest blog.

Installation

ThreatIngestor requires Python 3.6+, with development headers.

Install ThreatIngestor from PyPI:

pip install threatingestor

Install optional dependencies for using some plugins, as needed:

pip install threatingestor[all]

View the full installation instructions for more information.

Usage

Create a new config.yml file, and configure each source and operator module you want to use. (See config.example.yml for layout.) Then run the script:

threatingestor config.yml

By default, it will run forever, polling each configured source every 15 minutes.

View the full ThreatIngestor documentation for more information.

Plugins

ThreatIngestor uses a plugin architecture with “source” (input) and “operator” (output) plugins. The currently supported integrations are:

Operators

View the full ThreatIngestor documentation for more information on included plugins, and how to create your own.

Threat Intel Sources

Looking for some threat intel sources to get started? InQuest has a Twitter List with several accounts that post C2 domains and IPs: https://twitter.com/InQuest/lists/c2-feed. Note that you will need to apply for a Twitter developer account to use the ThreatIngestor Twitter Source. Take a look at config.example.yml to see how to set this list up as a source.

For quicker setup, RSS feeds can be a great source of intelligence. Check out this example RSS config file for a few pre-configured security blogs.

Support

If you need help getting set up, or run into any issues, feel free to open an Issue. You can also reach out to @InQuest on Twitter.

We’d love to hear any feedback you have on ThreatIngestor, its documentation, or how you’re putting it to work for you!

Contributing

Issues and pull requests are welcomed. Please keep Python code PEP8 compliant. By submitting a pull request you agree to release your submissions under the terms of the LICENSE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for threatingestor, version 1.0.0b6
Filename, size File type Python version Upload date Hashes
Filename, size threatingestor-1.0.0b6-py2.py3-none-any.whl (37.3 kB) File type Wheel Python version py2.py3 Upload date Hashes View hashes
Filename, size threatingestor-1.0.0b6.tar.gz (30.0 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page