Kestrel Threat Hunting Language
Project description
Overview
Kestrel threat hunting language provides an abstraction for threat hunters to focus on what to hunt instead of how to hunt. The abstraction makes it possible to codify resuable hunting knowledge in a composable and sharable manner. And Kestrel runtime figures out how to hunt for hunters to make cyber threat hunting less tedious and more efficient.
Kestrel language: a threat hunting language for a human to express what to hunt.
expressing the knowledge of what in patterns, analytics, and hunt flows.
composing reusable hunting flows from individual hunting steps.
reasoning with human-friendly entity-based data representation abstraction.
thinking across heterogeneous data and threat intelligence sources.
applying existing public and proprietary detection logic as analytics.
reusing and sharing individual hunting steps and entire hunt books.
Kestrel runtime: a machine interpreter that deals with how to hunt.
compiling the what against specific hunting platform instructions.
executing the compiled code locally and remotely.
assembling raw logs and records into entities for entity-based reasoning.
caching intermediate data and related records for fast response.
prefetching related logs and records for link construction between entities.
defining extensible interfaces for data sources and analytics execution.
Installation
Kestrel requires Python 3.x to run. Check Python installation guide if you do not have Python. It is preferred to install Kestrel runtime using pip, and it is preferred to install Kestrel runtime in a Python virtual environment.
Update Python installer.
$ pip install --upgrade pip setuptools wheel
Install Kestrel runtime.
$ pip install kestrel-lang
Install Kestrel Jupyter kernel if you use Jupyter Notebook to hunt.
$ pip install kestrel-jupyter
$ python -m kestrel_jupyter_kernel.setup
(Optional) download Kestrel analytics examples for the APPLY hunt steps.
$ git clone https://github.com/opencybersecurityalliance/kestrel-analytics.git
Hello World Hunt
Copy the following 3-step hunt flow into your favorite text editor:
# create four process entities in Kestrel and store them in the variable `proclist`
proclist = NEW process [ {"name": "cmd.exe", "pid": "123"}
, {"name": "explorer.exe", "pid": "99"}
, {"name": "firefox.exe", "pid": "201"}
, {"name": "chrome.exe", "pid": "205"}
]
# match a pattern of browser processes, and put the matched entities in variable `browsers`
browsers = GET process FROM proclist WHERE [process:name IN ('firefox.exe', 'chrome.exe')]
# display the information (attributes name, pid) of the entities in variable `browsers`
DISP browsers ATTR name, pid
Save to a file helloworld.hf.
Execute the hunt flow in a terminal (in Python venv if virtual environment is used):
$ kestrel helloworld.hf
Now you captured browser processes in a Kestrel variable browsers from all processes created:
name pid chrome.exe 205 firefox.exe 201 [SUMMARY] block executed in 1 seconds VARIABLE TYPE #(ENTITIES) #(RECORDS) process* proclist process 4 4 0 browsers process 2 2 0 *Number of related records cached.
Hunting In The Real World
How to develop hunts interactively in Jupyter Notebook?
How to connect to one and more real-world data sources?
How to write and match a TTP pattern?
How to find child processes of a process?
How to find network traffic from a process?
How to apply pre-built analytics?
How to fork and merge hunt flows?
Find more at Kestrel documentation hub and Kestrel blogs at OCA.
Kestrel Hunting Blogs
Learning/Sharing With the Community
Talks And Demos
Kestrel was debuted at RSA Conference 2021 with its goal of an efficient cyberthreat hunting symbiosis, its key design concepts entity-based reasoning and composable hunt flow, as well as a small-enterprise APT hunting demo with TTP pattern matching, cross-host provenance tracking, TI-enrichment, machine learning analytics, and more. Watch our session The Game of Cyber Threat Hunting: The Return of the Fun (30 minutes with demo) or the demo alone (15 minutes).
Kestrel was further introduced to the threat hunting community at SANS Threat Hunting Summit 2021 in session Compose Your Hunts With Reusable Knowledge and Share Your Huntbook With the Community to facilitate huntbook composition, sharing, and reuse. The session started from 3 simple hunt step demos—TTP pattern matching, provenance tracking, and data visualization analytics—then went into comprehensive hunt flow composition to convey the idea of hunting knowledge composition and reuse. The recording is currently available at SANS library and will be published by SANS.
Kestrel will be presented as part of the open hunting stack for hybrid cloud in Black Hat Europe Arsenal 2021 session: An Open Stack for Threat Hunting in Hybrid Cloud With Connected Observability. We will hunt an APT in a hybrid cloud that is a variant of a typical supply chain attack yet implemented in a more stealthy manner. The open stack consisting of Kestrel, SysFlow, and other open-source projects will be presented.
Connecting With The Community
Quick questions? Like to meet other users? Want to contribute?
Get a slack invitation to Open Cybersecurity Alliance workspace and join our kestrel channel.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for kestrel_lang-1.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 03f3f3f16c1743cf3435f68ec1caa13def62966855cda53d3915215a059cc5d9 |
|
MD5 | 2969e7f0288e10e45c214ab7eb22337a |
|
BLAKE2b-256 | dd54148b4d508b3165dafe3d2fc2638f6bf5549a58309ead4522ca0e38e7c233 |