kestrel-lang 1.2.0

Kestrel Threat Hunting Language

News

Log4Shell detection:

• The complete Kestrel Log4Shell huntbook

• The Log4Shell analytics to download for the huntbook

Overview

Kestrel threat hunting language provides an abstraction for threat hunters to focus on what to hunt instead of how to hunt. The abstraction makes it possible to codify resuable hunting knowledge in a composable and sharable manner. And Kestrel runtime figures out how to hunt for hunters to make cyber threat hunting less tedious and more efficient.

• Kestrel language: a threat hunting language for a human to express what to hunt.

  • expressing the knowledge of what in patterns, analytics, and hunt flows.

  • composing reusable hunting flows from individual hunting steps.

  • reasoning with human-friendly entity-based data representation abstraction.

  • thinking across heterogeneous data and threat intelligence sources.

  • applying existing public and proprietary detection logic as analytics.

  • reusing and sharing individual hunting steps and entire hunt books.

• Kestrel runtime: a machine interpreter that deals with how to hunt.

  • compiling the what against specific hunting platform instructions.

  • executing the compiled code locally and remotely.

  • assembling raw logs and records into entities for entity-based reasoning.

  • caching intermediate data and related records for fast response.

  • prefetching related logs and records for link construction between entities.

  • defining extensible interfaces for data sources and analytics execution.

Installation

Kestrel requires Python 3 to run. Check Python installation guide if you do not have Python. It is preferred to install Kestrel runtime using pip, and it is preferred to install Kestrel runtime in a Python virtual environment.

$ pip install --upgrade pip setuptools wheel
$ pip install kestrel-jupyter
$ python -m kestrel_jupyter_kernel.setup
$ jupyter notebook

Hello World Hunt

1. Copy the following 3-step hunt flow into your favorite text editor:

# create four process entities in Kestrel and store them in the variable `proclist`
proclist = NEW process [ {"name": "cmd.exe", "pid": "123"}
                       , {"name": "explorer.exe", "pid": "99"}
                       , {"name": "firefox.exe", "pid": "201"}
                       , {"name": "chrome.exe", "pid": "205"}
                       ]

# match a pattern of browser processes, and put the matched entities in variable `browsers`
browsers = GET process FROM proclist WHERE [process:name IN ('firefox.exe', 'chrome.exe')]

# display the information (attributes name, pid) of the entities in variable `browsers`
DISP browsers ATTR name, pid

2. Save to a file helloworld.hf.

3. Execute the hunt flow in a terminal (in Python venv if virtual environment is used):

$ kestrel helloworld.hf

Now you captured browser processes in a Kestrel variable browsers from all processes created:

       name pid
 chrome.exe 205
firefox.exe 201

[SUMMARY] block executed in 1 seconds
VARIABLE    TYPE  #(ENTITIES)  #(RECORDS)  process*
proclist process            4           4         0
browsers process            2           2         0
*Number of related records cached.

Hunting In The Real World

1. How to develop hunts interactively in Jupyter Notebook?

2. How to connect to one and more real-world data sources?

3. How to write and match a TTP pattern?

4. How to find child processes of a process?

5. How to find network traffic from a process?

6. How to apply pre-built analytics?

7. How to fork and merge hunt flows?

Find more at Kestrel documentation hub and Kestrel blogs at OCA.

Kestrel Hunting Blogs

1. Building a Huntbook to Discover Persistent Threats from Scheduled Windows Tasks

2. Practicing Backward And Forward Tracking Hunts on A Windows Host

3. Building Your Own Kestrel Analytics and Sharing With the Community

4. Setting Up The Open Hunting Stack in Hybrid Cloud With Kestrel and SysFlow

Learning/Sharing With the Community

• Kestrel huntbook repo

• Kestrel analytics repo

Talks And Demos

Kestrel was debuted at RSA Conference 2021: The Game of Cyber Threat Hunting: The Return of the Fun with the goal of an efficient cyberthreat hunting symbiosis, its key design concepts entity-based reasoning and composable hunt flow, and a small-enterprise APT hunting demo with TTP pattern matching, cross-host provenance tracking, TI-enrichment, machine learning analytics, and more.

Kestrel was further introduced to the threat hunting community at SANS Threat Hunting Summit 2021 in session Compose Your Hunts With Reusable Knowledge and Share Your Huntbook With the Community to facilitate huntbook composition, sharing, and reuse—from simple single hunt step demos (TTP pattern matching, provenance tracking, and data visualization analytics) to complex comprehensive hunt flow composition.

Kestrel, together with STIX-shifter, Elastic, and SysFlow constitute the open hunting stack demoed at Black Hat Europe 2021: An Open Stack for Threat Hunting in Hybrid Cloud With Connected Observability. A supply chain attack variant across a hybrid cloud (two clouds and on-premises machines) was hunted in the arsenal session.

Kestrel was demoed at Infosec Jupyterthon 2021 in session: Reason Cyber Campaigns With Kestrel. The live hunting demo explained the basics of Kestrel throughout the discovery of the hybrid cloud APT campaign developed for our Black Hat Europe 2021 session.

Connecting With The Community

Quick questions? Like to meet other users? Want to contribute?

Get a slack invitation to Open Cybersecurity Alliance workspace and join our kestrel channel.