Skip to main content

Automation and Scaling of Digital Forensics Tools

Project description

Turbinia

Unit tests e2e tests

Summary

Turbinia is an open-source framework for deploying, managing, and running distributed forensic workloads. It is intended to automate running of common forensic processing tools (i.e. Plaso, TSK, strings, etc) to help with processing evidence in the Cloud, scaling the processing of large amounts of evidence, and decreasing response time by parallelizing processing where possible.

How it works

Turbinia is composed of different components for the client, server and the workers. These components can be run in the Cloud, on local machines, or as a hybrid of both. The Turbinia client makes requests to process evidence to the Turbinia server. The Turbinia server creates logical jobs from these incoming user requests, which creates and schedules forensic processing tasks to be run by the workers. The evidence to be processed will be split up by the jobs when possible, and many tasks can be created in order to process the evidence in parallel. One or more workers run continuously to process tasks from the server. Any new evidence created or discovered by the tasks will be fed back into Turbinia for further processing.

Communication from the client to the server is currently done with either Google Cloud PubSub or Kombu messaging. The worker implementation can use either PSQ (a Google Cloud PubSub Task Queue) or Celery for task scheduling.

The main documentation for Turbinia can be found here. You can also find out more about the architecture and how it works here.

Status

Turbinia is currently in Alpha release.

Installation

There is an installation guide here.

Usage

The basic steps to get things running after the initial installation and configuration are:

  • Start Turbinia server component with turbiniactl server command
  • Start Turbinia API server component with turbiniactl api_server command if using Celery
  • Start one or more Turbinia workers with turbiniactl celeryworker if using Celery, or turbiniactl psqworker if using PSQ
  • Install turbinia-client via pip install turbinia-client
  • Send evidence to be processed from the turbinia client with turbinia-client submit ${evidencetype}
  • Check status of running tasks with turbinia-client status

turbinia-client can be used to interact with Turbinia through the API server component, and here is the basic usage:

$ turbinia-client -h
Usage: turbinia-client [OPTIONS] COMMAND [ARGS]...

  Turbinia API command-line tool (turbinia-client).

                          ***    ***
                           *          *
                      ***             ******
                     *                      *
                     **      *   *  **     ,*
                       *******  * ********
                              *  * *
                              *  * *
                              %%%%%%
                              %%%%%%
                     %%%%%%%%%%%%%%%       %%%%%%
               %%%%%%%%%%%%%%%%%%%%%      %%%%%%%
  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  ** *******
  %%                                                   %%  ***************
  %%                                (%%%%%%%%%%%%%%%%%%%  *****  **
    %%%%%        %%%%%%%%%%%%%%%
    %%%%%%%%%%                     %%          **             ***
       %%%                         %%  %%             %%%           %%%%,
       %%%      %%%   %%%   %%%%%  %%%   %%%   %%  %%%   %%%  %%%       (%%
       %%%      %%%   %%%  %%%     %%     %%/  %%  %%%   %%%  %%%  %%%%%%%%
       %%%      %%%   %%%  %%%     %%%   %%%   %%  %%%   %%%  %%% %%%   %%%
       %%%        %%%%%    %%%       %%%%%     %%  %%%    %%  %%%   %%%%%

  This command-line tool interacts with Turbinia's API server.

  You can specify the API server location in ~/.turbinia_api_config.json

Options:
  -c, --config_instance TEXT  A Turbinia instance configuration name.
                              [default: (dynamic)]
  -p, --config_path TEXT      Path to the .turbinia_api_config.json file..
                              [default: (dynamic)]
  -h, --help                  Show this message and exit.

Commands:
  config    Get Turbinia configuration.
  evidence  Get or upload Turbinia evidence.
  jobs      Get a list of enabled Turbinia jobs.
  result    Get Turbinia request or task results.
  status    Get Turbinia request or task status.
  submit    Submit new requests to the Turbinia API server.

Check out the turbinia-client documentation page for a detailed user guide.

You can also interact with Turbinia directly from Python by using the API library. We provide some examples here

Other documentation

Obligatory Fine Print

This is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

turbinia-20240614.tar.gz (309.2 kB view details)

Uploaded Source

Built Distribution

turbinia-20240614-py3-none-any.whl (493.4 kB view details)

Uploaded Python 3

File details

Details for the file turbinia-20240614.tar.gz.

File metadata

  • Download URL: turbinia-20240614.tar.gz
  • Upload date:
  • Size: 309.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.4

File hashes

Hashes for turbinia-20240614.tar.gz
Algorithm Hash digest
SHA256 2f19599ca0dcc4f7e6ba208260175c5ca976aea7344d9e509bef0d4a3201a219
MD5 00b3a92ade0a8cd9c9966f836d113fbb
BLAKE2b-256 f3a4417835bc2cf3352677c3a107a8963a20aafda7fddd66581d26b1d9124218

See more details on using hashes here.

File details

Details for the file turbinia-20240614-py3-none-any.whl.

File metadata

  • Download URL: turbinia-20240614-py3-none-any.whl
  • Upload date:
  • Size: 493.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.0.0 CPython/3.12.4

File hashes

Hashes for turbinia-20240614-py3-none-any.whl
Algorithm Hash digest
SHA256 e8bc583b90766f2654f7d2a3476434607b4e60f60c76c87c892cb2cf0d54a532
MD5 fed0eaf3d255148ac4de02b6593b806e
BLAKE2b-256 e0e4463e540420e3a9a9960f2e2ccb06ca66aa26129494b2ab1ad8e464347bfd

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page