Netzob 2.0.0

Protocol Reverse Engineering, Modeling and Fuzzing

See complete documentation here: https://netzob.github.io/netzob/

About Netzob

Netzob is an open source tool for reverse engineering, modelization, traffic generation and fuzzing of communication protocols.

Netzob is suitable for reversing network protocols, structured files and system and process flows (IPC and communication with drivers and devices). Netzob handles different types of protocols: text protocols (like HTTP and IRC), delimiter-based protocols, fixed fields protocols (like IP and TCP) and variable-length fields protocols (like TLV-based protocols).

Netzob can be used to infer the message format and the state machine of a protocol through passive and active processes. Its objective is to bring state of art academic researches to the operational field, by leveraging bio-informatic and grammatical inferring algorithms in a semi-automatic manner.

Once modeled or inferred, a protocol model can be used in our traffic generation engine, to allow simulation of realistic and controllable communication endpoints and flows.

Main features of Netzob

The main features of Netzob are:

Protocol Modelization

Netzob includes a complete model to represent the message format (aka its vocabulary) and the state machine of a protocol (aka its grammar).

Protocol Inference

The vocabulary and grammar inference component provides both passive and active reverse engineering of communication flows through automated and manuals mechanisms.

Traffic Generation

Given vocabulary and grammar models previously inferred or modelized, Netzob can understand and generate communication traffic with remote peers. It can thus act as either a client, a server or both.

Protocol Fuzzing

Netzob helps security evaluators by simplifying the creation of fuzzers for proprietary or undocumented protocols. Netzob considers the format message and state machine of the protocol to generate optimized and specific test cases. Both mutation and generation are available for fuzzing.

Import Communication Traces

Data import is available in two ways: either by leveraging the channel-specific captors (currently network and IPC – Inter-Process Communication), or by using specific importers (such as PCAP files, structured files and OSpy files).

Export Protocol Models

This module permits to export an model of a protocol in formats that are understandable by third party software or by a human. Current work focuses on export format compatible with main traffic dissectors (Wireshark and Scapy) and fuzzers (Peach and Sulley).

Netzob must be used as a Python 3 library. It can either be imported in your scripts or in your favorite interactive shell (ipython?).

More Information

Website:

https://github.com/netzob/netzob

Twitter:

Follow Netzob’s official accounts (@Netzob)

Netzob has been initiated by security auditors of AMOSSYS and the CIDre research team of CentraleSupélec to address the reverse engineering and fuzzing of communication protocols.

Documentation

The documentation is available online at: https://netzob.github.io/netzob/

If you want to build the documentation, run the following command:

$ sphinx-build -b html doc/documentation/source/ doc/documentation/build/

Get Started with Netzob

Install it

Installing Netzob system dependencies

First thing to do is to check the version of your python3 interpretor. Netzob requires at least Python 3.8:

$ python3 --version
Python 3.8.10

You have to install the following system dependencies:

$ apt-get install -y python3 python3-dev python3-setuptools virtualenv build-essential libpcap-dev libgraph-easy-perl libffi-dev

Then, create a virtualenv:

$ mkdir venv
$ virtualenv venv
$ source venv/bin/activate

Installing Netzob from Pypi

You can install Netzob from Pypi (recommended choice):

(venv) $ pip3 install netzob

Installing Netzob from sources

If you have retrieve Netzob sources, the installation procedure is:

(venv) $ pip3 install Cython==0.29.32  # Should be manually installed because of setup.py direct dependency
(venv) $ pip3 install -e .

API usage

Once installed, we recommend to use the Netzob API inside scripts, with the following statement to import Netzob:

from netzob.all import *

Start Netzob CLI

Netzob also provides its own CLI, in order to play interactively with it:

(venv) $ netzob

Contributing

There are multiple ways to help-us.

Defects and Features Requests

Help-us by reporting bugs and requesting features using the Bug Tracker.

Join the Development Team

To participate in the development, you need to get the latest version, modify it and submit your changes.

You’re interested in joining, please contact us!

Authors, Contributors and Sponsors

See the top distribution file AUTHORS.txt for the detailed and updated list of authors, contributors and sponsors.

Licenses

This software is provided under the GPLv3 License. See the COPYING.txt file in the top distribution directory for the full license text.

The documentation is under the CC-BY-SA licence.

Extra

Zoby, the official mascot of Netzob.

NEWS

v1.0.2 – 2017-04-30

Version name:

StompingFrilledShark

• major improvement

  • global improvement of code architecture and organization

  • enable custom operations on field values

  • a list of valid bytes can be specified to specify a subset of values a domain accepts

  • add SendReceived() method in all channels to simplify trafic generation

  • add traffic rate and duration constraints in channels and abstraction layer

  • new channel ‘RawEthernetClient’ to send raw Ethernet frames

  • new channel ‘IPClient’ to send raw IP frames

  • the Protocol class can load ZDL format and automata files

  • introduces Pseudo Field: a field used in the computation of another field but dont produce real content

• minor improvement

  • pcap importer can merge consecutives messages with same source and destination to mimic a flow

  • improve unit-tests on relationship finder

  • an AbstractMessage stores the type of message

  • remove typechecking in channel write() methods to allow better performances

  • presets can be used when using AbstractionLayer

  • increase default maximum data size to 65535 * 8 (i.e. 65535 bytes)

• major bug fix

  • fix symbol display when messages includes exotic encoded bytes

  • fix InternetChecksum computation

  • fix size field computation to allow payload of size > 23535 bytes

  • fix size field computation that depends on Repeat variable

  • fix InternetChecksum CRC computation to support bitarrays structures

• minor bug fix

  • remove nbUnits from Integer class

  • remove the layer attribute of Fields

  • fix the identification of data-type relationships

  • convert snippets in the documentation to python3

v1.0.1 – 2017-03-05

Version name:

StompingFrilledShark

• minor improvement

  • Improves code style (mostly PEP8 constraints)

  • File Importer

  • On-demand performance tests for parallel alignment (de)activation

  • Ensures determinism of ClusterBySize clustering method

  • Base64 and Zlib Encoding functions

• minor bug fix

  • Updates Copyright dates in source headers

  • Removes obsolete Rep module

v1.0 – 2017-02-03

Version name:

StompingFrilledShark

• major improvement

  • “FlowParser” to parse a succession of message made of different symbols

  • large test campaign created to ensure netzob’s quality

  • migration to python3

  • add SSL client and server channels

  • improve package hierarchy

• minor improvement

  • enable travis supervision

  • measure test coverage

  • Entropy measurement methods

  • support for timestamps detection

  • improve size field detection algorithm

  • enable RAWIP channels

  • network pcap importer supports ICMP message

• major bug fix

  • various bug fixes

  • multiple bug fixes in C alignment methods

• minor bug fix

  • fix Symbol comparison methods

  • fix logging verbosity

  • remove useless codes and resources

v0.4.1 – 2013-02-02

Version name:

WaddlingPeccary

• Export plugins

  • Automatic generation of Wireshark dissectors

  • Automatic generation of Peach fuzzers

• Workspaces and projects

  • Workspace manager

  • Project manager

  • Trace manager

• Pretty print of XML files

• Simplify the default Variable

• Provide extra compile arguments to the build process

v0.4 – 2012-11-15

Version name:

JumpingRhino

• User interface

  • New user-friendly graphical interface

  • Port Netzob to GTK+3

  • Allow specification of logging level in the UI

• New plugin architecture

• Internationalization of Netzob

• Vocabulary inference

  • Support of layers

  • Support customized transformation functions

  • Provide the edition of a variable

  • Support IPv4, MAC and random binary variables

  • Support filters for displayed messages

  • Allow export of a selection of fields as a new symbol

• Import

  • Importer for OSpy projects

  • Allow user to specify the import layer (2,3 or 4) while importing network messages

  • Allow to keep delimiter while file importing. Indicate the position of the delimiter

• Automatic Bug Reporter

v0.3.3 – 2012-06-06

Version name:

FlyingRazorback

• Graphical interface

  • Visualization and encoding filters

  • Mathematical filters (Base64, GZIP, BZ2)

  • Dedicated Search View

  • Preview of data rendering in contextual menu

  • Support format visualization at the symbol level

• Partitioning

  • Alignment and sequencing by field

  • Execute alignment on specified symbols

  • Split field by the right

  • Allow the partitioning of messages with specified boundaries

  • Allow partitioning at the project and symbol level

  • Similarity score based on number of common dynamic elements

  • Optimization of Needleman : don’t repeat the same computation twice

  • Implement native UPGMA algorithm

• Grammar inference

  • Infer the grammar of a network client

• Project/trace management

  • Export / Import projects

  • Importer for XML formated traces

v0.3.2 – 2012-02-23

• Upgrade Vocabulary Inference

  • Add Octal visualization

    • Feature #57: Resize columns

    • Feature #59: Allows to copy message/field to clipboard

    • Feature #60: Support simple alignment

    • Feature #62: Allow the deletion of multiple messages at a time

    • Feature #20: Show the current status of an alignment

    • Manual modification of the Regex of a field

  • Upgrade Grammar Inference

    • Feature #55: Dedicated GUI for the automatic inferring process

  • Upgrade Simulator

    • Feature #87: Specify source port for network simulator

• Upgrade Import/Export and Traces Management

  • Feature #22: Activate the management of traces

  • Feature #61: Traces must be compressed when stored in the trace manager

  • Feature #92: Handle cooked socket (SLL) packet format

  • Feature #83: Support of human readable format export

  • Support Unicode for filenames

• Extra

  • Workspace can be specified through a command line argument

  • Feature #73: Add manpage for Netzob

  • Feature #74: Add “.desktop” file in the official version

  • Apply pep8 quality repository on source code

v0.3.1 – 2012-01-12

• Small fixes

v0.3 – 2012-01-12

• Upgraded GUI and user experience

  • Add a menu

  • Simplify the Vocabulary inference panel

  • Add Workspaces and Projects definitions

• Upgrade Vocabulary Inference

  • Add alignment based on an arbitrary delimitor

  • Identification of the definition domain of a field

  • Add support for environmental dependencies

  • Add new visualization of data encoding

    • Format: hex, string and binary

    • Unit size: bit, 8-bits, 16-bits, 32-bits and 64-bits

    • Sign: signed and unsigned

    • Endianness: big and little endian

  • Add concept of variable :

    • Include Binary Value

    • Include Word Value

    • Include Aggregate Value

    • Include Alternate Value

• Add grammar inference module

  • Add the definition of the MMSTD model

  • Implementation of the Angluin L* algorithm

  • Implementation of the W-Method Algorithm

  • Add an alpha version of the automatic inferring process

• Add simulation module

  • Supports Network Server and Client simulations

• Add import modulesfiles and library calls

  • Add multiple files import

• Extra

  • SVN to GIT migration

  • Dedicated website (http://www.netzob.org)

v0.2 – 2011-09-01

• Add import modules : IPC, PCAP and Live network flows

• Add export module : raw XML format

• Improvement of Needleman and Wunsh performance with OpenMP

v0.1 – 2011-08-16

• Initial release