✨ Test Cases ✨

The ByteBlower Test Framework serves as the base for many of our customer’s test cases.

At Excentis, we also develop common test cases and love to share them with the community.

• Test Case: DOCSIS 4.0 ATP

  • Guided MULPIv4.0 LL-04.40 Part 1 certification traffic test and analysis.

• Test Case: TR-398

  • Guided TR-398 Wi-Fi 802.11n/ac/ax performance validation tests

• Test Case: RFC 2544 Throughput

  • Run an RFC 2544 network performance test with ease!

• Test Case: Low Latency

  • Run low latency validation tests on your network.

📢 New since v1.4.0! 📢

It is with great pleasure that we announce our new features of the ByteBlower Test Framework!

• Test Case: DOCSIS 4.0 ATP - ✨ New! ✨

  • Guided MULPIv4.0 LL-04.40 Part 1 certification traffic test and analysis.

• Extended IPv6 support

  • Running traffic tests with IPv6 ports behind a firewall

  • Manual IPv6 address configuration

• Features for post-processing

  • Port captures

  • Sequence numbers in frames

• Several small improvements in runtime and documentation.

Features

• Quick and easy automation of traffic tests using ByteBlower

• Straightforward building blocks for your own test scripts

  • Grouped in self-explaining categories

  • Designed with a small 😉 to the workflow you are used to from the GUI

• Command-line interface

  • Run traffic tests with nothing more than a JSON configuration file!

• Supported ByteBlower endpoint types

  • ByteBlower Port (on a physical interface of a ByteBlower server)

  • ByteBlower Endpoint (mobile testing via app on your device)

• Endpoint configuration

  • IPv4

    • DHCP and manual address configuration (ByteBlower Port only)

    • Automatic handling of endpoints located behind a NAT/NAPT gateway!

  • IPv6

    • DHCPv6, SLAAC and manual address configuration (ByteBlower Port only)

    • ByteBlower Endpoint: Automatic resolving of Endpoint host’s IPv6 address for each traffic flow.

• Traffic types

  • Stateless: frame blasting (UDP-based)

  • Stateful: HTTP (TCP-based)

  • Define IP Traffic class: DSCP & ECN

    • TCP Prague for Low Latency, Low Loss, and Scalable Throughput (L4S)

• Application simulations

  • Voice over IP (VoIP) using the G.711 codec

  • Traditional gaming (ByteBlower Endpoint only as destination)

  • Video streaming (Netflix, YouTube, …) (ByteBlower Port only)

• Standard analysis

  • Frame count: transmitted and received over time, frame loss and byte loss

  • Latency: Minimum, maximum, average and jitter over time

  • TCP

    • Average goodput on HTTP (layer 5)

    • Transmitted and received Bytes over time

    • TCP average throughput over time

    • Minimum, maximum and average Round Trip Time (RTT)

    • TCP Retransmission counts

    • ECN Congestion Experienced (CE) markings (for L4S-enabled TCP flows)

  • Aggregation of results over multiple flows (frame blasting only)

  • PASS/FAIL criteria can be provided to match your KPIs

  • Post-processing support - Port captures - Frame sequence number injection

• Application-specific analysis

  • Mean Opinion Score (MOS): Specific for Voice flows

  • Video buffer analysis: Specific for Video flows (ByteBlower Port only)

• Reporting

  • Summary and realtime results

  • HTML: Neat to share with your chief, customers, vendors, …

    • Incorporating our brand new style!

    • Interactive charts

    • Includes overview of all latency CCDF results

  • JSON: Allows for machine post-processing

  • XML JUnit: Useful for integration in automation tools

• Helpers

  • Ease-of-use for configuration and/or post-processing of the analyzed results

• Example scripts

Changelog

For all details, please have a look at our our Changelog.

Supported platforms

The ByteBlower Test Framework in general supports Python version 3.7 to 3.11.

The framework has been tested for the following operating system platforms and Python versions:

Prepare runtime environment

We recommend managing the runtime environment in a Python virtual environment. This guarantees proper separation of the system-wide installed Python and pip packages.

Install the ByteBlower Test Framework from PyPI

First make sure that you have activated your virtual environment:

1. On Unix-based systems (Linux, WSL, macOS):

   . ./.venv/bin/activate

   Note: For macOS: later on, when generating ByteBlower HTML reports, you might see SSL errors like:

   ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
   urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)>

   You can fix this by installing or updating the CA certificates for your Python version. In your activated Python virtual environment in a macOS Terminal, type (for example, for Python 3.12):

   open /Applications/Python\ 3.12/Install\ Certificates.command

2. On Windows systems using PowerShell:

   & ".\.venv\Scripts\activate.ps1"

Now install (or update) the ByteBlower Test Framework:

pip install -U byteblower-test-framework

Command-line interface

To get help for command line arguments:

1. As a command-line script:

   byteblower-test-framework --help

2. As a python module:

   python -m byteblower_test_framework --help

For a quick start, you can run a simple test using the JSON configuration of one of the example files below:

• Test scenario for ByteBlower Ports

• Test scenario for ByteBlower Endpoint

Save you configuration in your working directory as byteblower_test_framework.json. Please make sure you change the server and ports configuration according to the setup you want to run your test on.

The byteblower_test_framework.json can be used then to run the test in the command line interface using:

byteblower-test-framework

The resulting reports will be saved into the current directory.

To specify a different config file name and report path using:

byteblower-test-framework --config-file path/to/my_test_config.json  --report-path path/to/my_test_reports_directory

You can find more details on how to customize your own configuration file in Configuration file.