High level description

A complete application capable of testing performance using MLRsearch consists of three layers: Manager, Controller and Measurer. This library provides an implementation for the Controller only, including all the classes needed to define API between Controller and other two components.

Users are supposed to implement the whole Manager layer, and also implement the Measurer layer. The Measurer instance is injected as a parameter when the manager calls the controller instance.

The purpose of Measurer instance is to perform one trial measurement. Upon invocation of measure() method, the controller only specifies the intended duration and the intended load for the trial. The call is done using keyword arguments, so the signature has to be:

def measure(self, intended_duration, intended_load):

Usually, the trial measurement process also needs other values, collectively caller a traffic profile. User (the manager instance) is responsible for initiating the measurer instance accordingly. Also, the manager is supposed to set up SUT, traffic generator, and any other component that can affect the result.

For specific input and output objects see the example below.

Example

This is a minimal example showing every configuration attribute. The measurer does not interact with any real SUT, it simulates a SUT that is able to forward exactly one million packets per second (unidirectional traffic only), not one packet more (fully deterministic). In these conditions, the conditional throughput for PDR happens to be accurate within one packet per second.

This is the screen capture of interactive python interpreter (wrapped so long lines are readable):

>>> import dataclasses
>>> from MLRsearch import (
...     AbstractMeasurer, Config, MeasurementResult,
...     MultipleLossRatioSearch, SearchGoal,
... )
>>>
>>> class Hard1MppsMeasurer(AbstractMeasurer):
...     def measure(self, intended_duration, intended_load):
...         sent = int(intended_duration * intended_load)
...         received = min(sent, int(intended_duration * 1e6))
...         return MeasurementResult(
...             intended_duration=intended_duration,
...             intended_load=intended_load,
...             offered_count=sent,
...             forwarding_count=received,
...         )
...
>>> def print_dot(_):
...     print(".", end="")
...
>>> ndr_goal = SearchGoal(
...     loss_ratio=0.0,
...     exceed_ratio=0.005,
...     relative_width=0.005,
...     initial_trial_duration=1.0,
...     final_trial_duration=1.0,
...     duration_sum=21.0,
...     preceding_targets=2,
...     expansion_coefficient=2,
... )
>>> pdr_goal = dataclasses.replace(ndr_goal, loss_ratio=0.005)
>>> config = Config(
...     goals=[ndr_goal, pdr_goal],
...     min_load=1e3,
...     max_load=1e9,
...     search_duration_max=1.0,
...     warmup_duration=None,
... )
>>> controller = MultipleLossRatioSearch(config=config)
>>> result = controller.search(measurer=Hard1MppsMeasurer(), debug=print_dot)
....................................................................................
....................................................................................
...................>>> print(result)
{SearchGoal(loss_ratio=0.0, exceed_ratio=0.005, relative_width=0.005, initial_trial_
duration=1.0, final_trial_duration=1.0, duration_sum=21.0, preceding_targets=2, expa
nsion_coefficient=2, fail_fast=True): fl=997497.6029392382,s=(gl=21.0,bl=0.0,gs=0.0,
bs=0.0), SearchGoal(loss_ratio=0.005, exceed_ratio=0.005, relative_width=0.005, init
ial_trial_duration=1.0, final_trial_duration=1.0, duration_sum=21.0, preceding_targe
ts=2, expansion_coefficient=2, fail_fast=True): fl=1002508.6747611101,s=(gl=21.0,bl=
0.0,gs=0.0,bs=0.0)}
>>> print(f"NDR conditional throughput: {float(result[ndr_goal].conditional_throughp
ut)}")
NDR conditional throughput: 997497.6029392382
>>> print(f"PDR conditional throughput: {float(result[pdr_goal].conditional_throughp
ut)}")
PDR conditional throughput: 1000000.6730730429
>>>