Oslo messaging benchmarking orchestrator
Message bus evaluation framework
This is a framework to benchmark the communication middleware supported by oslo.messaging. It's primary goal is to address the evaluation of https://docs.openstack.org/performance-docs/latest/test_plans/massively_distribute_rpc/plan.html.
It is build on top of:
- EnOSlib. This library helps to describe the experimental workflow and enforce it : from the deployment to the performance metrics analysis.
- ombt. This will coordinate the benchmark once all the agents are up and running.
From a high level point of view the framework is able to deploy
- a communication bus (e.g RabbitMQ, qdr aka qpid-dispatch-router),
- a set of client/server that will communicate
- start a benchmark while gathering metrics
A typical test consists in the following components:
Client 1---------+ +----------------------+ +-----> Server 1 | | | | +----> | Communication | ----+-----> Server 2 Client 2--------------> | Middleware | | +----> | (e.g qdr, rabbitms) | | ... | | | | | +----------------------+ +------> Server n Client n---------+ | / \ / \ | / \ -- -- -- -- -- Monitoring -- -- -- -- --
- Clone the repository:
git clone https://github.com/msimonin/ombt-orchestrator cd ombt-orchestrator
- Install the dependencies
pip install -U pip pip install -e .
On Grid'5000 you can launch this command from any frontend.
The default configurations are currently defined in the
Command line interface
> oo Usage: oo [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: backup Backup environment logs [after test_case_*]. campaign Perform a TEST according to the (swept)... deploy Claim resources from a PROVIDER and configure... destroy Destroy all the running dockers (keeping... g5k Claim resources on Grid'5000 (frontend). inventory Generate the Ansible inventory [after g5k,... prepare Configure available resources [after g5k,... test_case_1 Run the test case 1: one single large... test_case_2 Run the test case 2: multiple distributed... test_case_3 Run the test case 3: one single large... test_case_4 Run the test case 4: multiple distributed... vagrant Claim resources on vagrant (localhost).
Workflow to run a test case
- Deploying and launching the benchmark (default driver
brokeris defined in the configuration file)
# default confs.yaml on $PWD will be read > oo deploy --driver=broker vagrant # Launch the one benchmark > oo test_case_1 --nbr_clients 10 --nbr_servers 2
Adapt to the relevant provider (e.g
- Real-time metrics visualisation
Grafana is available on the port 3000 of the control node (check the inventory file).
- Backuping the environment
> oo backup
The files retrieved by this action are located in
current/backupdir by default.
- Some cleaning and preparation for the next run
# Preparing the next run by cleaning the environment > oo destroy > oo deploy vagrant # Next run > oo test_case_1 --nbr_clients 20 --nbr_servers 2
It's possible to force an experimentation dir with
Note also that scripting from python is also possible using the function defined in
## Workflow to run a campaign
- A campaign is a batch execution of several configurations for a given test case. Deployment and execution of a benchmark is read from a configuration file. For example, to run the first test case enabled on the framework run:
> oo campaign --provider g5k test_case_1
Alternatively a campaign can be executed in a incremental mode in which deployments are performed only when a different
call_typeis defined. Incremental campaigns are executed with a different semantic on the parameters defined in the configuration. With the incremental option the semantics is based on the combination of parameters by means of a dot product between a set of them in the configuration file (i.e., a
zipoperation between the lists of parameters). These parameters are defined by test case as follows:
- Test case 1:
- Test case 2:
- Test case 3:
- Test case 4:
- Test case 1:
To execute an incremental campaign be sure to use the ombt version
msimonin/ombt:singletoninstead of the default and execute:
> oo campaign --incremental --provider g5k test_case_1
- Bound clients or servers to specific bus agents:
To bind ombt-clients to a specific bus instance you can declare the following
roles: [bus, bus-client].
Following the same idea ombt-servers can be bound to a specific bus instance using
roles: [bus, bus-server]
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for ombt_orchestrator-1.1.1-py2.py3-none-any.whl