open-aea-cli-benchmark 2.2.1

CLI extension for AEA framework benchmarking.

AEA CLI Benchmark Plug-in

CLI extension for benchmarking the AEA framework. Adds an aea benchmark command group that runs performance benchmark cases covering agent construction, message throughput, memory usage, decision maker, ACN, and related scenarios.

Installation and usage

Make sure you have aea installed.

Then, install the plug-in:

pip install open-aea-cli-benchmark

The tx-generate case spins up a dockerised fetchai ledger node and is therefore an optional extra. Install it with:

pip install "open-aea-cli-benchmark[tx-generate]"

Without the [tx-generate] extra the tx-generate command is not registered and the rest of the benchmark commands work as normal.

Now you should be able to run aea benchmark:

Usage: aea benchmark [OPTIONS] COMMAND [ARGS]...

  Run one of the performance benchmarks.

Options:
  --help  Show this message and exit.

Commands:
  acn-communication             ACN end-to-end message throughput
  acn-startup                   ACN node startup time
  agent-construction-time       Time to construct an AEA instance
  decision-maker                Decision maker throughput
  dialogues-memory-usage        Memory footprint of dialogue state
  mem-usage                     Agent memory usage under load
  messages-memory-usage         Memory footprint of messages
  multiagent                    Multi-agent throughput
  multiagent-http-dialogues     Multi-agent HTTP dialogue throughput
  proactive                     Proactive skill behaviour benchmark
  reactive                      Reactive skill handler benchmark
  tx-generate                   Transaction generation throughput (requires the [tx-generate] extra)

Run any command with --help for per-case options.