A pure Python tool and library to setup ADQ for Intel NICs
Project description
adqsetup
SPDX-License-Identifier: BSD-3-Clause
Copyright (c) 2022, Intel Corporation
Dependencies: Python 2.7-3.10
Distro Support:
- Redhat: 7.1-9.x
- Fedora: 28-35
- Ubuntu: 19.04-22.04
- Debian: 11
Installation
Python Package Index (pip):
python -m pip install adqsetup
Included with driver:
python scripts/adqsetup/adqsetup.py install
Usage
The basic usage is: adqsetup [options] <command> [parameters ...]
Please see the output of adqsetup help
for a complete list of
command line options.
Commands
- help: Show help message
- examples: Create an 'examples' subdirectory
The examples subdirectory - created in the current directory - contains a set of sample config files - apply {filename}: Apply a config file
- {filename}: Config file (relative or full path)
If empty or '-', config file is read from stdin.
- {filename}: Config file (relative or full path)
- create { [{name}] { {key} {value} }... }...: Create a config from the command line
Each section consisting of a bracketed name and one or more {key} {value} pairs.- [{name}]: User-defined name of section
Must be unique within a configuration, '[globals]' is reserved but can be used. - {key} {value}: Configuration Parameter
One or more space-seperated key and value pairs. See the above Class Configuration Parameter list for possible keys and values.
- [{name}]: User-defined name of section
- reset: Remove ADQ traffic classes and filters
Attempts to perform a cleanup of any ADQ-related setup. Note: '--priority=skbedit' option must be included to remove the egress filters. - persist {filename}: Persist a config file across reboots
Creates a systemd service unit set to run once on boot after the network is running. One config per network interface, new configs overwrite old ones.- {filename}: Config file (relative or full path)
If empty or '-', config file is read from stdin.
- {filename}: Config file (relative or full path)
- install: Install the adqsetup script
Installs the current script at /usr/local/bin
Configuration Parameters
Globals Section
- arpfilter: (bool) Enable selective ARP activity
- bpstop: (bool) Channel-packet-clean-bp-stop feature
- bpstop-cfg: (bool) Channel-packet-clean-bp-stop-cfg feature
- busypoll: (integer) busy_poll value
- busyread: (integer) busy_read value
- cpus: (integer list|'auto') CPUs to use for handling 'default' traffic, default 'auto'
- numa: (integer|'local'|'remote'|'all') Numa node to use for 'default' traffic, default 'all' (prefer local)
- dev: (string) Network interface device to configure
- optimize: (bool) Channel-inspect-optimize feature
- priority: ('skbedit') Method to use for setting socket priority, default none
- queues: (integer) Number of queues in 'default' traffic class, default 2
- txring: (integer) Transmit ring buffer size
- txadapt: (bool) Adaptive transmit interrupt coalescing
- txusecs: (integer) Usecs for transmit interrupt coalescing
- rxring: (integer) Receive ring buffer size
- rxadapt: (bool) Adaptive receive interrupt coalescing
- rxusecs: (integer) Usecs for receive interrupt coalescing
User-defined Section (for each application or traffic class)
- addrs: (string list) Local IP addresses of traffic
- cpus: (integer list|'auto') CPUs to use for handling traffic, default 'auto'
- mac: (string) Local MAC addresses of traffic
- mode: ('exclusive'|'shared') Mode for traffic class
- numa: (integer|'local'|'remote'|'all') Numa node to use for traffic, default 'all' (prefer local)
- pollers: (integer) Number of independent pollers, default 0
- poller-timeout: (integer) Independent poller timeout value, default 10000
- ports: (integer list) Local IP ports of traffic
- protocol: ('tcp'|'udp') IP Protocol of traffic
- queues: (integer) Number of queues in traffic class
- remote-addrs: (string list) Remote IP addresses of traffic
- remote-ports: (integer list) Remote IP ports of traffic
Sample Usage
adqsetup help
adqsetup examples
adqsetup apply memcached.conf
adqsetup --dev=eth4 apply nginx.conf
adqsetup --dev=eth3 persist eth3.conf
cat memcached.conf | adqsetup apply
adqsetup create [myapp] queues 4 ports 11211
adqsetup --verbose create \
[globals] priority skbedit \
[myapp] queues 2 ports 11211
adqsetup --verbose create \
[app1] mode shared queues 4 ports 6379-6382
[app2] queues 2 ports 11211 pollers 2
Sample Usage Bash Script
#!/bin/bash
QUEUES=8
# this will loop through a range
# of busy_poll values
for BP in {10000..50000..5000}; do
adqsetup create [globals] busypoll $BP [nginx] queues $QUEUES ports 80
# run test here
done
Sample Usage With Pipes From Bash Script
#!/bin/bash
QUEUES=8
# this will loop through a range
# of busy_poll values
for BP in {10000..20000..5000}; do
adqsetup apply <<EOF
[globals]
dev=eth2
busypoll=$BP
[nginx]
queues=$QUEUES
ports=80
EOF
# run test here
done
Sample Usage With Pipes From External Script
python makeconf.py | adqsetup --json apply
makeconf.py
import json
conf = {
"globals": {
"dev": "eth2",
"busypull": 10000
},
"app1": {
"queues": 4,
"ports": "80,443"
}
}
print(json.dumps(conf))
Notes
-
To load/use a different device driver while creating the setup, the
--driver
parameter may be used. Device driver path is the full path to the .ko file (ex: ice-1.9.x/src/ice.ko). Interface must be set to come up automatically with an ip address (via NetworkManager or other). adqsetup will wait up to three seconds for this to occur before erroring out. Conversely, you can load the driver and setup the interface manually before running the adqsetup. -
The independent pollers argument passed to adqsetup doesn’t map directly to the qps_per_poller arguments passed to the driver. adqsetup allows the user to specify how many pollers for a particular TC instead of having to specify qps_per_poller.
-
adqsetup 1.x required updated versions of the 'tc', 'ethtool', and 'devlink' commands to be installed on the system. With adqsetup 2.x and onward, this requirement has been removed.
Common Issues
-
If you get a
/usr/bin/env: ‘python’: No such file or directory
error when you run the script, please install Python. If you have already installed Python, then trywhereis python
and you should see a message like:python: /usr/bin/python2.7 /usr/bin/python3.6 /usr/bin/python3.6m /usr/bin/python3.9
on the first line of the output. Either run the version you wish to use manually:python3.6 adqsetup.py help
, or create a 'python' symbolic link on the path:ln -s /usr/bin/python3.6 /usr/local/bin/python
-
Many advanced features, such as pollers and the per-tc flow director may not be supported by older versions of the driver or kernel. adqsetup will attempt to use an equivalent fallback feature, and if none are available a descriptive error will be provided. Please refer to the ADQ Config Guide for more information.
Other Issues
Please run the malfunctioning config with the command line --debug
option,
which should include a short stack trace at the end of the output. Send the
configuration file (if used), full commmand line, and program output to your
Intel support contact.
JSON Support
adqsetup accepts configurations in the JSON format from either a file or stdin
with the --json
option. Parameters are the same as listed above, using the
following basic structure:
{
"globals": {
"dev": "eth4",
"priority": "skbedit"
},
"app1": {
"queues": 2,
"ports": 11211
},
"app2": {
"queues": 4,
"mode": "shared",
"ports": "6379-6382"
}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.