dpugen 0.2.3

Multi-vendor DPU library to generate its configuration

• dpugen
  • Use Cases
  • Installation
  • Command-line vs. Module
    • Command-line mode
    • Module Mode
• Architecture

dpugen

dpugen is a Python package. Its purpose is to generate device configurations ranging from simple to complex. Its name derives from the types of devices it was designed for: DPUs (Data Processing Units), IPUs (Infrastructure Processing Units) or SmartNICs, as the case may be.

Use Cases

The initial version supports the DASH project. Specifically, dpugen generates entries used to configure a DASH-capable device for networking services. See DASH documentation for more details.

The initial use-case focuses specifically on the DASH SAI or Switch-Abstraction Interface. SAI has been extended from traditional L2/L3 datacenter swtches and routers, to now support L4 stateful processing, offloaded by DPUs.

Installation

The easiest way to install is via Pip:

pip3 install dpugen

Command-line vs. Module

dpugen can be used either as a standalone executable or a Python module imported into another program.

Command-line mode

In command-line mode, the program can generate text or files containing JSON "records." These records comprise the configuration of the device under test (DUT). The JSON would typically be stored in a file or observed/captured on "standard out," then used by some other tool such as a test runner: the file is read, then converted into comands to configure the DUT.

When loaded from a file using a package such as Python json, the resulting data structure is a list of dictionary items, each one comprising a configuration item. This item might be one table entry, or a list of bulk-table entries.

Command-line Usage:

Assuming you've pip-installed the module:

python3 -m dpugen.sai [options]

If you've cloned the source but not installed as a module:

dpugen$ dpugen/sai.py [options]

Examples:

python3 -m dpugen.sai -h              # show help

python3 -m dpugen.sai                 # generate default JSON to stdout

python3 -m dpugen.sai -o sample.json  # generate default JSON to a file

Module Mode

When used as a module, the package is imported by another program and used to generate streaming "records" via an items() iterator method. This allows the consuming program to fetch items one at a time, in the correct order, and apply them to the device. The items fetched are dictionary structures, corresponding exactly to the equivalent JSON objects emitted in command-line mode.

Module Usage: See dpugen/examples/ for sample programs. Here is a minimal example, generating a default configuration to standard out:

#!/usr/bin/python3
# configuration using default scaling parameters.
import dpugen
from saigen.confbase import *
from saigen.confutils import *
from pprint import pprint

if __name__ == '__main__':
    # Instantiate
    conf = dpugen.sai.SaiConfig()
    conf.generate()
    pprint(conf.items())

Architecture

The figure below depicts the architecture of dpugen on the left, and potential use-cases on the right.

The top-level generator e.g. sai.py contains various sub-generators, each one of which is reponsible for generating a particular type of configuration object. The sub-generator code modules are stored under a subdirectory, e.g. the dpugen/saigen/ directory contains sub-generators for sai.py.

For example, enis.py generates ENIs (Elastic Network Interfaces), acl_groups.py generates ACLs (Acess Control lists) and ACL Groups, etc. Each sub-generator can be used independently as a module or command-line executable, but the normal practice is to use the top-level generator.

The generator can be fed scaling parameters to control the number of entities it generates. It also has defaults, allowing it to generate a configuration such as shown in the examples above with no parameters. Furthermore, you can specify things like starting IP address, address step, etc., but these lower-level parameters can usually be left to their defaults.

The output is meant to be consumed by a test runner such as Pytest, which in turn would configure a device using its native API. The SAI records are in a generic format and need to be translated into device RPC calls. This is outside the scope of this document. One example test framework which does this is SAI Challenger , and it is used in the DASH project.

As explained above, dpugen can be used in command-line mode to emit JSON records, or in module mode to be imported by another program. The diagram above shows both of these modes.