cligen 0.6.5

generate __main__.py with argparse setup generated from YAML

cligen

cligen is is a utility to generate command-line parsing code, code for two column help output, and command-line completion code. It does so from a specification for sub-commands, options, and arguments, in YAML. The generated code can be pure Python, Zig code for the command-line parsing then calling Python (i.e. Zig for generating help or command-line completion), or Zig (0.15+) only code.

example

For a commandline utility direction that needs the subcommands left and right, and where the subcommand left can have the option --u-turn (assuming you drive on the right side of the road), and both subcommands could have a --verbose option, the specification would look like:

!Cli 0:
- !Instance driving.Direction
- !Option [verbose, v, !Help increase verbosity level, !Action count]
- left:
  - !Help turning to the left
  - !Option [u-turn, U, !Type bool, !Help make a U-turn]
- right:
  - !H turning to the right

With the result that direction left -h will show:

usage: direction left [-h] [--u-turn] [--verbose]

optional arguments:
  -h, --help     show this help message and exit
  --u-turn, -U   make a U-turn
  --verbose, -v  increase verbosity level

When direction left is called from the command-line, the code in __main__.py will create an instance of the class Direction imported from driving.py, providing the result of the parsing as argument to the intialisation, and then call the method left_subcommand or method left (trying in that order) of that class. cligen can alternatively generate code that calls functions imported from a Python file, or call code inserted from the YAML specification in __main__.py itself (in this case the ..._subcommand is not tried, and the results of parsing passed in to the function).

YAML

The YAML document uses various tags, many of which have a short version (e.g. !H is equivalent to using !Help).

Having the commandline options and argument data in a programmatically modifiable format like YAML, makes it more easy to check or manipulate all your utilities. E.g. if you want to make sure that all utilities that have a --verbose option also have a --quiet option that decreases the verbosity level. In principle any other format that has similar capabilities could be used as input, but YAML's tagging and well defined deduplication (anchor/alias) help keeping non-trivial specifications managable.

Python

The YAML document can either be in a file cli.yaml on its own, or, if you work with Python and want to diminish file clutter in your project root, it can be stored in the variable _cligen_data in __init__.py, this means between the following two lines in that file:

_cligen_data = """\
"""

Invoking cligen should recognise your project as a Python project, if it doesn't cligen python forces pure Python output.

Zig calling Python

The main advantage of Zig calling Python is in the area of command-line completion. cligen generated code for completion invokes the program to fullfil the completion. As the Zig part of the program will not invoke the Python shared object, the completion can be much faster than if Python code would be loaded (or even do the completion).

The same options for normal Python (calling methods on an instantiated class, calling functions in a module, etc are available when calling Python from Zig.

The completion code will only be included if the sequence that is the value for the root level !Cli node contains the element - !Complete

The code generation and compilation can be achieved by doing cligen zig, or cligen zig --subtype python, when autodetection is not available.

pure Zig

Zig only processing was first implemented to generate a program, zigclc capable of full command-line completion of the zig compiler itself. zig has several subcommand, some with (dynamic) sub-subcommands, so this required several changes, and some new, zig specific, extensions (using the user definable extension mechanism for types, actions, restrictions, and dynamic completion generation).

Assume the following example specification cli.yaml:

!Cli 0:
- !Source cli.zig
- !Instance
- !Complete
- !Opt [verbose, !Action count, !Help Increase program verbosity]
- init:
  - !Option
    - type
    - T
    - !Restriction [basic: minimal example code, complex: full-fledged example code]
    - !Help type of utility
  - !Arg [name]
- gen:
  - zig:  # sub-sub cmd
    - !Option [version, !Help Zig version to use]
  - !Option [fast, !Type bool]

This defines two subcommands, with the second having a sub-subcommand. After invoking cligen zig the parsing, help and command-line completion source code will be in the file cli.zig.

In your zig file you have to import the source file and call its command_line_parser() function:

const std = @import("std");
const cli = @import("cli.zig");

pub fn main(init: std.process.Init) !void {
    const args = try cli.command_line_parser(init);
    switch (args) {
        ._base => |cmd| std.debug.print("verbose {?}\n", .{cmd.verbose}),
        .init => |cmd| std.debug.print("verbose: {?}, type: '{?s}'\n", .{cmd.verbose, cmd.type}),
        .gen => |cmd| std.debug.print("verbose: {?}, fast: {?}\n", .{cmd.verbose, cmd.fast}),
        .gen_zig => |cmd| std.debug.print("verbose: {?}, fast: {?}, version: '{?s}'\n", .{cmd.verbose, cmd.fast, cmd.version}),
    }
}

The line - !Instance in the spec indicates that you process the values set for the options/arguments, by switching on the result of command_line_parser().

Alternatively you can replace - !Instance with - !StructFunction '{}_sub_command' and have a processed by calling function on a structure you pass to command_line_parser():

const std = @import("std");
const cli = @import("cli.zig");

const Dispatcher = struct {
    out: *std.Io.Writer,

    pub fn init(writer: *std.Io.Writer) Dispatcher {
        return .{
            .out = writer,
        };
    }

    pub fn _base_sub_command(self: *Dispatcher, verbose: ?i32) !void {
        try self.out.print("verbose: {?}\n", .{verbose});
    }
    pub fn init_sub_command(self: *Dispatcher, verbose: ?i32, typ: ?[]const u8, name: ?[]const u8) !void {
        try self.out.print("verbose: {?}, type: '{?s}', name: '{?s}'\n", .{verbose, typ, name});
    }
    pub fn gen_sub_command(self: *Dispatcher, verbose: ?i32, fast: ?bool) !void {
        try self.out.print("verbose: {?}, fast: {?}\n", .{verbose, fast});
    }
    pub fn gen_zig_sub_command(self: *Dispatcher, verbose: ?i32, fast: ?bool, version: ?[]const u8) !void {
        try self.out.print("verbose: {?}, fast: {?}, version: {?s}\n", .{verbose, fast, version});
    }
};

pub fn main(init: std.process.Init) !void {
    var std_out_buf: [1024]u8 = undefined;
    var std_out_writer = std.Io.File.stdout().writer(init.io, &std_out_buf);
    const args = try cli.command_line_parser(init);
    var disp = Dispatcher.init(&std_out_writer.interface);
    try args.struct_function(&disp);
    _ = &std_out_writer.interface.flush();
}

Feature list

• multiple long (--date) and short options (-D) can be associated with one target.

• additional smart types/actions. E.g. an option with type 'date' defaults to datetime.date.today() and you can specify an argument like yesterday or -w-2 (two weeks ago)

• default values, with optionally some of the defaults defaults updated from a configuration file (YAML/INI/PON)

• nested parsers for subcommands, and subcommands inheriting options from their parent. This allows you to insert parent options before or after the subcommand, without the cligen code needing to re-arrange the commandline.

• an optional default subcommand/parser, which is used if unknown options/arguments are found.

• Optional shorthands in the form of alternative executable names for often used subcommand and/or options strings.

• no dependencies on anything but the Python standard library, unless your config file is in a format that requires some installed library ( YAML -> ruamel.yaml )

• allow YAML aliases for tagged scalars to be used by other tags:

  - !Help &xhelp text1 text2 text3  # this is the same as: "&xhelp !Help text1 text2 text3"
  - !Prolog [*xhelp]                # xhelp is a tagged scalar, "!Prolog *xhelp" would error
                                    # the value for !Prolog is automatically un-sequenced
  

Using !Config

In its most explicit form the tag !Config can takes a two element sequence as value. The first element indicates the type (pon, yaml, ini, TBI: json), the second the path to the file. A path starting with a tilde (~) will be expanded. A path not starting with tilde, or (forward-)slash (/), will be appended to your users config directory.

If !Config is followed by a scalar that looks like a path (i.e. the value starts with ~ or includes a /) the extension of the path is taken to be the type. In other cases !Config is assumed to be followed by a type and the basename is derived from the package name (_package_data['full_package_name']) in your users config directory.

A user config directory is based on XDG config locations (on Windows, the config information is expected under %APPDATA%)

When !Config is specified the inserted code will check for --config some_explicit_path on the commandline and load the config data from the path specified.

config file format

Config files are assumed to contain a dictionary at the root level (for formats like .ini the data is converted to a dictionary during loading). This dictionary contains keys that correspond to the various subparsers. A section global (or optionally glbl in PON to prevent use of a reserved keyword, renamed to global after loading), is used for defaults for options that come before the subparser as well as for global options. Each section consists of key-value pairs, where the key corresponds to a long option (--verbose) or if that is not available the short option (-v), either without the leading dashes.

Assuming you have the following configuration:

!Cli 0:
- !Opt [verbose, v, !H increase verbosity, !Action count]
- !Config [pon, /usr/local/etc/myutil.pon]
- subp1: []

your myutil.pon can use:

dict(glbl=dict(verbose=2))

to set the verbosity (you might want to format your PON somewhat nicer.

The same with a YAML file:

!Cli 0:
- !Opt [verbose, v, !H increase verbosity, !Action count]
- !Config YAML
- subp1: []

your ~/.config/your_util_name/your_util_name.yaml would be:

global:
  verbose: 2

argparse

An earlier version of cligen generated argparse commmands in the __main__.py file. The current python output doesn't use argparse anymore, resulting in code that is about twice as big, but also twice as fast.

The old, unmaintained, code generator can be invoked by providing --type argparse