cs-cmdutils 20240316

Convenience functions for working with the Cmd module, the BaseCommand class for constructing command line programmes, and other command line related stuff.

Convenience functions for working with the Cmd module, the BaseCommand class for constructing command line programmes, and other command line related stuff.

Latest release 20240316:

• New @uses_cmd_options decorator to provide an "options" parameter being the prevailing BaseCommandOptions instance.
• BaseCommandOptions.popopts: get common options from BaseCommandOptions.COMMON_OPT_SPECS.

Class BaseCommand

A base class for handling nestable command lines.

This class provides the basic parse and dispatch mechanisms for command lines. To implement a command line one instantiates a subclass of BaseCommand:

class MyCommand(BaseCommand):
    GETOPT_SPEC = 'ab:c'
    USAGE_FORMAT = r"""Usage: {cmd} [-a] [-b bvalue] [-c] [--] arguments...
      -a    Do it all.
      -b    But using bvalue.
      -c    The 'c' option!
    """
    ...

and provides either a main method if the command has no subcommands or a suite of cmd_subcommand methods, one per subcommand.

Running a command is done by:

MyCommand(argv).run()

Modules which implement a command line mode generally look like this:

... imports etc ...
def main(argv=None, **run_kw):
    """ The command line mode.
    """
    return MyCommand(argv).run(**run_kw)
... other code ...
class MyCommand(BaseCommand):
... other code ...
if __name__ == '__main__':
    sys.exit(main(sys.argv))

Instances have a self.options attribute on which optional modes are set, avoiding conflict with the attributes of self.

Subclasses with no subcommands generally just implement a main(argv) method.

Subclasses with subcommands should implement a cmd_subcommand(argv) instance method for each subcommand. If a subcommand is itself implemented using BaseCommand then it can be a simple attribute:

cmd_subthing = SubThingCommand

Returning to methods, if there is a paragraph in the method docstring commencing with Usage: then that paragraph is incorporated automatically into the main usage message. Example:

def cmd_ls(self, argv):
    """ Usage: {cmd} [paths...]
          Emit a listing for the named paths.

        Further docstring non-usage information here.
    """
    ... do the "ls" subcommand ...

The subclass is customised by overriding the following methods:

• apply_opt(opt,val): apply an individual getopt global command line option to self.options.
• apply_opts(opts): apply the opts to self.options. opts is an (option,value) sequence as returned by getopot.getopt. The default implementation iterates over these and calls apply_opt.
• cmd_subcmd(argv): if the command line options are followed by an argument whose value is subcmd, then the method cmd_subcmd(subcmd_argv) will be called where subcmd_argv contains the command line arguments following subcmd.
• main(argv): if there are no command line arguments after the options or the first argument does not have a corresponding cmd_subcmd method then method main(argv) will be called where argv contains the command line arguments.
• run_context(): a context manager to provide setup or teardown actions to occur before and after the command implementation respectively, such as to open and close a database.

Editorial: why not arparse? Primarily because when incorrectly invoked an argparse command line prints the help/usage messgae and aborts the whole programme with SystemExit. But also, I find the whole argparse add_argument thing cumbersome.

Method BaseCommand.__init__(self, argv=None, *, cmd=None, options=None, **kw_options): Initialise the command line. Raises GetoptError for unrecognised options.

Parameters:

• argv: optional command line arguments including the main command name if cmd is not specified. The default is sys.argv. The contents of argv are copied, permitting desctructive parsing of argv.
• cmd: optional keyword specifying the command name for context; if this is not specified it is taken from argv.pop(0).
• options: an optional keyword providing object for command state and context. If not specified a new self.Options instance is allocated for use as options. The default Options class is BaseCommandOptions, a dataclass with some prefilled attributes and properties to aid use later. Other keyword arguments are applied to self.options as attributes.

The cmd and argv parameters have some fiddly semantics for convenience. There are 3 basic ways to initialise:

• BaseCommand(): argv comes from sys.argv and the value for cmd is derived from argv[0]
• BaseCommand(argv): argv is the complete command line including the command name and the value for cmd is derived from argv[0]
• BaseCommand(argv, cmd=foo): argv is the command arguments after the command name and cmd is set to foo

The command line arguments are parsed according to the optional GETOPT_SPEC class attribute (default ''). If getopt_spec is not empty then apply_opts(opts) is called to apply the supplied options to the state where opts is the return from getopt.getopt(argv,getopt_spec).

After the option parse, if the first command line argument foo has a corresponding method cmd_foo then that argument is removed from the start of argv and self.cmd_foo(argv,options,cmd=foo) is called and its value returned. Otherwise self.main(argv,options) is called and its value returned.

If the command implementation requires some setup or teardown then this may be provided by the run_context context manager method, called with cmd=subcmd for subcommands and with cmd=None for main.

Class BaseCommandCmd(cmd.Cmd)

A cmd.Cmd subclass used to provide interactive use of a command's subcommands.

The BaseCommand.cmdloop() class method instantiates an instance of this cand calls its .cmdloop() method i.e. cmd.Cmd.cmdloop.

Class BaseCommandOptions(cs.threads.HasThreadState, cs.context.ContextManagerMixin)

A base class for the BaseCommand options object.

This is the default class for the self.options object available during BaseCommand.run(), and available as the BaseCommand.Options attribute.

Any keyword arguments are applied as field updates to the instance.

It comes prefilled with:

• .dry_run=False
• .force=False
• .quiet=False
• .verbose=False and a .doit property which is the inverse of .dry_run.

It is recommended that if ``BaseCommandsubclasses use a different type for theirOptions` that it should be a subclass of `BaseCommandOptions`. Since `BaseCommandOptions` is a data class, this typically looks like:

@dataclass
class Options(BaseCommand.Options):
    ... optional extra fields etc ...

Function docmd(dofunc)

Decorator for cmd.Cmd subclass methods to supply some basic quality of service.

This decorator:

• wraps the function call in a cs.pfx.Pfx for context
• intercepts getopt.GetoptErrors, issues a warning and runs self.do_help with the method name, then returns None
• intercepts other Exceptions, issues an exception log message and returns None

The intended use is to decorate cmd.Cmd do_* methods:

from cmd import Cmd
from cs.cmdutils import docmd
...
class MyCmd(Cmd):
    @docmd
    def do_something(...):
        ... do something ...

Function uses_cmd_options(func, cls=<class 'cs.cmdutils.BaseCommandOptions'>, options_param_name='options')

A decorator to provide a default parameter containing the prevailing BaseCommandOptions instance as the options keyword argument, using the cs.deco.default_params decorator factory.

This allows functions to utilitse global options set by a command such as options.dry_run or options.verbose without the tedious plumbing through the entire call stack.

Parameters:

• cls: the BaseCommandOptions or BaseCommand class, default BaseCommandOptions. If a BaseCommand subclass is provided its cls.Options class is used.
• options_param_name: the parameter name to provide, default options

Examples:

@uses_cmd_options
def f(x,*,options):
    """ Run directly from the prevailing options. """
    if options.verbose:
        print("doing f with x =", x)
    ....

@uses_cmd_options
def f(x,*,verbose=None,options):
    """ Get defaults from the prevailing options. """
    if verbose is None:
        verbose = options.verbose
    if verbose:
        print("doing f with x =", x)
    ....

Release Log

Release 20240316:

• New @uses_cmd_options decorator to provide an "options" parameter being the prevailing BaseCommandOptions instance.
• BaseCommandOptions.popopts: get common options from BaseCommandOptions.COMMON_OPT_SPECS.

Release 20240211:

• Include the first sentence of the subcommand description in the short help.
• BaseCommandOptions: move the runstate_signals into this directly.
• BaseCommand: move the run() options stuff into run_context() and have it work on a copy of the original options.
• BaseCommandCmd: implement get_names(), provide docstrings for the do_* attributes, thus help.
• BaseCommand.run_context: make runstate and upd keyword only parameters.

Release 20240201:

• BaseCommand.run: catch CancellationError and return 1.
• BaseCommandCmd.getattr: recognise EOF, exit and quit to terminate the cmdloop.

Release 20231129: BaseCommandOptions: define a runstate field.

Release 20230703: Small internal changes.

Release 20230612:

• BaseCommand.cmdloop: fix intro parameter.
• Other small fixes.

Release 20230407:

• BaseCommand: use @uses_runstate when preparing the command, store as self.options.runstate.
• Make BaseCommandOptions a data class.
• Drop any pretence at python 2 support, we're long past that.
• BaseCommand: new cmdloop method to run a cmd.Cmd instance to run subcommand interactively.
• BaseCommand: rename shell to repl, add cmd_shell to call cmdloop().
• Drop BaseCommand.apply_defaults in favour of the Options dataclass.
• BaseCommand: do setup_logging before initiating the Options instance.

Release 20230212:

• BaseCommand.run_context: update RunState support.
• BaseCommand.run_context: always be having an self.options.upd.

Release 20230211: BaseCommand: new shell() method to present an interactive Python prompt for use by subclasses cmd_shell method if desired.

Release 20221228: Move a lot of the context logic from BaseCommand.run to BaseCommand.run_context, which now must be correctly overridden in subclasses.

Release 20220918:

• BaseCommand.run_context: expand default signals to include SIGHUP, expose as BaseCommand.DEFAULT_SIGNALS.
• BaseCommand.run: pass in the subclass handle_signal method if present.

Release 20220626:

• BaseCommand.poparg: fix positional argument handling.
• BaseCommand.poparg: new unpop_on_error=False parameter to support pushing a bad argument back onto the front of the argument list.

Release 20220606: BaseCommand.run: remove the Upd bodge, too annoying, now fixed in cs.upd I believe.

Release 20220605:

• BaseCommand: new popopts(argv,...) compact getopt wrapper.
• BaseCommand: new poparg(argv,...) compact validating argument consumer.
• BaseCommand: drop run_argv, provided no utility.
• BaseCommand.run: get the RunState signal list from self.options.runstate_signals.
• BaseCommand.apply_opts: support multiple individual options raising GetoptError, as I hate commands which abort at the first bad option.
• Assorted other small things.

Release 20220429:

• BaseCommand: fold dots in argv[0] into underscores, supports subcommands like "setup.py".
• BaseCommand: new popargv(argv[,help_text[,parse[,validate[,unvalidated_message]]]]) helper class method.
• BaseCommand: accept dashed-form of the underscored_form subcommand name.
• BaseCommand: new self.options.runstate_signals=SIGINT,SIGTERM specifying singals to catch-and-cancel, shuffle run() context managers.

Release 20220318: BaseCommand.init: handle main() method in the New Scheme.

Release 20220315: _BaseSubCommand.init: hook in the class USAGE_KEYWORDS for methods.

Release 20220311: BaseCommand: big refactor of subcommand internals and make the "cmd_foo=FooCommand" implementation work properly.

Release 20211208: BaseCommand: better handle an unknown subcommand.

Release 20210927:

• Usage: show only the per subcommand usage for in-subcommand GetoptError.
• Usage: show terse usage when the subcommand cannot be recognised.
• Usage: support bare -h, -help, --help.

Release 20210913: New BaseCommand.apply_preargv method to gather special arguments before subcommands.

Release 20210906:

• BaseCommand.cmd_help: bugfix obsolete parameter list.
• BaseCommand.SUBCOMMAND_ARGV_DEFAULT: support a single str value, turn into list.

Release 20210809: Bugfix BaseCommand.cmd_help for modern API.

Release 20210731:

• BaseCommand.run: apply optional keyword arguments to self.options during the run.
• Look for self.SUBCOMMAND_ARGV_DEFAULT if no subcommand is supplied.
• Bugfix case for "main" method and no "cmd_*" methods.
• Bugfix BaseCommand.cmd_help.

Release 20210420:

• BaseCommand.getopt_error_handler: replace error print() with warning().
• Docstring improvements.

Release 20210407.1: BaseCommand: bugfix for init_subclass docstring update.

Release 20210407:

• BaseCommand.init_subclass: behave sanely if the subclass has no initial doc.
• BaseCommand: new .run_argv convenience method, obviates the "def main" boilerplate.

Release 20210404: BaseCommand subclasses: automatically add the main usage message to the subclass docstring.

Release 20210306:

• BREAKING CHANGE: rework BaseCommand as a more normal class instantiated with argv and with most methods being instance methods, getting the former options parameter from self.options.
• BaseCommand: provide default apply_opt and apply_opts methods; subclasses will generally just override the former.

Release 20210123: BaseCommand: propagate the format mapping (cmd, USAGE_KEYWORDS) to the subusage generation.

Release 20201102:

• BaseCommand.cmd_help: supply usage only for "all commands", full docstring for specified commands.
• BaseCommand: honour presupplied options.log_level.
• BaseCommand.usage_text: handle missing USAGE_FORMAT better.
• BaseCommand.run: provide options.upd.
• BaseCommand subclasses may now override BaseCommand.OPTIONS_CLASS (default SimpleNamespace) in order to provide convenience methods on the options.
• BaseCommand.run: separate variable for subcmd with dash translated to underscore to match method names.
• Minor fixes.

Release 20200615: BaseCommand.usage_text: do not mention the "help" command if it is the only subcommand (it won't be available if there are no other subcommands).

Release 20200521.1: Fix DISTINFO.install_requires.

Release 20200521:

• BaseCommand.run: support using BaseCommand subclasses as cmd_* names to make it easy to nest BaseCommands.
• BaseCommand: new hack_postopts_argv method called after parsing the main command line options, for inferring subcommands or the like.
• BaseCommand: extract "Usage:" paragraphs from subcommand method docstrings to build the main usage message.
• BaseCommand: new cmd_help default command.
• Assorted bugfixes and small improvements.

Release 20200318:

• BaseCommand.run: make argv optional, get additional usage keywords from self.USAGE_KEYWORDS.
• @BaseCommand.add_usage_to_docstring: honour cls.USAGE_KEYWORDS.
• BaseCommand: do not require GETOPT_SPEC for commands with no defined options.
• BaseCommand.run: call cs.logutils.setup_logging.

Release 20200229: Improve subcommand selection logic, replace StackableValues with stackattrs, drop cmd from arguments passed to main/cmd_* methods (present in options).

Release 20200210:

• New BaseCommand.add_usage_to_docstring class method to be called after class setup, to append the usage message to the class docstring.
• BaseCommand.run: remove spurious Pfx(cmd), as logutils does this for us already.

Release 20190729: BaseCommand: support for a USAGE_FORMAT usage message format string and a getopt_error_handler method.

Release 20190619.1: Another niggling docstring formatting fix.

Release 20190619: Minor documentation updates.

Release 20190617.2: Lint.

Release 20190617.1: Initial release with @docmd decorator and alpha quality BaseCommand command line assistance class.