bg-cliBuilder 0.1.0

An argparse alternative that requires much less coding overhead

bg_cliBuilder

bg_cliBuilder is a lightweight alternative to argparse for building complex command-line interfaces with much less boilerplate.

It is designed around:

• feature parity with argparse plus more capability
• declarative command definitions
• reduced parser ceremony
• help content separation from code supporting language translations
• builtin, innovative bash completion support
• full *nix argument and option convention. i.e. tar -vcf <path>

The project originated from the needs of large multi-command CLI applications such as bg-agent, where standard argparse patterns became repetitive and difficult to maintain.

---

Installation

pip install bg_cliBuilder

---

Quick Example

bg_cliBuilder style:

from bg_cliBuilder import CliBuilder

def getAgentIDs():
  return ["one", "two", "three"]

cli = CliBuilder(helpNotation=helpNotation)

cli.cmds("""
    run --agent|-a=<id> <target>
    sessions [list]
    sessions create <name> <maxRuns:int=5>
    """)

cli.addTo("--agent", completer=getAgentIDs)
cli.addTo("--agent", hitFn=lambda agent: print(f"{agent} specified on the command line"))

cli.parseArgs()

match args.cmdPath:
    case ("run",):
      runTarget(cli.target, cli.agentID)
    case ("sessions",):
      match sessionsCmd:
        case "list":
          listSessions()
        case "create":
          createSession(cli.name, cli.maxRun)

# this can also be provided in an external file (e.g. <cmdName>.help.en-US) if desired
# for each command or argument provide helpText. text can be multiple lines.
helpNotation="""
  Text at the start (before any [] section header) is the
  general description of the command...
    * format is preserved in all texts after []
    * also, long lines are wrapped
  [cmd:run]
  Run somthing, do something
  [cmd:sessions]
  Sessions is a branch cmd for several sub cmds below it...
  [cmd:sessions list]
  view the available session ...
  [cmd:sessions create]
  create a new session...
  [arg:<target>]
  what is a target? this should tell you
  [arg:<id>]
  say what <id> is for...
  [arg:<name>]
  say what <name> is for...
  [arg:<maxRuns>]
  say what <maxRuns> is for...
  [opt:--agent]
  say what this option does...
"""

Traditional argparse:

import argparse

parser = argparse.ArgumentParser()

subparsers = parser.add_subparsers(dest="cmdPath0")

runParser = subparsers.add_parser("run")
runParser.add_argument("--agent", "-a", dest="agentID")
runParser.add_argument("target")

sessionsParser = subparsers.add_parser("sessions")
sessionsSubparsers = sessionsParser.add_subparsers(
    dest="sessionsCmd",
    required=False,
)

sessionsListParser = sessionsSubparsers.add_parser("list")

sessionsCreateParser = sessionsSubparsers.add_parser("create")
sessionsCreateParser.add_argument("name")
sessionsCreateParser.add_argument(
    "maxRuns",
    type=int,
    nargs="?",
    default=5,
)

args = parser.parse_args()

cmdPath = (args.cmdPath0,)

match cmdPath:
    case ("run",):
        runTarget(args.target, args.agentID)

    case ("sessions",):
        match args.sessionsCmd or "list":
            case "list":
                listSessions()
            case "create":
                createSession(args.name, args.maxRuns)

---

Parsing Concepts

Any command like contains three types of arguments.

Type	Notation
sub command token literals	subcmdName
positional arguments	<varName>
optional arguments	--varName -x

1. options are removed from the command line.
   • exception raised if any are not valid for the specified sub command
   • their corresponding variables set according to various actions that can be specified.
2. the sub command is identified
   • all tokens up to the last string literal are considered sub commands tokens and removed.
   • some sub command tokens can also be <positional> arguments. The single positional argument at a command line position becomes the default command path when the entered token does not match any of the other sub command literals.
3. the rest are positional arguments to the identified sub command
   • their varNames are set.

• Various complications are handled.
  • default parameters
  • options do not have to be placed at any particular location. If its valid for the sub command being invoked it will be accepted regardless of its location
  • separating combined short options (i.e. -vcf treated as three different options)
  • option arguments can be specified as the next token or in the same token with an = like...
    • --file /my/path or --file=/my/path
    • -f /my/path or -f=/my/path)

---

Setting Variables

Positional arguments and the required arguments of optional arguments create a <varName> in the parsed namespace.

1. for all arguments that could be specified for the identified sub command, the argument's arg.varName is created with the default value:
   ns[arg.varName]=arg.default or None
2. For any argument that is matched with a token on the command line the argument's hit function is invoked which typically modifies ns[arg.varName]
   arg.hit(token) See actions

Option actions and hitFn

Optional arguments have an action similar to argparse but the available actions are much simplified.

action	handling
store (default)	ns[arg.varName]=token
append	ns[arg.varName].append(token) (coerced to a list)
increment	ns[arg.varName]++ (coerced to int)
decrement	ns[arg.varName]-- (coerced to int)
count	alias for increment

In addition to performing the action, an optional argument can be given a hitFn that will be invoked. The signature of the hitFn can contain any of the following context variables.

variable name	description
argToken | token	the token from the command line that matches this argument
<varName>	any of the <varName> present in argsNS. Only those occurring earlier on the command line will be set with their final value
args | argsNS | ns	the parsed namespace containing the values set from the command line
ctx | parsingContext	the parsing context
cli | cliBuilder	the CliBuilder instance

Example:

def doSomething(agentID:str):
  somethingAboutAgents(agentID)

cli.addTo("--agentID", hitFn=doSomething)

# assumes that --agentID is declared to have a required argument
# something like --agentID:str or --agentID=<id>

---

Argument Declaration

Shared Positional and Optional Attributes:

The commonly used attributes can be specified in the Positional or Optional argument string notation but any of these attribute can be set or overridden by calling cli.addTo after cli.cmds:

cli.addTo(<argumentName>, <attribName>=<value>)

<argumentName> are any of the str names used to create the argument.
               e.g. "<session>", "<agent>", "--foo", "-f", etc...

Shared Argument Attributes

attribName	Description
optional	true means not required, false means error if not present
key	The name that uniquely identifies the argument. Typical this is automatically chosen and can be ignored.
varName	The name of the variable that is created in argsNS when the argument is active
type	The python type varName's value is coerced to. Typically one of str,int,float,bool,Path but any immutable type might work.
default	The default value used when argument is active but not present on the command line.
choices	a list of literal strings the value is restricted to. error if the actual token does not match. bash completion encourages a correct value.
completer	a callback function that returns domain specific bash completion suggestions for argument.
helpText	hard coded helpText (Typically use the helpNotation instead)

Positional Arguments:

String Notation

<<varName>>[:<metaSpec>][=<defaultSpec>]
<<varName>> := (name of the varName surrounded by <>)
metaSpec    := <type>[?][...]
<type>      := str,int,float,bool,Path (other immutables may work)
defaultSpec := <defaultValue>
defaultSpec := one|two*|three
defaultSpec := None or <emptyStr>  <- converts to None and sets optional to True

Where:
? means option. If a default value is given ? is implied
... means consume the remainder of positional arguments.
    This implies that varName will be a list
one|two*|three is a list of literal strings that the value is restricted to
* indicates the default value in a list of literals

Positional-only Attributes

Used with cli.addTo(<argName>, <attribName>=<value>, ...)

attribName	Description
optional	true means not required
remainder	true means consume the remainder of positional args

Optional Arguments:

String Notation

<optName>[|<optName>][:<metaSpec>][{ |=}<valueSpec>]
optName := --<longOpt>
optName := -?   # <shortOpt> ? is a char
metaSpec:= <type>
<type>      := str,int,float,bool,Path (other immutables may work)
metaSpec:= +
valueSpec := <valueName>
valueSpec := one|two*|three
valueSpec := constValue

Where:
+ is declares action=="increment"
one|two*|three is a list of literal strings that the value is restricted to
* indicates the default value in a list of literals

Positional-only Attributes

Used with cli.addTo(<argName>, <attribName>=<value>, ...)

attribName	Description
requiresValue	true means a value token must follow this option
valueName	the label for the option's required value (when requiresValue==True). used in generated help and usage text default is varName surrounded in <>
constValue | const	this is the value varName is set to when the option is present and requiresValue==False default is True
action	store | append | increment | decrement | count (count is an alias for increment) Note that store_true, etc.. are not needed because you can set both default and constValue directly to achieve any of those outcomes
htiFn | hitFN | activationFN	a callback function invoked when this option is encountered on the command line

When requiresValue==False the default is that varName will be false if the option is not present and true if it is. If you want different behavior set both default, constValue to achieve any outcome you want. The types do not need to match

cli.addTo("--myOpt", default="foo", constValue="bar")
cli.addTo("--myOpt", default=100, constValue=0)
cli.addTo("--myOpt", constValue=MyCls())  # leave default=False

---

Bash Completion Support

Once the BGBC completion driver is installed on the user's host any python script that uses CliParser gets bash completion automatically.

The BGBC driver allows some new features of completion.

• messages in the form <argName> appear before the actual suggestions to let the user knows the name of the argument being completed. This avoids the user having to lookup the command syntax.
• options are suppressed and replaced with a message <optsAvail> until the user enters a '-' and then offers the options valid at that point in the sub command completion. Since option lists can get large, this helps to keep the suggestions list small and meaningful.

The user of your script can install the BGBC driver by running with this option. It copies two BGBC driver files to $HOME/.local/share/bg_cliBuilder/ and add lines at the end of $HOME/.bashrc to source those files.

<yourScriptName> --installBashCompletion

BGBC driver chainloads an automatic completion loader into place. Whenever <tab> is pressed on the command line of a new command if its file contains "CliBuilder" it will start using the BGBC driver with integrates with the CliBuilder code in the script.

Custom Bash Handler Functions

Out of the box, much of the command's syntax is supported with suggestions. The only arguments it does not how to suggest are the domain specific values. This allows for a much better experience because just completing the command line tells the user something about the data the command works with.

For those you can write a function that returns the suggestions. Those functions can accept any varName that is available on the command line. The entire partial command line being completed is parsed so that the function completing the argument can have all the information from the command line to use to affect the suggestions list.

These functions can either print their suggestions to stdout or return them in a string or list.

In addition to value suggestion literals, they can output directives to the BGBC driver. For example ...

• "$(_filedir)" tells the driver to use the builtin file path completion.
• Tokens surrounded by <> are displayed as is at the front of the suggestions list. The user will not be able to complete on these message tokens because the unescaped '<' is only a valid char after the command as redirection.

See man _bgbc-complete-viaCmdDelegation for more details.

Help Text

Help text can and should be provided for the command summary, every sub command and every argument.

The compact notion this library uses does not allow for the help texts to be supplied inline. Instead a helpText string is defined in the file (typically at the end) which contains all the help text.

This has the advantages:

• allows the text to easily be translated to a language.
  • if a file .help. exists for the user's configured locale, it will override the builtin help text.
• promotes consistency by having all text in one place
• the entire help and usage text is generated from this information and the syntax specifications.

The format of the text is simple.

[<identifier>]
Free form help text up to the next bracketed identifier line.

To create a new language translation:

1. run the command...
   <cmd> --helpCreateFile
2. the file extension will reflect the locale configured on the host but the contents will be in the language of the builtin help text.
3. edit the text parts of the file to reflect the target language. (do not translate the [<identifier>]s)
4. Please make pull requests to add any translations.

The -h|--help option is supported by default. These options can be placed anywhere on the command line and will invoke the help for the most specific sub command entered on the command line. Its ok to have options and positional values on the command line when help is requested.

---

Features

CliBuilder offers a handful of features that you can choose to enable.

Verbosity

cli.enableVerbosityFeature()

Adds

-v|-q  -> cli.verbosity is an int that counts the
          number of -v minus the number of -q

Current Working Folder

cli.enablePwdFeature()

cli.pwd will be set to <folderPath> and also the cwd (aka pwd) of the process is change to <folderPath>

Adds

--pwd|-C <folderPath>  -> the cwd (aka pwd) of the process is change to \<folderPath>

Find Project Root Folder

cli.enableFindProjectRootFeature(".git")

After the the pwd is changed (if enablePwdFeature is used) this looks for the specified folder in the pwd and its parents and sets cli.projectFolder to the first folder found where it exists. This implements the behavior of git where you can run repo commands from sub folders.

Adds

cli.projectRoot and cli.projectFolder variables are set to the found folder.

Bash Completion Install

cli.enableBashCompletionInstallFeature()

The only requirement to have bash completion for commands using CliBuilder is that the BGBC driver is installed. When this is enabled an option is available to install the drivers. The user only has to do that once on a host.

Adds

--installBashCompletion -> installs the BGBC driver files and modifies the user's .bashrc file to source them.

Help

cli.enableHelpFeature() Help is such a universal feature that it is currently being enabled in the CliBuilder.init function. If you want to disable it you have to command out that line in bg_cliBuilder.py

Adds

--help|-h  -> show help text
--helpCreateFile -> write the builtin help text notation to a file that can be translated into the local language.

If no sub command is specified on the command line the full help and usage text will be shown otherwise the usage syntax and help text for the sub command is shown.

--helpCreateFile: This create an external file in the current folder with the name of the command plus the ".help." extension.

1. that external file can be translated into the locale language and optional copied to a system folder.
2. this is also useful for developing good help descriptions so that you can try different descriptions before committing them to code.

---

Lower Layer Building Interfaces

This module was developed in stages. The lower level interfaces are still available and may still be useful in some cases.

The higher levels rely on string parsing to identify what each part of the syntax is. Lower level are more explicit about what is sub command tokens, positional arguments, and optional arguments.

Current Interface - Notation for entire Cli:

cli.cmds("""
 <one sub command per text line>
 ...
""")

cli.addTo(<argName>, <attribName>=<value>, ...)
cli.addTo(<cmdPath>, <attribName>=<value>, ...)

Where:
<argName> can be any positional or optional argument name declared in the cmds text.  
<cmdPath> can be a string of space separated cmd tokens for any subcommand declared in the cmds text.

Single Command Notation:

This can be used when it is ambiguous whether a token is a catchall cmd token or a postional argument.

cli.cmd(<cmdPath>, args=["strSpec" | pos(...) | opt(...)], <attribName>=<value>, ...)

Where:
<cmdPath> can be a string of space separated cmd tokens specifying a new sub command.
<attribName> these are attributes of the command. Currently there are no useful sub command attributes to set.

Example:

In this example we have multiple sub cmds under <sessionID> but we want to declare an option that will be valid in any of those sub commands. We could repeat the option in each sub command but that is repetitive. If the first part of the of these sub commands were a literal token it would be unambiguous. But any sub command path that ends in a catchall token is ambiguous.

cli.cmds("""
  ...
  <sessionID> --agentClass:str  # <- WRONG: this is not interpreted as intended
  <sessionID> [list]
  <sessionID> create
  ...
""")
# this does correctly interpret <sessionID> as a catchall command token
cli.cmd("<sessionID>", args=["--agentClass:str"])

So in this case we can exclude the line from the cmds() text and use a call to cmd() to make it clear that <sessionID> is a command token.

Individual Argument Notation:

In the args list of the single command notation, each argument can be specified with the string notation or a call to pos() or opt()

args=[
  <argStrSpec>,
  pos("<name>", <attribName>=<value>, ...),
  opt("-C", "--pwd", <attribName>=<value>, ...)
]

The pos() and opt() helpers are very declarative but I am not aware of any case of a <argStrSpec> that would be ambiguous as the type of argument because the '-' conventions is so strong.

The was also useful in the early iterations of CliBuilder before <argStrSpec> was added because it allowed arguments to be created in the argument array and allowed arbitrary attributes to be specified in the named argument. In the current interface the combination of the inline <argStrSpec> notation and the cli.addTo() covers all known cases.

Status

Early public release.

The API may evolve as the project is used in larger real-world systems.

Feedback and pull requests are welcome.