No project description provided
Project description
PAI Argument Parser
The PAI Argument Parser extends the common python argument parser by allowing to automatically parse and add arguments based on dataclasses.
See the following example for usage:
from typing import List
from paiargparse import pai_dataclass, pai_meta, PAIArgumentParser
from dataclasses import dataclass, field
@pai_dataclass
@dataclass
class SubmoduleParams:
required_float_arg: float
list_arg: List[int] = field(default_factory=lambda: [1, 2])
arg_with_custom_help: str = field(default="You can specify a custom help string", metadata=pai_meta(
help="Custom help string."
))
@pai_dataclass
@dataclass
class MyArguments:
required_int_arg: int
optional_str_arg: str = "This is cool stuff"
sub_params: SubmoduleParams = field(default_factory=lambda: SubmoduleParams(required_float_arg=2))
if __name__ == "__main__":
parser = PAIArgumentParser()
parser.add_root_argument("myArgs", MyArguments)
args = parser.parse_args()
Call with
python my_program.py --myArgs.required_int_arg 1 --myArgs.sub_params.required_float_arg 0.1
Since the parsing is performed dynamically, the help (-h
) always shows the current state of parsing.
Setup
To setup install the paiargparse pip package:
pip install paiargparse
Supported Types
The following is a list of supported types, and how to access/write their values via the command line
Type | (example) Name | Cmd Line |
---|---|---|
int , str , float , bool |
my_primitive |
--my_primitive 4 |
Enum(str) |
my_enum |
--my_enum value |
List[int/str/float/bool/Enum(str)] or Set[...] |
my_primitive_list |
--my_primitive_list 4 2 1 |
Dict[str/int, int/str/float/bool/Enum] |
my_primitive_dict |
--my_primitive_dict first=1 second=2 |
dataclass |
my_sub_class |
--my_subclass ClassName --my_subclass.PARAMETER |
List[dataclass] |
my_list |
--my_list ClassName ClassName --my_list.0.PARAMETER --my_list.1.PARAMETER |
Dict[str, dataclass] |
my_dict |
--my_dict first=ClassName second=ClassName --my_dict.first.PARAMETER --my_dict.second.PARAMETER |
Examples
The following shows some examples for the different types
Primitive Types
@pai_dataclass
@dataclass
class MyArguments:
required_int_arg: int
optional_str_arg: str = "This is cool stuff"
if __name__ == "__main__":
parser = PAIArgumentParser()
# If passing flat, the root arg name is not passed in the shell (see below)
parser.add_root_argument("myArgs", MyArguments, flat=True)
args = parser.parse_args()
Call with
python my_program.py --required_int_arg 1
Enumerations
class MyEnum(Enum(str)):
Tree = 'tree'
Leaf = 'leaf'
@pai_dataclass
@dataclass
class MyArguments:
enum: MyEnum = MyEnum.Tree
enum2: MyEnum = MyEnum.Tree
if __name__ == "__main__":
parser = PAIArgumentParser()
parser.add_root_argument("myArgs", MyArguments)
args = parser.parse_args()
Call with
python my_program.py --myArgs.enum leaf
Lists and Sets of Primitives
@pai_dataclass
@dataclass
class MyArguments:
int_list: List[int]
str_set: Set[str]
if __name__ == "__main__":
parser = PAIArgumentParser()
parser.add_root_argument("myArgs", MyArguments)
args = parser.parse_args()
Call with
python my_program.py --myArgs.int_list 1 2 3 4 --myArgs.str_set Bert Susi Donald
Dictionaries of Primitives
@pai_dataclass
@dataclass
class MyArguments:
dict1: Dict[int, str]
dict2: Dict[str, str]
if __name__ == "__main__":
parser = PAIArgumentParser()
parser.add_root_argument("myArgs", MyArguments)
args = parser.parse_args()
Call with
python my_program.py --myArgs.dict1 1=one 2=two 3=three --myArgs.dict2 one=one two=two three=three
Child dataclass
@pai_dataclass
@dataclass
class MySubArgs:
any_arg: int = 0
@pai_dataclass
@dataclass
class MySubArgsAlt(MySubArgs):
any_other_arg: int = 0
@pai_dataclass
@dataclass
class MyArguments:
sub: MySubArgs
if __name__ == "__main__":
parser = PAIArgumentParser()
parser.add_root_argument("myArgs", MyArguments)
args = parser.parse_args()
Call with
python my_program.py --myArgs.sub.any_arg 2
python my_program.py --myArgs.sub MySubArgsAlt --myArgs.sub.any_other_arg 2
Lists of dataclasses
@pai_dataclass
@dataclass
class MySubArgs:
any_arg: int = 0
@pai_dataclass
@dataclass
class MySubArgsAlt(MySubArgs):
any_other_arg: int = 0
@pai_dataclass
@dataclass
class MyArguments:
sub: List[MySubArgs]
if __name__ == "__main__":
parser = PAIArgumentParser()
parser.add_root_argument("myArgs", MyArguments)
args = parser.parse_args()
Call with
python my_program.py --myArgs.sub MySubArgs MySubArgs MySubArgsAlt --myArgs.sub.0.any_arg 1 --myArgs.sub.2.any_other_arg 3
Dicts of dataclasses
@pai_dataclass
@dataclass
class MySubArgs:
any_arg: int = 0
@pai_dataclass
@dataclass
class MySubArgsAlt(MySubArgs):
any_other_arg: int = 0
@pai_dataclass
@dataclass
class MyArguments:
sub: dict[str, MySubArgs]
if __name__ == "__main__":
parser = PAIArgumentParser()
parser.add_root_argument("myArgs", MyArguments)
args = parser.parse_args()
Call with
python my_program.py --myArgs.sub one=MySubArgs two=MySubArgs three=MySubArgsAlt --myArgs.sub.one.any_arg 1 --myArgs.sub.three.any_other_arg 3
Further examples
Have a look at the various tests for additional examples.
Exporting/Importing to dict/json
Since a pai_dataclass
inherits dataclass_json
, a dataclass can be writting into a dict and json and read back while preserving the actual types of dataclasses.
This is achieved by an additional __cls__
field which is added to each pai_dataclass
. For example:
@pai_dataclass
@dataclass
class SubmoduleParams:
required_float_arg: float
list_arg: List[int] = field(default_factory=lambda: [1, 2])
arg_with_custom_help: str = field(default="You can specify a custom help string", metadata=pai_meta(
help="Custom help string."
))
@pai_dataclass
@dataclass
class MyArguments:
required_int_arg: int
optional_str_arg: str = "This is cool stuff"
sub_params: SubmoduleParams = field(default_factory=lambda: SubmoduleParams(required_float_arg=2))
args = MyArguments(required_int_arg=0, sub_params=SubmoduleParams(required_float_arg=1))
print(args.to_dict()) # {'required_int_arg': 0, 'optional_str_arg': 'This is cool stuff', 'sub_params': {'required_float_arg': 1, 'list_arg': [1, 2], 'arg_with_custom_help': 'You can specify a custom help string', '__cls__': '__main__:SubmoduleParams'}, '__cls__': '__main__:MyArguments'}
assert(MyArguments.from_dict(args.to_dict()) == args) # True
assert(MyArguments.from_json(args.to_json()) == args) # True
Meta-Data
Set the metadata
-argument of field
to pai_meta
to enrich the information for the argument parser:
argname | default | description | example |
---|---|---|---|
help | None |
The help string to print when calling -h | pai_meta(help="Show the help") |
separator | "." |
The separator for concatenating hierarchical args, e.g. / to use parent/sub |
pai_meta(separator="/") |
mode | "snake" |
Use "ignore" to ignore this field from the command line, use "flat" to add the argument as a new root argument, i.e. no prefixes will be added, use "snake" for the default mode of snaking all arguments with the separator, use "ssnake" to drop only the parent prefix of one hierachy` |
pai_meta(mode="ignore") |
required | None |
Set to True or False to force if this parameter must be set from the command line even though a default value is given |
pai_meta(required=True) |
nargs | * |
Override the default nargs field for list, set, or dict fields. The alternative + forces to add at least one element to the list. |
pai_meta(nargs="+") |
choices | None |
A list of choices the select from when using lists, sets or subdataclasses. Only the class name must be set in the command line if the class is present in choices (instead of the full path) | pai_meta(choices=[SubClass1, SubClass2]) |
disable_subclass_check | False |
By default changing the type of a class via the command line requires that the new class is a subclass of the type of the dataclass field. Use this flag to disable this check. | pai_meta(disable_subclass_check=True |
enforce_choices | None |
Override the default choices checking. For dataclasses, it is permitted to select dataclasses that are not in choices, for primitive types they must be within choices | pai_meta(enforce_choices=True) |
fix_dc | False |
Set to true to forbid overriding of a dataclass via the command line. | pai_meta(fix_dc=True) |
tuple_like | False |
This enables also to set values similar to tuples by passing a list to the dataclass argument instead of accessing all child arguments. Automatically sets "fix_dc" | pai_meta(tuple_like=True) |
Development
paiargparse
uses black code style.
Install the development_requirements.txt
and run pre-commit install
once to automatically run black on commits.
To upgrade the pre-commit
packages call pre-commit autoupdate
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file paiargparse-1.1.2.tar.gz
.
File metadata
- Download URL: paiargparse-1.1.2.tar.gz
- Upload date:
- Size: 27.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6059786c2fb2174d8f18e45135818df51a03fe4bd9d3be23f2a2f4b704e50f5 |
|
MD5 | 878e39ae995ec24dc9c48a7b31206ac6 |
|
BLAKE2b-256 | 440bcc0885c5faf7a2f5ecb51bc48e7b04e2181c4b84fa0e5d9a94e521c368ed |
File details
Details for the file paiargparse-1.1.2-py3-none-any.whl
.
File metadata
- Download URL: paiargparse-1.1.2-py3-none-any.whl
- Upload date:
- Size: 34.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e21f1e1c945da6e77bd2084f25a96b95f80294104db96d1951902b62cde9115a |
|
MD5 | 096cf9d8f46d282c91591215beaf6422 |
|
BLAKE2b-256 | 9766c028cf561f427dd8da3a6d7930499b20ab9e28d704256053dff41063a830 |