PySide6 based user-interface tools to create and manage machine learning training/testing-configurations and run them automatically and/or remotely..
Project description
Configurun
Configurun is a cross-platform PySide6-based package that implements an application for managing, creating and (remotely) running python configurations.
It was designed mainly with machine-learning tasks in mind, but can be used for any python script that takes arguments as an input. The editor-UI is created automatically using either an argparse.Argumentparser
or python-@dataclass
(es).
The Configurun-app is especially useful for scripts/experiments that require a lot of arguments to be tweaked across many experiment-runs. It also makes the process of running experiments remotely much easier by enabling the user to edit, add and schedule tasks on any running Configurun server-instance reachable via a network connection.
This package was created in tandem with pyside6-utils.
Table of contents
- Configurun
- Table of contents
- Features
- Installation
- How to run?
- Option-source
- Target Function
- Configuration
- Option metadata
Features
Configuration Editor
The configuration editor allows the user to specify a configuration-template using either (groups of) @dataclass
-class or an ArgumentParser
-instance. The editor will then automatically create a UI based on the provided template. Editors are specifically created for each option-property based on provided template-types (and extra constraints). Help-messages are displayed on hover, required arguments are highlighted when not filled in, etc.
We can also define our own option-source-method to dynamically create new option-groups based on the current configuration. This can be useful if we want to group options together, and only show certain groups when an attribute of another group is set to a certain value. E.g: only show ExtendedExampleModel
-options if property model_type
in MainOptions
is set to "ExtendedExampleModel"
.
Configurations can be saved and loaded, a file-explorer view for the current workspace is made available.:
Run Queue
The run-queue window manages the currently running items. This could either be locally when using a local-app, or remotely, when using a client-app and a server-instance on which the actual Run-Queue is running. The Run-Queue allows us to add/remove items, pause/resume items, change the queue-order of items, and start autoprocessing, which will automatically start the next item in the queue when the current item is finished. We can set the number of processors as well, to run multiple items in parallel.
Configurations are passed to the user-provided run-function in separate processes. The stdout/stderr of each of the items is captured and displayed as a selectable console-output-view in the command-line-output window:
Remote-processing
Instead of using the local-app to manage and run the configurations on your own machine, we can use the client-app to connect to a server-instance on a remote machine. The client-app works analogous to the local-app and allows us to create and manage new configuration, but the Run-Queue runs on the connected remote Configurun-server:
Installation
This package can downloaded from this repository, or can be installed directly from PyPi by using pip:
pip install configurun
How to run?
Creating the app is done via the configurun.create
-module. We can create 3 different types of apps:
- Local app - For running everything locally on your machine
- Client app - For running the configurations on a remote machine, connects to a
server
-instance - Server instance - Command-line instance that listens to connections from
client
-apps. If login is succesful, accepts control of theRunQueue
from theclient
-app.
On the client-side, the options_source
should be set - the template of the settings used to create the configuration-editor.
On the server/running-machine, the target_function
should be set - the function that actually runs the task/experiment (example).
Local App
A local app is an all-in-one app that can be used to create and run configurations locally on your machine.
To run the example app, we can either call run_example_app()
from configurun.examples
or run the following code to construct the app ourselves:
### This example will run the app with an example configuration
# Also see `configurun/examples/example_target_function.py`
# Also see `configurun/examples/example_deduce_new_option_class_types.py`
import os
from configurun.create import local_app
from configurun.examples import example_target_function, example_deduce_new_option_classes
if __name__ == "__main__": #Makes sure bootstrapping process is done when running app
local_app( #Create and runs a local configurun app-instance
target_function=example_target_function, #The function that will be called with the configuration
options_source=example_deduce_new_option_classes, #Template for UI-optiosn: Callable/@datclass/ArgumentParser
workspace_path = os.path.join( #Settings, configs and the Run-Queue will be saved/loaded from/to here
os.getcwd(),
"LocalExampleWorkspace"
)
)
In this example, example_target_function
runs a dummy task that logs to a file for 20 seconds. We can specify our own run-function to run our own scripts.
We can specify our own options source to create our own options-class for the configuration-editor, for example by using an existing ArgumentParser
-object.
Client App
We can create a client-app and use it to login to running server-instances. We can then use the client-app analogous to the local-app to create new confiugrations and add/run/manage configurations on the remote machine.
# Opens a client-side app that we can use to connect to and control
# the server-instance
import os
from configurun.create import client
from configurun.examples import example_deduce_new_option_classes
if __name__ == "__main__":
client(
options_source=example_deduce_new_option_classes,
workspace_path=os.path.join(os.getcwd(), "ClientExampleWorkspace"),
)
Server-instance
The server-instance is a command-line app that listens to connections from client
-instance(s) to receive new configurations and commands to manage its RunQueue. The actual run-functions are ran on this machine.
NOTE: after authentication, pickle
/dill
is used to transmit data, which indirectly enables arbitrary code execution on the server-side if the password is known. Please run the server on trusted network environments only. Run at your own risk!
# Opens a server-instance which tries to connect with clients and allows
# them to add configurations to the queue to be run on this machine
import os
from configurun.create import server
from configurun.examples.example_target_function import example_target_function
if __name__ == "__main__":
# WARNING:
# THIS ALLOWS OTHER MACHINES THAT RESIDE ON THE SAME NETWORK
# TO EXECUTE ARBITRARY CODE ON THIS MACHINE IF THEY KNOW THE
# PASSWORD. PLEASE RUN IN A TRUSTED NETWORK ENVIRONMENT ONLY
# RUN AT YOUR OWN RISK!
server(
target_function=example_target_function,
workspace_path=os.path.join(os.getcwd(), "ServerExampleWorkspace"),
password="password", #Password to connect to the server, make sure to change this!
port=5454 #Port to connect to the server, defaults to 5454
)
Option-source
When creating an app using the create
-module, we can define a custom source, using the options_source=...
, so we can construct the UI using our own options.
We can use the following types as an options-source:
Custom Options (@dataclass
)
NOTE: Using fields results in more control over the final UI, for a more thorough example, please see this section and/or the example implementations in configurun/examples/example_options/example_options.py.
NOTE: When implementing custom option-classes, don't forget to add the @dataclass
-decorator, and always inherit from BaseOptions
import os
from dataclasses import dataclass
from configurun.configuration.base_options import BaseOptions
from configurun.create import local_app
from configurun.examples import example_target_function
@dataclass #Don't forget to add this(!) - otherwise the app will not recognize the fields
class MyCustomOptions(BaseOptions): #Always inherit from BaseOptions (required to run config)
simple_int : int = 1
# etc...
if __name__ == "__main__":
local_app(
target_function=example_target_function,
options_source=MyCustomOptions, #Simple: each configuration consists of a single options-class
workspace_path = os.path.join(os.getcwd(), "ExampleDataclassOptions")
)
Custom Options (ArgumentParser
)
We can use a ArgumentParser
-object as an options source, this will internally convert the argument parser into a @dataclass
-object, which is then used as an options-class. Certain arguments are also parsed to control the UI (e.g. required=True
, help="Will be displayed on hover"
).
import argparse
import os
from configurun.create import local_app
from configurun.examples import example_target_function
parser = argparse.ArgumentParser()
parser.add_argument("--required_arg", type=str, required=True, help="Required argument help")
#... add more arguments here
if __name__ == "__main__":
local_app(
target_function=example_target_function,
options_source=parser, #Parser is converted internally to a dataclass-class which is used as the options-class
workspace_path = os.path.join(os.getcwd(), "ExampleArgparseOptions")
)
Custom Options (Callable
)
A configuration is a collection of option-instances, which are grouped toghether in a Configuration
-wrapper, which enables us to access the attributes of all enclosed options-instances using the configuration[attribute]
/configuration.<attribute>
/option_class.get(attribute, default)
. For more information, see this section.
We define an option
-class as a class that has the @decorator
and inherits from the BaseOptions
-class.
As an options-source, we can create a callable which takes the current Configuration-instance as an argument and returns 1 or more new options-classes (not instances) which is called every time a setting is changed. If the types-change, the UI will be updated to reflect the new templates. This can be useful if we want to group options together, and only show certain groups when an attribute of another group is set to a certain value. For example:
#In this example, we will create a callable which returns new options-classes based on the
# current configuration
import os
import typing
from dataclasses import dataclass
from configurun.create import local_app
from configurun.examples import example_target_function
from configurun.configuration import BaseOptions, Configuration
@dataclass #NOTE: Always use @dataclass for options
class AlwaysTheSame(BaseOptions): #NOTE: Always use BaseOptions as base class for options
base_int : int = 1
#...
@dataclass
class CustomOptionsDefault(BaseOptions):
simple_int : int = 1
#...
@dataclass
class CustomOptionsUnderConditions(BaseOptions):
simple_int : int = 2
some_more_options : str = 'Some string'
#...
def deduce_new_option_classes(configuration: Configuration)\
-> typing.Dict[str, typing.Type[BaseOptions | None]]: #Always return a dict of option
# classes the key of the dict is the name/group of the option-class
# the value is the option-class (@dataclass & BaseOptions) itself
if configuration.options is None or len(configuration.options) == 0:
pass #If initial configuration is being retrieved -> return default dict
elif configuration.base_int == 2 and configuration.simple_int != 1:
#Only return the CustomOptionsUnderConditions-class when base_int == 2 & simple_int != 1
#NOTE: if we're not sure if attributes exist, we can use the `.get(key, default)` method
return { #UI will be built using the following option-classes, each key gets a tab/window:
'always_the_same' : AlwaysTheSame,
'custom_options' : CustomOptionsUnderConditions
}
return { #UI will be built using the following option-classes, each key gets a tab/window:
'always_the_same' : AlwaysTheSame,
'custom_options' : CustomOptionsDefault
} #NOTE: we must ALWAYS return a dictionary with at least 1 option class
if __name__ == '__main__':
local_app(
target_function=example_target_function,
options_source=deduce_new_option_classes,
workspace_path = os.path.join(os.getcwd(), "ExampleCallableOptions")
)
Target Function
The target function is the function that does all the work. This is the function that is being called when an item starts "running" in the Run-Queue.
It takes a single argument: a Configuration
-instance - instance.
The configuration-object contains all settings as set by the user when "add to queue" was pressed.
This example uses the example-configuration from the Configurtion section, we simply print the values of the configuration to the console:
def target_function(configuration: Configuration):
#Do something with the configuration
print(configuration.simple_int)
print(configuration.some_other_int)
#etc.
If you have replaced an argparse.Argumentparser
in the previous example, this is the place where you insert the user-provided settings to the script that uses the ArgumentParser
-object. For example:
# parsed_args = parser.parse_args() #will be done by user in UI
# your_framework_that_used_parsed_args(parsed_args) #Will be called in target_function
def target_function(configurtion : Configuration):
# Since we can use the Configuration-instance as a dict
# and as configuration.<attribute> we can just
# pass it to the framework compatible with the ArgumentParser:
your_framework_that_used_parsed_args(configuration)
Configuration
Configurun works with configuration
-objects. A configuration is a collection of option-instances (=@dataclass
-instances that inherit from BaseOptions
), which are grouped toghether in a Configuration
-wrapper.
We can think of the option-instances as the different groups of options we want to edit and use in our run (e.g. GeneralOptions()
, LogOptions()
, ModelOptions()
, etc.).
In the simplest case, we have 1 single option-instance which contains all the options, AllOptions()
.
The Configuration
-wrapper enables us to access the attributes of all enclosed options-instances using configuration[attribute]
/configuration.<attribute>
/option_class.get(attribute, default)
.
An example of how to use a Configuration
-instance:
from dataclasses import dataclass
from configurun.configuration import Configuration
from configurun.configuration import BaseOptions
@dataclass
class GeneralOptionsClass(BaseOptions):
simple_int : int = 1
#etc.
@dataclass
class OtherOptionClass(BaseOptions):
some_other_int : int = 2
#etc.
config = Configuration()
config.options['general_options'] = GeneralOptionsClass()
config.options['other_options'] = OtherOptionClass()
#Accessing the options, all of the following are equivalent:
print(config['simple_int'])
print(config.simple_int)
print(config.get('simple_int', -1)))
#These are also equivalent:
print(config['some_other_int'])
print(config.some_other_int)
print(config.get('some_other_int', -1)))
# Note that we can use the config.<attr>-notation to our advantage
# when we want to use autocomplete in our editor. For example:
# def target_function(configuration: GeneralOptionsClass)
# Would result in our editor of choice recognizing/autocompleting
# the `configuration.simple_hint` way of accessing `simple_hint`
Option metadata
The UI is mainly built around the field()
functionality of python-dataclass
, which allows the display-model to make use of the default values, type hints and other information.
For each attribute in our option
-definition, we can provide additional information in the metadata
attribute of field()
. This provides additional information to the UI, which is used to determine the editor-type, constraints etc.
For example:
from configurun.configuration import base_options
from dataclasses import field, dataclass
#Used to constrain the editors: (can also be imported from sklearn)
from pyside6_utils.utility.constraints import Interval
@dataclass
class TestOptions(BaseOptions):
test_int_property : int | None = field(
default=None, #The default value used in the UI
metadata=dict( #Contains additional information for the UI
display_name="Test property", #The display-name
help="This is a test property that can also be none", #On-hover help-messagem
required=True, #If required, the field is red if not filled in
constraints = [ #Limit editors (min/max, options, etc.)
#The following constrains the editor to have value > 1
Interval(type=int, left=1, right=None, closed="both"),
None #Or value can be None
]
# etc...
)
)
For more examples, please see the example-options.
The following metadata-keys are supported:
Metadata Key | Type | Description |
---|---|---|
"display_name" |
str |
Name to display for this attribute in the view - defaults to the variable name itself |
"display_path" |
str |
Path to display this attribute - we can group/structure items. If parent does not exist, creates folders. Format as " |
"help" |
str |
Help-message which will be shown when the user hovers over this item - empty by default |
"constraints" |
List[constraint] |
Additional constraints on which the editor will be determined to apply to the field**, if none provided, use typehint of the field |
"required" |
bool |
Whether this field is required to be filled in - if true - a red background will appear if the value is not set |
"editable" |
bool |
Whether this field is editable - if false - the editor will be disabled |
**: Constraints are sourced from the sklearn.utils._validation
module and provides a way to constrain the dataclass fields such that the user can only enter valid values. They are also packed into the pyside6-utils package under utility.constraints
. The following constraints are supported:
Constraint | Description | Editor Type |
---|---|---|
type |
The type of the value should match the type of the constraint | based on type |
Options / Container |
The value should be one of the options provided in the constraint | QComboBox |
StrOptions |
The value should be one of the str-options provided in the constraint | QComboBox |
Interval |
The value should be within the interval provided in the constraint | QSpinBox or QDoubleSpinBox (limited) |
None |
None is a valid value for this field (same as typing.Optional ) |
Adds reset-button to editor |
Range |
The value should be within the range provided in the constraint | QSpinBox (limited) |
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for configurun-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5d7e289bd2865403a66ff11cdc64cf9267e8e3053067b52b7ec257b26223b9e3 |
|
MD5 | 0bec1702ef33745eb01cb7d20a9af0d2 |
|
BLAKE2b-256 | 8d78e9fa7ff5222cc72093802e03cfd2c04e4b25d85162efc91364c84a4ad7cf |