inex-launcher 3.1.11

InEx is a lightweight highly configurable Python launcher based on microkernel architecture

InEx Launcher

InEx (Initialize & Execute) is a lightweight, highly configurable Python launcher based on a microkernel architecture. You describe a pipeline in a single YAML (or JSON) file: a ordered list of plugins builds Python objects, then an execute block runs the final function.

Use it to build CLI utilities, data pipelines, training jobs, and inference tools without writing bespoke entry-point scripts for every experiment.

---

Table of contents

• Installation
• Quick start
• Command-line interface
• Configuration overview
• Writing Python modules
• Module reference syntax
• imports and options
• exports
• Plugin features
• Built-in resolvers
• inex.helpers
• Third-party modules
• Examples
• Test suite
• Architecture

---

Installation

pip install -U inex-launcher

Requirements: Python ≥ 3.8, omegaconf, networkx.

Development install from source:

git clone https://github.com/speechpro/inex-launcher.git
cd inex-launcher
pip install -e .
python -m pytest tests/

---

Quick start

1. Python module (myproject/cli.py):

def run(loader, epochs: int):
    data = loader.load()
    print(f"Training for {epochs} epochs on {len(data)} samples")

2. Config (config.yaml):

#!/bin/env inex

epochs: 10

plugins:
  - loader

loader:
  module: myproject.data/Loader
  options:
    path: data/train.csv

execute:
  method: myproject.cli/run
  imports:
    loader: plugins.loader
  options:
    epochs: ${epochs}

3. Run:

inex config.yaml
inex config.yaml -u epochs=20
inex config.yaml -l DEBUG -s .

The -s . flag adds the current directory to sys.path so myproject is importable.

---

Command-line interface

inex [-h] [--version] [--log-level LOG_LEVEL] [--log-path LOG_PATH]
     [--sys-path SYS_PATH] [--merge MERGE] [--update UPDATE]
     [--stop-after STOP_AFTER] [--final-path FINAL_PATH]
     config_path

Option	Short	Description
config_path	—	Path to YAML/JSON config, or inline YAML string
--log-level	-l	Root logger level (DEBUG, INFO, WARNING, …)
--log-path	-g	Write log to file
--sys-path	-s	Paths to append to sys.path (separated by :, ;, ,, or | on Linux)
--merge	-m	Merge additional config file(s); repeatable
--update	-u	Override values using dot notation (key.subkey=value); repeatable
--stop-after	-a	Stop after initializing the named plugin (debugging)
--final-path	-f	Write fully resolved config to file

Examples:

# Merge override file and set nested values
inex config.yaml -m experiment.yaml -u trainer.epochs=20 model.hidden=64

# Dump resolved config without running execute (stop after last plugin)
inex config.yaml -a trainer -f final_config.yaml

# Config can also be inline YAML
inex "plugins: []"

Config-level alternatives to CLI flags:

__log_level__: INFO
__sys_path__: .

---

Configuration overview

A typical config has four layers:

#!/bin/env inex              # optional shebang — marks file as inex config

# 1. Top-level parameters (hyperparameters, paths, flags)
batch_size: 32
data_path: ???                # required — must be set via -u or merge

# 2. Special keys
__log_level__: INFO
__mute__: [__all__]           # suppress per-plugin debug logs

# 3. Plugin chain (initialization order)
plugins:
  - loader
  - model
  - trainer

loader:
  module: myproject.data/Loader
  options:
    path: ${data_path}

model:
  module: myproject.nn/Model
  options:
    hidden: 128

trainer:
  module: myproject.training/Trainer
  imports:
    model: plugins.model
    data: plugins.loader
  options:
    epochs: 10

# 4. Final execution
execute:
  method: myproject.training/run
  imports:
    trainer: plugins.trainer

Config keys reference

Key	Scope	Purpose
plugins	top	Ordered list of plugin names to initialize
execute	top	Final method to call after all plugins
module	plugin	Python class or function to instantiate
method	execute	Same as module — function to call at the end
options	plugin / execute	Keyword arguments (and __args__ / __kwargs__)
imports	plugin / execute	Wire values from plugin state
exports	plugin	Publish attributes as plugin_name.attr
depends	plugin	Explicit dependency list (usually auto-inferred)
is_done	plugin	Skip plugin if marker file exists
before / after	plugin	Filesystem hooks (exists, mkdir, delete)
title	plugin	Print banner when plugin is created
__log_level__	top	Logging level
__sys_path__	top	Extra sys.path entries
__mute__ / __unmute__	top	Control debug logging per plugin

If plugins: is omitted, inex auto-detects plugin keys as any top-level dict containing module or method.

Placeholders: use ??? for values that must be overridden before resolution (via -u or --merge).

Interpolation: standard OmegaConf syntax — ${param}, ${..parent}, ${list[0]}.

---

Writing Python modules

Design Python code so its constructor or function signature matches what the YAML will pass via options and imports.

Binding styles

YAML	Python	Result
pkg.mod/MyClass	class MyClass: def __init__(self, ...)	Instantiate class
pkg.mod/my_func	def my_func(...)	Call function
pkg.mod/Class.method	class method	Call bound method
pkg.mod (no /Name)	def create(*args, **kwargs)	Call module factory
plugins.prev/method	instance method	Call method on existing plugin
plugins.model/parameters	attribute	Access attribute on plugin instance

Class plugin:

# myproject/data.py
class Loader:
    def __init__(self, path: str):
        self.path = path

    def load(self):
        ...

loader:
  module: myproject.data/Loader
  options:
    path: data/train.csv

Function plugin:

# myproject/utils.py
def return_value(value):
    return value

value1:
  module: myproject.utils/return_value
  options:
    value: 5

Method on prior plugin:

class Tokenizer:
    def Load(self, model_file: str):
        ...

load_tokenizer:
  module: plugins.tokenizer/Load
  options:
    model_file: model.bin

Factory (uncommon):

def create(**kwargs):
    return MyClass(**kwargs)

handler:
  module: myproject.handlers
  options:
    mode: train

Design guidelines

1. Plugin objects → imports; literals and config values → options.
2. Parameter names in Python must match keys in imports and options.
3. Each plugin should return the object downstream steps need.
4. Prefer explicit /ClassName or /function over bare pkg.mod + create().
5. Use exports only for attributes referenced as plugin_name.attr in the config (see exports).

---

Module reference syntax

Format: package.module/Name

myproject.data/Loader          → class Loader from myproject.data
myproject.cli/run              → function run from myproject.cli
plugins.loader/process         → call .process() on plugins.loader instance
plugins.model/parameters       → access .parameters on plugins.model
numpy/array                    → third-party constructor
/max                           → Python builtin max()

Indexing: append ^N to take the N-th element: plugins.array^0 → plugins.array[0].

State keys after plugin foo is created:

Key	Content
plugins.foo	plugin return value
foo.attr	each name listed in exports

---

imports and options

imports — wire plugin state

trainer:
  module: myproject.training/Trainer
  imports:
    model: plugins.model          # whole plugin instance
    data: plugins.loader
    waveform: audio.waveform      # exported attribute (requires exports)

List form passes positional arguments:

object1:
  module: myproject.utils/Object
  imports:
    __args__: [1, 2, [3, 4]]

options — literals and configuration

loader:
  module: myproject.data/Loader
  options:
    path: ${data_path}
    batch_size: 32

Expand a dict into kwargs:

loader_opts:
  batch_size: 32
  shuffle: true

loader:
  module: torch.utils.data.dataloader/DataLoader
  options:
    __kwargs__: ${loader_opts}

Positional arguments:

array:
  module: numpy/array
  options:
    __args__: [[1, 2, 3]]

A bare list as options is treated as __args__:

array:
  module: numpy/array
  options: [[1, 2, 3]]

---

exports

The exports field publishes selected attributes from a plugin instance so other plugins can import them as plugin_name.attribute_name.

Rules

1. Include a property in exports only if some imports block in the same config references it as plugin_name.property_name.
2. Exclude properties that are not referenced that way.
3. Runtime error if a reference like audio.waveform exists but waveform is missing from the audio plugin's exports.

Importing the whole plugin (plugins.audio) does not require individual properties in exports.

Whole plugin — no exports

audio:
  module: myproject.audio/MonoWaveform
  options:
    audio_path: ${audio_path}

execute:
  method: myproject.cli/run
  imports:
    audio: plugins.audio

Attribute references — exports required

audio:
  module: myproject.audio/MonoWaveform
  exports: [waveform, sample_rate]
  options:
    audio_path: ${audio_path}

segmenter:
  module: myproject.audio/Segmenter
  imports:
    waveform: audio.waveform
    sample_rate: audio.sample_rate

Special export values

Value	Effect
[model]	publish plugin_name.model
[length]	publish plugin_name.length (e.g. for schedulers)
[__all__]	publish all non-callable public attributes

Example from tests/test_basic.yaml:

value2:
  module: tests.test_basic/Value
  exports: [value]
  options:
    value: 7

execute:
  method: inex.helpers/assign
  imports:
    value:
      - plugins.value1
      - value2.value

---

Plugin features

Skip-if-done (is_done)

Skip plugin initialization when a marker file already exists; touch the file after success.

is_done: exp/run/.done

stage1:
  module: myproject.pipeline/run
  is_done: ${is_done}
  options:
    ...

Filesystem hooks (before / after)

stage1:
  module: myproject.pipeline/run
  before:
    exists: input/data.txt
    mkdir: output/
    delete: [output/old.txt, temp/]
  after:
    delete: [temp/]

Commands: exists, mkdir, delete.

Dependency graph

bind_plugins() auto-builds dependencies from:

• imports values referencing plugins.X or X.attr
• module: plugins.X/... references
• explicit depends: [...] lists

Plugins must appear in plugins: before anything that imports them.

---

Built-in resolvers

Registered at startup; use in config values as ${__name__:...}:

Resolver	Example
__evaluate__	${__evaluate__:'{a} + {b}', a: 2, b: 3}
__fetch__	${__fetch__:other.yaml} or ${__fetch__:other.yaml, key.subkey}
__getenv__	${__getenv__:HOME} or ${__getenv__:PORT, int}
__setenv__	${__setenv__:MY_VAR, value}
__read_text__	${__read_text__:file.txt}
__num_lines__	${__num_lines__:file.txt}
__path_parent__	${__path_parent__:${config_path}}
__path_name__	filename with extension
__path_stem__	filename without extension
__path_suffix__	file extension
__path_is_file__	assert path is a file
__path_is_dir__	assert path is a directory
__path_exists__	assert path exists

Preflight checks:

exists:
  - ${__path_is_file__:${data_path}}

See tests/test_resolvers.yaml for a full resolver demo.

---

inex.helpers

Built-in utilities available as inex.helpers/FunctionName:

Function	Purpose
assign	Return value unchanged; bundle results in execute
evaluate	Evaluate Python expression (expression, optional initialize)
_import_	Load a plugin from another inex config (cached per config path)
compose	Merge configs; optionally write result to file
attribute	Load attribute from module (modname, attname)
show	Debug-print plugin values (type, id, value)
stage	Run a sub-config with done-mark and filesystem checks
execute	Run external command via subprocess
system	Run shell command

_import_ — reuse plugins from another config:

blank_model:
  module: inex.helpers/_import_
  options:
    plugin: model
    config: ${model_dir}/final_config.yaml

model:
  module: myproject.lightning/load_model
  imports:
    model: plugins.blank_model
  options:
    ckpt_path: ${ckpt_path}

compose — merge and write config:

write_params:
  module: inex.helpers/compose
  options:
    config: ${params}
    result_path: exp/params.yaml

evaluate — derived values:

total_steps:
  module: inex.helpers/evaluate
  imports:
    epoch_size: train_dataset.length
  options:
    expression: 'int(1.001 * {num_epochs} * {epoch_size})'
    num_epochs: ${num_epochs}

---

Third-party modules

Any importable Python module can be used directly:

device:
  module: torch/device
  options:
    device: cuda

loader:
  module: torch.utils.data.dataloader/DataLoader
  imports:
    dataset: plugins.dataset
  options:
    batch_size: 32

execute:
  method: torch/save
  imports:
    obj: plugins.state_dict
  options:
    f: ${save_path}

Built-in Python callables use a leading /:

max_val:
  module: /max
  options:
    __args__: [1, 2]

---

Examples

Minimal — function and class with exports

From tests/test_basic.yaml:

plugins:
  - value1
  - value2

value1:
  module: tests.test_basic/return_value
  options:
    value: 5

value2:
  module: tests.test_basic/Value
  exports: [value]
  options:
    value: 7

execute:
  method: inex.helpers/assign
  imports:
    value:
      - plugins.value1
      - value2.value

Data pipeline with PyTorch

plugins:
  - device
  - dataset
  - loader

device:
  module: torch/device
  options:
    device: cuda

dataset:
  module: myproject.data/MyDataset
  options:
    path: ${data_path}

loader:
  module: torch.utils.data.dataloader/DataLoader
  imports:
    dataset: plugins.dataset
  options:
    batch_size: 32
    shuffle: true

execute:
  method: myproject.cli/process
  imports:
    loader: plugins.loader
    device: plugins.device
  options:
    output_dir: ${output_dir}

Inference with checkpoint reload

plugins:
  - blank_model
  - model
  - features
  - scorer

blank_model:
  module: inex.helpers/_import_
  options:
    plugin: model
    config: ${model_dir}/final_config.yaml

model:
  module: myproject.lightning/load_model
  imports:
    model: plugins.blank_model
  options:
    ckpt_path: ${model_dir}/best.ckpt

features:
  module: myproject.data/FeatureSet
  options:
    path: ${feats_path}

scorer:
  module: myproject.infer/Scorer
  imports:
    model: plugins.model
    features: plugins.features

execute:
  method: myproject.cli/write_scores
  imports:
    scorer: plugins.scorer
  options:
    output_path: ${output_path}

Training with scheduler step count

train_dataset:
  module: myproject.data/Dataset
  exports: [length]
  options:
    path: ${train_path}

total_steps:
  module: inex.helpers/evaluate
  imports:
    epoch_size: train_dataset.length
  options:
    expression: 'int({num_epochs} * {epoch_size})'
    num_epochs: ${num_epochs}

scheduler:
  module: torch.optim.lr_scheduler/OneCycleLR
  imports:
    optimizer: plugins.optimizer
    total_steps: plugins.total_steps
  options:
    max_lr: ${lr}

---

Test suite

The tests/ directory contains contract tests — minimal YAML configs that exercise every feature. Run:

python -m pytest tests/

Test file	Demonstrates
test_basic.yaml	plugins + execute; function vs class; exports
test_args.yaml	__args__ via options and imports
test_kwargs.yaml	__kwargs__ via options and imports
test_pos_args.yaml	mixed positional/kwargs
test_export.yaml	exports; ^index on attributes
test_item.yaml	plugins.value^N indexing
test_import.1–4.yaml	_import_ helper; cache sharing
test_eval.yaml	evaluate; plugin names with +
test_resolvers.yaml	all built-in resolvers
test_fetch.*.yaml	__fetch__ resolver
test_is_done.yaml	is_done; before/after commands
test_compose.*.yaml	??? placeholders; compose
test_built_in.yaml	/max, /eval, /tuple
test_stop_after.yaml	partial init (-a flag)
test_numpy.yaml	third-party module binding

---

Architecture

inex config.yaml
       │
       ▼
  load & resolve (OmegaConf + resolvers)
       │
       ▼
  bind_plugins() ──► dependency graph
       │
       ▼
  for each plugin in plugins[]:
       create_plugin() ──► state['plugins.<name>']
       │
       ▼
  create_plugin('execute') ──► final result

Module	Role
inex/inex.py	CLI, resolvers, start()
inex/engine.py	execute() — plugin loop
inex/utils/configure.py	create_plugin(), bind_plugins(), config loading
inex/helpers.py	assign, compose, _import_, stage, execute, …
inex/utils/fsystem.py	before/after filesystem commands

---

License

MIT — see LICENSE.