kraken-core 0.5.0

No project description provided

kraken-core

The kraken-core package provides the primitives to describe a dependency graph for the purpose of task orchestration.

Packages

• kraken.api – This module can be imported from in a .kraken.py build script to get access to the current build context and project.
• kraken.core – The core API that consists of primitives to describe tasks with dependencies, as well as Pytest fixtures.

Concepts

• Context: The build context is the "root object" which contains a reference to the root project as well as the path to a designated build directory. The context can hold metadata that is addressable globally by the Python type (see Context.metadata).
• Project: A project represents a directory on the file system and tasks that are associated with the contents of that directory and the build script loaded from it. A project's members are named and either Tasks or other Projects. A project is uniquely identified by it's "path" which is similar to a filesystem path only that the separator is a colon (:). The root project is identifier with just a single colon, while members of a project are identified by concatenating the project path with the member name (such as :subproject:task).
• Task: A task is a unit of work that can is related to a project. Tasks can have relationships to other tasks that describe whether it needs to run before or after the related task. The relationship can also be strict (default) or optional, in which case only the order of execution is informed. Tasks have properties that when passed to properties of other tasks inform a strict dependency relationship.
• Task factory: A task factory is a function that is a convenient utility for Kraken build scripts to define one or more tasks in a project. The Project.do() method in particular is often used to create task, allowing users to conveniently set task property values directly instead of interfacing with the property API.
• Group tasks: Group tasks are a special kind of task that store a list of tasks as their dependencies, effectively grouping the tasks under their name. There is some special treatment for group tasks when the task graph is constructed, but otherwise they behave like normal tasks that don't actually do any work themselves. Every Kraken project always has the following groups by default: fmt, lint, build and test.
• Property: A property is a typed container for a value. It can receive a static value or another task's property to inform a strict dependency relationship between the property owners. Properties have a .set(value), .get() and .get_or() method.
• Task graph: The task graph represents a fully wired graph of the tasks in a context. The task graph must only be constructed after Context.finalize() was called to allow tasks to perform one final update before nothing can be changed anymore. After constructing a graph from a set of initially required tasks, it only contains the tasks that are transitively required by the initial set. The graph can be further trimmed to remove weakly connected components of the graph (such as group tasks if they were of the initial set or dependencies that are not strictly required by any other task).
• Task selector: A task selector is an absolute or relative task or project path (example :, :task, :project:task, task, project:task). On the command line, all tasks are resolved relative to the root project. Within a project, absolute task paths are resolved within that project (i.e. : represents the current project and not the root project). A plain task name without a separator (:) selects all task with that name in all projects. A selector can be suffixed with a question mark to mark it as optional, allowing it to resolve to no tasks instead of raising an error.

Example

Check out the example/ directory.

Remarks for writing extensions

Use typing aliases when defining Task properties for pre-3.10 compatibility

The Kraken code base uses the 3.10+ type union operator | for type hints where possible. However, special care needs to be taken with this operator when defining properties on Kraken tasks. The annotations on task objects are eveluated and will cause errors in Python versions lower than 3.10 when using the union operator | even with __future__.annotations enabled.

The following code will cause a TypeError when executed even when using from __future__ import annotations:

class MyTask(Task):
    my_prop: Property[str | Path]  # unsupported operand type(s) for |: 'type' and 'type'

Property value adapters

Properties will perform runtime type checking when passing a static value into it. The type checking is currently limited to the root data type and does not take type parameters and sub-structures into account (e.g. a List[str] property will validate that the received value is a list but that that it's items are str).

The runtime type check is not currently performed when a supplier (another property for example) is passed into the property instead.

In addition to type checks, "value adapters" can be registered using the Property.value_adapter() decorator. A default adapter is registered for the Path type which will coerce string values to path objects. This means that the order the order of the union members matters:

a: Property[Union[str, Path]]
b: Property[Union[Path, str]]

# ...

a.set("foo")
assert a.get() == "foo"

b.set("foo")
assert b.get() == Path("foo")