Make-like build tool & task runner for Python
Project description
Gird
Gird is a Make-like build tool & task runner for Python.
Gird can be used to manage any project where some tasks need to be executed automatically when some dependencies are updated. In Make, this usually means updating files based on changes in other files. Compared to Make, Gird provides
- a simple Python interface,
- good Python integrability, and
- additional dependency management features.
Comparison with other tools
- Make has many features that could be replaced by a comprehensive Python integration/API (Gird). Dedicated single-purpose rule definition syntax.
- doit's API is unclear/absent. The database can more or less be a black box.
- Invoke doesn't support file-based update chaining.
- Snakemake & CMake are heavyweight & complicated with too many responsibilities.
- See https://wiki.python.org/moin/ConfigurationAndBuildTools
Installation
Install Gird from PyPI with pip install gird
, or from sources with
pip install .
.
Requirements
Gird is built & tested for Python versions 3.8 & above.
Gird requires Make to be available on the system. Most versions of Make
will do, as long as they support the .PHONY
& .ONESHELL
special targets.
Usage
Define "rules" in girdfile.py. Depending on the composition of a rule definition, the rule can, for example,
- define a recipe to run a task, e.g., to update a target file,
- define prerequisites for the target, such as dependency files or other rules, and
- use Python functions for more complex target & recipe functionality.
A rule is invoked by gird <target_name>
. To list all targets, run
gird --list
.
Example girdfile.py
This is the girdfile.py of the project itself.
from itertools import chain
from pathlib import Path
from gird import Phony, rule
from scripts import assert_readme_updated, get_wheel_path, render_readme
WHEEL_PATH = get_wheel_path()
rule_pytest = rule(
target=Phony("pytest"),
recipe="pytest",
help="Run pytest.",
)
rule_assert_formatting = rule(
target=Phony("assert_formatting"),
recipe=[
"black --check gird scripts girdfile.py",
"isort --check gird scripts girdfile.py",
],
help="Check formatting with Black & isort.",
)
rule_assert_readme_updated = rule(
target=Phony("assert_readme_updated"),
recipe=assert_readme_updated,
help="Check that README.md is updated based on README_template.md.",
)
rules_test = [
rule_pytest,
rule_assert_formatting,
rule_assert_readme_updated,
]
rule(
target=Phony("test"),
deps=rules_test,
help="\n".join(f"- {rule.help}" for rule in rules_test),
)
rule(
target=Path("README.md"),
deps=list(
chain(
*(Path(path).iterdir() for path in ("scripts", "gird")),
[Path("girdfile.py")],
),
),
recipe=render_readme,
help="Render README.md based on README_template.md.",
)
rule(
target=WHEEL_PATH,
recipe="poetry build --format wheel",
help="Build distribution packages for the current version.",
)
rule(
target=Phony("publish"),
deps=WHEEL_PATH,
recipe=f"twine --repository gird upload {WHEEL_PATH}",
help="Publish packages of the current version to PyPI.",
)
Respective output from gird --list
:
pytest
Run pytest.
assert_formatting
Check formatting with Black & isort.
assert_readme_updated
Check that README.md is updated based on README_template.md.
test
- Run pytest.
- Check formatting with Black & isort.
- Check that README.md is updated based on README_template.md.
README.md
Render README.md based on README_template.md.
dist/gird-1.2.3-py3-none-any.whl
Build distribution packages for the current version.
publish
Publish packages of the current version to PyPI.
Example rules
A rule with files as its target & dependency. When the rule is invoked, the recipe is executed only if the dependency file has been or will be updated, or if the target file doesn't exist.
import pathlib
import gird
wheel = pathlib.Path("package.whl")
rule_build = gird.rule(
target=wheel,
deps=pathlib.Path("module.py"),
recipe=f"python -m build --wheel",
)
A (phony) rule with no target file. Phony rules are always executed when invoked.
rule_test = gird.rule(
target=gird.Phony("test"),
deps=wheel,
recipe="pytest",
)
A rule with other rules as dependencies, to group multiple rules together, and to set the order of execution between rules.
gird.rule(
target=gird.Phony("all"),
deps=[
rule_test,
rule_build,
],
)
A rule with a Python function recipe.
import json
JSON1 = pathlib.Path("file1.json")
JSON2 = pathlib.Path("file2.json")
def create_target():
JSON2.write_text(
json.dumps(
json.loads(
JSON1.read_text()
).update(value2="value2")
)
)
gird.rule(
target=JSON2,
deps=JSON1,
recipe=create_target,
)
A Python function as a dependency to arbitrarily trigger rules.
import datetime
EPOCH = datetime.datetime(2030, 1, 1)
@gird.dep
def unconditional_until_epoch():
"""Return the "updated" state of this dependency. Here, render a
depending target outdated (trigger its recipe to be executed) always
before EPOCH.
"""
return datetime.datetime.now() < EPOCH
gird.rule(
target=JSON2,
deps=[JSON1, unconditional_until_epoch],
recipe=create_target,
)
Compound recipes for, e.g., setup & teardown. All subrecipes of a rule are run in a single shell instance.
gird.rule(
target=JSON2,
deps=JSON1,
recipe=[
"export VALUE2=value2",
create_target,
"unset VALUE2",
],
)
Define rules in a loop, or however you like.
rules = [
gird.rule(
target=source.with_suffix(".json.gz"),
deps=source,
recipe=f"gzip -k {source.resolve()}",
)
for source in [JSON1, JSON2]
]
Implementation of Gird
Internally, Gird generates Makefiles & uses Make to run tasks, but interacting with Make in any way isn't obligatory when using Gird. In the future, Make as a dependency of Gird might be replaced altogether.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.