Your faithful companion
Project description
Zrb (WIP): Your Faithful Companion
Zrb is a task runner to help you automate day-to-day tasks.
Installation
pip install zrb
Create a project
A project is a directory containing a Python file named zrb_init.py
.
The recommended way to create a Zrb project is by using Zrb built-in generator. To create a Zrb project using the built-in generator, you can invoke the following command:
zrb project create --project-dir=my-project
To start working on the project, you can invoke the following command:
source project.sh
The command will make you a Python virtual environment, as well as install necessary Python packages.
Create a very minimal project
Aside from the built-in generator, you can also make a project manually by invoking the following command:
mkdir my-project
cd my-project
touch zrb_init.py
This might be useful for demo/experimentation.
Define tasks
Zrb comes with many types of tasks:
- Python task
- Cmd task
- Docker compose task
- Http checker
- Port Checker
- Path Checker
- Resource maker
Every task has its inputs, environments, and upstreams. By defining the upstreams, you can make several tasks run in parallel. Let's see the following example:
install-pip --------------------------------------> run-server
|
install-node-modules ---> build-frontend ----
# File location: zrb_init.py
from zrb import CmdTask, HttpChecker, EnvFile, runner
# Install pip package for requirements.txt in src directory
install_pip_packages = CmdTask(
name='install-pip',
cmd='pip install -r requirements.txt',
cwd='src'
)
# Install node modules in src/frontend directory
install_node_modules = CmdTask(
name='install-node-modules',
cmd='npm install --save-dev',
cwd='src/frontend'
)
# Build src/frontend
# To build the frontend, you need to make sure that node_modules has already been installed.
build_frontend = CmdTask(
name='build-frontend',
cmd='npm run build',
cwd='src/frontend',
upstreams=[install_node_modules]
)
# Start the server.
# In order to start the server, you need to make sure that:
# - Necessary pip packages has been already installed
# - Frontend has already been built
# By default it should use environment defined in `src/template.env`.
# You can set the port using environment variable WEB_PORT
# This WEB_PORT environment will be translated into PORT variable internally
# You can use the port to check whether a server is ready or not.
run_server = CmdTask(
name='run-server',
envs=[
Env(name='PORT', os_name='WEB_PORT', default='3000')
],
env_files=[
EnvFile(env_file='src/template.env', prefix='WEB')
]
cmd='python main.py',
cwd='src',
upstreams=[
install_pip_packages,
build_frontend
],
checkers=[HTTPChecker(port='{{env.PORT}}')],
)
runner.register(run_server)
Once defined, you can start the server by invoking the following command:
zrb run-server
Zrb will make sure that the tasks are executed in order based on their upstreams.
You will also see that install-pip-packages
and install-node-modules
are executed in parallel since they are independent of each other.
Define a Python task
Defining a Python task is simple.
from zrb import python_task, Env, StrInput, runner
@python_task(
name='say-hello',
inputs=[
StrInput(name='name')
],
envs=[
Env(name='PYTHONUNBUFFERED', default=1)
],
runner=runner
)
def say_hello(*args, **kwargs) -> str:
name = kwargs.get('name')
greetings = f'Hello, {name}'
task = kwargs.get('_task')
task.print_out(greetings)
return greetings
You can then run the task by invoking:
zrb say-hello --name=John
Python task is very powerful to do complex logic. You can also use async
function if you think you need to.
Define a Cmd task
You can define a Cmd task by using CmdTask
class.
from zrb import CmdTask, StrInput, Env, runner
say_hello = CmdTask(
name='say-hello',
inputs=[
StrInput(name='name')
],
envs=[
Env(name='SOME_ENV')
],
cmd='echo {{input.name}}'
)
runner.register(say_hello)
If you need a multi-line command, you can also define the command as a list:
from zrb import CmdTask, StrInput, Env, runner
say_hello = CmdTask(
name='say-hello',
inputs=[
StrInput(name='name')
],
envs=[
Env(name='SOME_ENV')
],
cmd=[
'echo {{input.name}}',
'echo Yeay!!!'
]
)
runner.register(say_hello)
However, if your command is too long, you can also load it from other file:
from zrb import CmdTask, StrInput, Env, runner
say_hello = CmdTask(
name='say-hello',
inputs=[
StrInput(name='name')
],
envs=[
Env(name='SOME_ENV')
],
cmd_path='hello_script.sh'
)
runner.register(say_hello)
You can then run the task by invoking:
zrb say-hello --name=John
Define a Docker Compose task
Docker Compose is a convenient way to run containers on your local computer.
Suppose you have the following Docker Compose file:
# docker-compose.yml file
version: '3'
services:
# The load balancer
nginx:
image: nginx:1.16.0-alpine
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "${HOST_PORT:-8080}:80"
You can define a task to run your Docker Compose file (i.e., docker compose up
) like this:
from zrb import DockerComposeTask, HTTPChecker, Env, runner
run_container = DockerComposeTask(
name='run-container',
compose_cmd='up',
compose_file='docker-compose.yml',
envs=[
Env(name='HOST_PORT', default='3000')
],
checkers=[
HTTPChecker(
name='check-readiness', port='{{env.HOST_PORT}}'
)
]
)
runner.register(run_container)
Define checkers
Some tasks might run forever, and you need a way to make sure whether those tasks are ready or not.
Let's say you invoke npm run build:watch
. This command will build your Node.js App into dist
directory, as well as watch the changes and rebuild your app as soon as there are some changes.
- You need to start the server after the app has been built for the first time.
- You can do this by checking whether the
dist
folder already exists or not. - You can use
PathChecker
for this purpose
Let's see how to do this:
from zrb import CmdTask, PathChecker, Env, EnvFile, runner
build_frontend = CmdTask(
name='build-frontend',
cmd='npm run build',
cwd='src/frontend',
checkers=[
PathChecker(path='src/frontend/dist')
]
)
run_server = CmdTask(
name='run-server',
envs=[
Env(name='PORT', os_name='WEB_PORT', default='3000')
],
env_files=[
EnvFile(env_file='src/template.env', prefix='WEB')
]
cmd='python main.py',
cwd='src',
upstreams=[
build_frontend
],
checkers=[HTTPChecker(port='{{env.PORT}}')],
)
runner.register(run_server)
Aside from PathChecker
, Zrb also has HTTPChecker
and PortChecker
.
Define a resource maker
ResourceMaker is used to generate resources. Let's say you have a template
folder containing a file named app_name.py
:
# file: template/app_name.py
message = 'Hello world_name'
print(message)
You can define a ResourceMaker like this:
from zrb import ResourceMaker, StrInput, runner
create_hello_world = ResourceMaker(
name='create-hello-world',
inputs=[
StrInput('app-name'),
StrInput('world-name'),
],
replacements={
'app_name': '{{input.app_name}}',
'world_name': '{{input.world_name}}',
},
template_path='template',
destination_path='.',
)
runner.register(create_hello_world)
Now when you invoke the task, you will get a new file as expected:
zrb create-hello-world --app-name=wow --world-name=kalimdor
echo ./wow.py
The result will be:
# file: template/wow.py
message = 'Hello kalimdor'
print(message)
This is a very powerful building block to build anything based on the template.
Using Zrb to build an application (WIP)
You can use Zrb to build a powerful application with a few commands:
# Create a project
zrb project create --project-dir my-project --project-name "My Project"
cd my-project
# Create a Fastapp
zrb project add fastapp --project-dir . --app-name "fastapp" --http-port 3000
# Add library module to fastapp
zrb project add fastapp-module --project-dir . --app-name "fastapp" --module-name "library"
# Add entity named "books"
zrb project add fastapp-crud --project-dir . --app-name "fastapp" --module-name "library" \
--entity-name "book" --plural-entity-name "books" --column-name "code"
# Add column to the entity
zrb project add fastapp-field --project-dir . --app-name "fastapp" --module-name "library" \
--entity-name "book" --column-name "title" --column-type "str"
# Run Fastapp
zrb project start-fastapp
# Run Fastapp as container
zrb project start-fastapp-container
# Deploy fastapp
zrb project deploy-fastapp
You should notice that every module in fastapp
can be deployed/treated as microservices.
Autoloaded tasks
Zrb will automatically load the following task definitions:
- Every task definition in
ZRB_INIT_SCRIPTS
.- You can use a colon separator (
:
) to define multiple scripts inZRB_INIT_SCRIPTS
. For example:ZRB_INIT_SCRIPTS=~/personal/zrb_init.py:~/work/zrb_init.py
- You can use a colon separator (
- Every task definition in
zrb_init.py
in your current directory.- If Zrb cannot find any in your current directory, it will look at the parent directories until it finds one.
- Every built-in task definition given
ZRB_SHOULD_LOAD_BUILTIN
equals1
or unset.
How to run tasks programmatically
To run a task programmatically, you need to create a main loop
.
For example:
from zrb import CmdTask
cmd_task = CmdTask(
name='sample',
cmd='echo hello'
)
main_loop = cmd_task.create_main_loop(env_prefix='')
result = main_loop() # This run the task
print(result.output) # Should be "hello"
Enable shell completion
To enable shell completion, you need to set _ZRB_COMPLETE
variable.
For bash
:
eval $(_ZRB_COMPLETE=bash_source zrb)
For zsh
:
eval $(_ZRB_COMPLETE=zsh_source zrb)
Once set, you will have a shell completion in your session:
zrb <TAB>
zrb md5 hash -<TAB>
Visit click shell completion for more information.
Configuration
The following configurations are available:
ZRB_HOME_DIR
: Zrb home directory.- Default: Zrb home directory
ZRB_LOGGING_LEVEL
: Logging verbosity.- Default:
WARNING
- Possible values:
CRITICAL
ERROR
WARNING
WARN
(The same asWARNING
)INFO
DEBUG
NOTSET
- Default:
ZRB_INIT_SCRIPTS
: List of task registration script that should be loaded by default.- Default: Empty
- Possible values: List of script paths, separated by colons(
:
). - Example:
~/personal/zrb_init.py:~/work/zrb_init.py
ZRB_ENV
: Environment prefix that will be used when loading Operating System's environment.- Default: Empty
- Possible values: Any combination of alpha-numeric and underscore
- Example:
DEV
ZRB_SHOULD_LOAD_BUILTIN
: Whether load builtin tasks or not- Default:
1
- Possible values:
1
0
- Default:
ZRB_SHELL
: Default shell for running cmdTask- Default:
bash
- Possible value:
/usr/bin/bash
/usr/bin/sh
- Default:
ZRB_SHOW_ADVERTISEMENT
: Whether show advertisement or not.- Default:
1
- Possible value:
1
0
- Default:
ZRB_SHOW_PROMPT
: Whether show prompt or not.- Default:
1
- Possible value:
1
0
- Default:
Quirks
- No one is sure how to pronounce Zrb. Let's keep it that way.
- If not set,
PYTHONUNBUFFERED
will be set to1
. - Once
zrb_init.py
is loaded, Zrb will automatically:- Set
ZRB_PROJECT_DIR
tozrb_init.py
's parent directory. - If loaded as CLI, Zrb will also:
- Adding
ZRB_PROJECT_DIR
toPYTHONPATH
.
- Adding
- Set
- Zrb passes several keyword arguments that will be accessible from the task's run method:
_args
: Shell argument when the task is invoked._task
: Reference to the current task.
- You can access the built-in command groups by importing
zrb.builtin_group
. - How environments are loaded:
env_files
has the lowest priority, it will be overridden byenv
env
will override each other, the last one takes greater priority- If you define a
DockerComposeTask
, it will automatically fill your environment with the ones you use in your docker-compose file. The environment defined that way will have a very low priority. They will be overridden by bothenv_files
andenv
.
For contributors
There is a toolkit you can use to test whether Zrb is working as intended.
To use the toolkit, you can invoke the following command:
source ./project.sh
Once you load the toolkit, you can start playing around.
# Run test and serve coverage.
zrb test
# Test zrb in playground
zrb prepare-playground
For maintainers
To publish Zrb, you need a Pypi
account:
- Log in or register to https://pypi.org/
- Create an API token
You can also create a TestPypi
account:
- Log in or register to https://test.pypi.org/
- Create an API token
Once you have your API token, you need to create a ~/.pypirc
file:
[distutils]
index-servers =
pypi
testpypi
[pypi]
repository = https://upload.pypi.org/legacy/
username = __token__
password = pypi-xxx-xxx
[testpypi]
repository = https://test.pypi.org/legacy/
username = __token__
password = pypi-xxx-xxx
To publish Zrb, you can do the following:
source ./project.sh
# Publish Zrb to TestPypi
zrb publish-test
# Publish Zrb to Pypi
zrb publish
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.