dapp-runner 0.3.0

Golem dapp-runner - a high-level interface for running decentralized applications using the Golem Network.

Golem dApp Runner

dapp-runner is a utility that allows you to run decentralized applications on Golem. It uses simple application descriptors expressed in yaml, similar to those used by tools like docker-compose.

dapp-runner runs alongside the Golem daemon and uses yapapi, Golem's Python high-level API to communicate with it. As opposed to using plain yapapi though, deployment of applications on Golem using dapp-runner requires no code and no experience in Python.

GAP-16 / Multi-service application deployment framework

In its present form, the dapp-runner constitutes an initial reference implementation of the multi-service application deployment framework described in GAP-16.

Following features of the framework are currently supported:

• Descriptor "Apply" operation
• Single-YAML package support
• Merging descriptor files
• GAOM explicit dependency syntax
• GAOM object dependency graph [currently limited to the services' explicit dependency syntax]

Relationship with dapp-manager

While the dapp-runner is perfectly capable of running decentralized apps on its own, we are also providing a separate tool to facilitate running and managing multiple applications on a single machine, namely, the dapp-manager.

dApp Manager keeps track of the launched apps and allows you to easily query their output streams. It uses the dapp-runner as its back-end and both require the yagna daemon to communicate with the rest of the Golem Network.

Quick start

Yagna daemon

To run Golem apps, dapp-runner requires a properly configured yagna daemon. In the future, you'll be able to provision apps using external supervisor machines which will run a yagna daemon on your behalf.

For now, please follow the "Requestor development: a quick primer" tutorial and ensure that your yagna is up and running. Only the first part of this tutorial is required - you don't need to run the blender example.

Most importantly, make sure you have set the YAGNA_APPKEY in your evironment, e.g. with:

export YAGNA_APPKEY=insert-your-32-char-app-key-here

or, on Windows:

set YAGNA_APPKEY=insert-your-32-char-app-key-here

and if you don't know what your app-key is, you can always query yagna with:

yagna app-key list

Python environment

First, ensure you have Python 3.8 or later:

python3 --version

[ depending on the platform, it may be just python instead of python3 ]

If your Python version is older, consider using pyenv.

Once your python interpreter reports a version 3.8 or later, you can set-up your virtual environment:

python3 -m venv ~/.envs/dapp-runner
source ~/.envs/dapp-runner/bin/activate

or, if you're on Windows:

python -m venv --clear %HOMEDRIVE%%HOMEPATH%\.envs\dapp-runner
%HOMEDRIVE%%HOMEPATH%\.envs\dapp-runner\Scripts\activate.bat

DApp runner

Clone the repository

git clone --recurse-submodules https://github.com/golemfactory/dapp-runner.git

Install the dependencies

cd dapp-runner
pip install -U pip poetry
poetry install

Run an example application

Make sure your yagna daemon is running, you have initialized the payment driver with yagna payment init --sender, and that you have set the YAGNA_APPKEY environment variable.

Then run:

dapp-runner start --config configs/default.yaml dapp-store/apps/webapp.yaml

You should see the application being deployed on the Golem Network and once it's up, you'll be greeted with:

{"http": {"local_proxy_address": "http://localhost:8080"}}

You can connect to this address using your local browser, and you'll see our minimalistic web application example running.

Press Ctrl-C in the terminal where you ran dapp-runner to initiate its shutdown.

Application descriptor

As mentioned above, the decentralized applications that are deployed on Golem by the dapp-runner are described in yaml files, conforming to the schema described in GAP-16.

Example descriptor

Here's an example application descriptor (http-proxy.yaml), that provisions a single instance of a simple, static website served with nginx:

payloads:
  nginx:
    runtime: "vm"
    params:
      image_hash: "16ad039c00f60a48c76d0644c96ccba63b13296d140477c736512127"
nodes:
  http:
    payload: "nginx"
    init:
        - ["/docker-entrypoint.sh"]
        - ["/bin/chmod", "a+x", "/"]
        - ["/bin/sh", "-c", 'echo "Hello from inside Golem!" > /usr/share/nginx/html/index.html']
        - ["/bin/rm", "/var/log/nginx/access.log", "/var/log/nginx/error.log"]
        - ["/usr/sbin/nginx"]
    http_proxy:
      ports:
        - "80"  # specify just the remote port, allow the local port to be automatically chosen

Web application

And here's an example of a slightly more complex application (webapp.yaml), that uses two kinds of services and explicitly connects them within a specified network:

payloads:
  db:
    runtime: "vm"
    params:
      image_hash: "85021afecf51687ecae8bdc21e10f3b11b82d2e3b169ba44e177340c"
  http:
    runtime: "vm"
    params:
      image_hash: "c37c1364f637c199fe710ca62241ff486db92c875b786814c6030aa1"
nodes:
  db:
    payload: "db"
    init:
      - ["/bin/run_rqlite.sh"]
    network: "default"
    ip:
      - "192.168.0.2"
  http:
    payload: "http"
    init:
      - ["/bin/bash", "-c", "cd /webapp && python app.py --db-address 192.168.0.2 --db-port 4001 initdb"]
      - ["/bin/bash", "-c", "cd /webapp && python app.py --db-address 192.168.0.2 --db-port 4001 run > /webapp/out 2> /webapp/err &"]
    http_proxy:
      ports:
        - "5000"  # specify just the remote port, allow the local port to be automatically chosen
    network: "default"
    ip:
      - "192.168.0.3"
    depends_on:
      - "db"
networks:
  default:
    ip: "192.168.0.0/24"

Implicit properties

The networks definition and the vpn capability

As can be seen in the http_proxy example above, the networks definition can be omitted.

Adding a http_proxy element to a nodes entry, causes the dapp-runner to implicitly add the networks object with a default of a single IPv4 network. Additionally, it adds the vpn capability to the requested parameters of the deployed vm runtime.

Note: The networks and capabilities objects will only be implicitly added if they are not already present in the descriptor. If the application specifies any of those objects, it is assumed that the application authors know what they're doing.

The manifest-support capability

Similarly, specifying the payload as vm/manifest implicitly adds manifest-support to the requested capabilities for the runtime.

Note: Again, this is only done if the payload.params doesn't already contain the capabilities object.

Usage

Currently, the dapp-runner implements a single CLI command, start:

Usage: dapp-runner start [OPTIONS] DESCRIPTORS...

which allows the following options:

  -d, --data PATH    Path to the data file.
  -l, --log PATH     Path to the log file.
  -s, --state PATH   Path to the state file.
  --stdout PATH      Redirect stdout to the specified file.
  --stderr PATH      Redirect stderr to the specified file.
  -c, --config PATH  Path to the file containing yagna-specific config.
                     [required]
  --silent
  --help             Show this message and exit.

The --data, --log, --state, --stdout, and --stderr arguments specify the locations of files to which the respective streams are written. If unspecified, all streams are written to the console which the dapp-runner is invoked from.

Streams

Data

The data stream consists of JSON-formatted output of specific components that are run as part of the services. Currently it carries the command execution events from exescript commands, e.g.:

{"db": {"0": [{"command": {"run": {"entry_point": "/bin/run_rqlite.sh", "args": [], "capture": {"stdout": {"stream": {}}, "stderr": {"stream": {}}}}}, "success": true, "stdout": null, "stderr": null}]}}

and the parameters of any started instances of Local HTTP proxies:

{"http": {"local_proxy_address": "http://localhost:8080"}}

The keys in the outermost dictionaries refer to names of service cluster as specified in the yaml descriptor file. For exescript commands, the secondary layer's keys refer to indices of instances within the specific cluster.

State

The state stream consists of JSON-formatted descriptions of the state of the dapp after each state change, e.g.:

{"db": {"0": "running"}, "http": {"0": "starting"}}

Here, again, the keys in the topmost dictionary refer to the names of service clusters defined in the yaml descriptor file and the secondary layer's keys refer to indices of specific instances.

Log

The log stream is a text stream of log messages emitted from dapp-runner.

Stdout / Stderr

Finally, stdout and stderr refer to the standard output streams of the dapp-runner script.

Config

This is a mandatory argument, specifying a path to a yaml file containing a description of a configuration to connect to your yagna daemon, e.g.:

yagna:
  app_key: "$YAGNA_APPKEY"
  subnet_tag: "devnet-beta"

payment:
  budget: 1.0  # GLM
  driver: "erc20"
  network: "rinkeby"

Descriptors

One or more application descriptors, as specified in the "Application descriptor" section above.

If more than one yaml descriptor file is given, all of the yaml files are merged into one descriptor before being processed further by the dapp-runner. The files are merged using a deep-merge strategy with contents of each subsequent yaml file overriding the colliding keys of the former ones.