Golem VM Image builder.
Golem VM Image builder used as companion app for Golem Registry: https://registry.golem.network
Running docker engine is required. Tool supports Linux, Windows and macOS. Note that when using macOS ARM version use --platform linux/amd64 option for docker builds.
You can install gvmkit-build using pip (python3 with pip installer is required)
pip install gvmkit-build
or install form npm (npm installation is required)
npm install -g gvmkit-build
or install using cargo (Rust toolchain is required, a bit slow because it compiles from sources)
cargo install gvmkit-build
or download prebuild from github releases page:
or build from sources, you can find binary in target/release/gvmkit-build (or gvmkit-build.exe on Windows)
cargo build --release
Golem Network is using gvmi images as base for creating VMs for tasks. These images are basically squashfs images with some additional metadata. They can be prepared from docker images using this (gvmkit-build) tool.
Make sure your docker service is running and you have gvmkit-build installed
Go to the folder with your dockerfile and run
docker build . -t my_image
- Create account on registry portal https://registry.golem.network
Let's assume your user name is golem
- Create repository on registry portal
Let's assume you created repository named my_example
Create and copy personal access token from registry portal
Run (you will be asked for login and personal access token)
gvmkit-build my_image --push-to golem/my_example:latest
- Your tag
golem/my_example:latestis ready to use in one of Golem Network APIs
The tool as main argument takes docker image name.
Docker image name can be resolved using ImageId + tag or Repository name + tag.
Repository name can be composed of maximum two parts:
python resolves to
python:3.8 resolves to
golemfactory/blender resolves to
You can also use image id instead of name, use
docker image ls to find your image id.
Following command will build image and create *.gvmi file in current directory.
If image not exist locally the tool is trying to pull it from docker hub.
To use it in Golem Network, you have to upload it to registry portal.
To successfully add image to registry portal you have to name image accordingly or use
gvmkit-build <user_name>/<image_name>:<tag> --push
or if your local image name is not compatible use
gvmkit-build <docker_image_id> --push-to <user_name>/<image_name>:<tag>
Build process explained a bit
Tool is creating new container and is copying data from given image to new container. After copying is finished mksquashfs command is used inside tool container to create squashfs image. After adding metadata *.gvmi file is created and tool container removed. Note that tool image will stay downloaded on your machine for future use (it is quite small so no worry about disk space)
[Optional] - building squashfs-tools image
If you want to use your own tool without pulling from dockerhub: Go to squashfs-tools directory and run
docker build . -t my_squash_fs_builder
add environment variable to .env in folder where you run gvmkit-build
For managing docker images and containers bollard library is used. https://docs.rs/bollard/latest/bollard/
Troubleshooting login to registry portal
The tool is using https://registry.golem.network as default registry portal.
You can change this behaviour by setting
REGISTRY_URL environment variable.
You should create account on registry portal and generate access token for your user.
If you really don't want to create account you can use anonymous upload (see section below about anonymous upload).
The tool will ask for login when --login --push or --push-to option is specified.
For storing login information rpassword library is used: https://docs.rs/crate/rpassword/latest
Only one instance of login/token is kept saved at "gvmkit-build-rs/default". So if you login with new user/token old pair will be forgotten.
This option will ask for login to registry portal
You can use command for check if your login information is correct
If you want to forget your login information you can use, it will clear your login information
Above command are optional, because you will be asked for login automatically if you use --push or --push-to option.
On some systems when no secure store is provided you will be forced to use following method of keeping login/token pair. You can optionally put them in .env file from convenience in your working directory. Remember to set proper permissions to the file (chmod 600 .env) if using on shared machine.
When REGISTRY_USER/REGISTRY_TOKEN is set in environment variables, secure rpassword storage won't be used.
Uploading to multiple tags
The tool cannot upload to multiple tags, but you can call it multiple times with different tags. All steps of operations are cached so no worry about re-uploading same file multiple times.
Uploading large files
The tool is using chunked upload with default chunk size 10MiB for images greater than 500MiB (changing not recommended). Four upload workers are created by default (you can increase/decrease number of workers using --upload-workers argument depending on your network conditions). If you think your upload is stuck you can always stop and run tool again to finish download. Only chunks that were not uploaded will be uploaded again.
Note: Total limit of chunks is set to 1000 (so around 10GB by default). If you want to upload larger file you have to set greater chunk size accordingly.
Uploading image without login
You can upload images anonymously. Note that lifetime of such images is limited, and they can be removed from registry portal after some time without notice
cargo run --release -- <image_name> --push --nologin
Uploading image without building part
If you are sure that you have proper *.gvmi file for example my-test.gvmi you can use
gvmkit-build --direct-file-upload my-test.gvmi --push-to <user_name>/<image_name>:<tag>
gvmkit-build --direct-file-upload my-test.gvmi --push --nologin
Changing squashfs options when creating image
You can change compression used in mksquashfs to produce more or less compact images.
Look for help for more information. Note that currently zstd is not supported by Golem Network (you can use xz instead for extra compact images).
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for gvmkit_build-0.3.18-py3-none-win_amd64.whl
Hashes for gvmkit_build-0.3.18-py3-none-musllinux_1_2_x86_64.whl
Hashes for gvmkit_build-0.3.18-py3-none-musllinux_1_2_aarch64.whl
Hashes for gvmkit_build-0.3.18-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Hashes for gvmkit_build-0.3.18-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Hashes for gvmkit_build-0.3.18-py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl
Hashes for gvmkit_build-0.3.18-py3-none-macosx_11_0_arm64.whl
Hashes for gvmkit_build-0.3.18-py3-none-macosx_10_12_x86_64.whl