The fun Linux kernel builder
Project description
The fun Linux kernel builder
Tuxbuild is a cloud-native highly concurrent Linux kernel build service.
[[TOC]]
Status: Early Access
Tuxbuild is currently under active development, but we want to hear from you! If you are interested in joining the waiting list, or have questions, feedback, or feature requests, please email us at tuxbuild@linaro.org.
Install and Configure
Install using pip
Tuxbuild requires Python version 3.6 or greater, and is available using pip.
To install tuxbuild on your system globally:
sudo pip3 install -U tuxbuild
To install tuxbuild to your home directory at ~/.local/bin:
pip3 install -U --user tuxbuild
To upgrade tuxbuild to the latest version, run the same command you ran to install it.
Install using docker
Tuxbuild is also available as a docker container at tuxbuild/tuxbuild.
For example, to run tuxbuild via docker:
docker run tuxbuild/tuxbuild tuxbuild build --help
Setup Config
The Authentication token needs to be stored in ~/.config/tuxbuild/config.ini
.
The minimal format of the ini file is given below:
$ cat ~/.config/tuxbuild/config.ini
[default]
token=vXXXXXXXYYYYYYYYYZZZZZZZZZZZZZZZZZZZg
Alternatively, the TUXBUILD_TOKEN
environment variable may be provided.
If you do not have a tuxbuild token, please reach out to us at tuxbuild@linaro.org.
Examples
tuxbuild build
Submit a build request using the tuxbuild command line interface. This will wait for the build to complete before returning by default.
tuxbuild build --git-repo 'https://github.com/torvalds/linux.git' --git-ref master --target-arch arm64 --kconfig defconfig --toolchain gcc-9
tuxbuild build-set
Create a tuxbuild config file with a basic set of build combinations defined.
cat <<EOF > basic.yaml
sets:
- name: basic
builds:
- {target_arch: arm64, toolchain: gcc-9, kconfig: defconfig}
- {target_arch: arm64, toolchain: gcc-9, kconfig: allmodconfig}
- {target_arch: arm64, toolchain: gcc-9, kconfig: allyesconfig}
- {target_arch: arm, toolchain: gcc-9, kconfig: allmodconfig}
- {target_arch: x86, toolchain: gcc-9, kconfig: allmodconfig}
- {target_arch: x86, toolchain: clang-9, kconfig: allmodconfig}
- {target_arch: x86, toolchain: gcc-9, kconfig: allyesconfig}
- {target_arch: i386, toolchain: gcc-9, kconfig: allmodconfig}
- {target_arch: riscv, toolchain: gcc-9, kconfig: allyesconfig}
EOF
# Build the build set defined in the config file named 'basic.yaml'
tuxbuild build-set --git-repo 'https://github.com/torvalds/linux.git' --git-ref master --tux-config basic.yaml --set-name basic
All the parameters can be specified in the build-set itself and invoke tuxbuild "tuxbuild build-set --tux-config .yaml --set-name "
Argument Reference
target_arch
target_arch supports arm64
, arm
, x86
, i386
, mips
, arc
, riscv
toolchain
toolchain supports gcc-8
, gcc-9
, clang-8
, clang-9
kconfig
The kconfig argument is a string or a list of strings that are used to define what kernel config to use.
The first argument must be a defconfig argumnet that ends in "config", such as "defconfig" or "allmodconfig".
Subsequent arguments may be specified to enable/disable individual config
options, an config fragment that exists in tree at kernel/configs/
, or a url
to an externally hosted config fragment.
All config options and fragments specified will be merged in the order that they are specified.
kconfig Examples
Simple defconfig build:
tuxbuild --kconfig defconfig ...
- yaml (string):
kconfig: defconfig
- yaml (list):
kconfig: [defconfig]
Enable or disable individual options:
tuxbuild --kconfig defconfig --kconfig "CONFIG_COMPILE_TEST=y" --kconfig "CONFIG_PROFILE_ALL_BRANCHES=n"
- yaml:
kconfig: [defconfig, "CONFIG_COMPILE_TEST=y", "CONFIG_PROFILE_ALL_BRANCHES=n"]
Using external fragment files:
tuxbuild --kconfig defconfig --kconfig "https://gist.githubusercontent.com/danrue/9e1e4d90149daadd5199256cc18a0499/raw/752138764ec039e4593185bfff888250a3d7692f/gistfile1.txt"
- yaml:
kconfig: [defconfig, "https://gist.githubusercontent.com/danrue/9e1e4d90149daadd5199256cc18a0499/raw/752138764ec039e4593185bfff888250a3d7692f/gistfile1.txt"]
Using in-tree fragment files. The file referenced needs to exist in
kernel/configs/
:
tuxbuild --kconfig defconfig --kconfig "kvm_guest.config"
- yaml:
kconfig: [defconfig, "kvm_guest.config"]
All of these options can be combined. They will be merged in the order they are specified:
tuxbuild --kconfig allnoconfig --kconfig "kvm_guest.config" --kconfig "https://gist.githubusercontent.com/danrue/9e1e4d90149daadd5199256cc18a0499/raw/752138764ec039e4593185bfff888250a3d7692f/gistfile1.txt" --kconfig CONFIG_COMPILE_TEST=y --kconfig "CONFIG_PROFILE_ALL_BRANCHES=n"
- yaml:
kconfig:
- allnoconfig
- kvm_guest.configs
- https://gist.githubusercontent.com/danrue/9e1e4d90149daadd5199256cc18a0499/raw/752138764ec039e4593185bfff888250a3d7692f/gistfile1.txt
- CONFIG_COMPILE_TEST=y
- CONFIG_PROFILE_ALL_BRANCHES=n
json-out
The --json-out <filename.json>
command-line option accepts a filesystem path,
where it will write a status file in json format at the end of a build or a
build-set. The file will contain, for example:
{
"build_key": "_YNU6WjSnKv_Akdajrnhyw",
"build_status": "pass",
"download_url": "https://builds.tuxbuild.com/_YNU6WjSnKv_Akdajrnhyw/",
"errors_count": 0,
"git_describe": "v5.6-rc6-9-gac309e7744be",
"git_sha": "ac309e7744bee222df6de0122facaf2d9706fa70",
"git_short_log": "ac309e7744be (\"Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/hid/hid\")",
"status_message": "build completed",
"tuxbuild_status": "complete",
"warnings_count": 0
}
The --json-out result of a build set will contain a list of entries.
kconfig_allconfig
The kconfig_allconfig argument is a string that is used to pass a filename to be used with allyesconfig/allmodconfig/allnoconfig/randconfig kconfig parameter.
The parameter can not be used with any other defconfig.
The argument is passed as an environment variable to the make command as "KCONFIG_ALLCONFIG=".
example:
tuxbuild --kconfig allmodconfig --kconfig-allconfig "arch/arm64/configs/defconfig"
API
The API examples below assume an environmental variable named TUXBUILD_TOKEN exists and contains a valid token. The following is not a valid token, but it is an example of what it looks like. If you do not have a valid token, please reach out to us at tuxbuild@linaro.org.
export TUXBUILD_TOKEN="gFkCz8wRHEA4BILyY4CtDfokKT5jPq1x413oGXvq3487ZAsZg9e_LJc4VUFrlahFRvp5kaKsnxZdnP7YdF4D-w"
Additionally, tuxbuild's default API endpoint is at
https://api.tuxbuild.com/v1
Submit build with curl
The /build API endpoint takes the same arguments as the tuxbuild cli, formatted as a list of json objects.
$ curl -X POST --header "Authorization: $TUXBUILD_TOKEN" \
--header "Content-Type: application/json" \
--data '[{"git_repo": "https://github.com/torvalds/linux.git", "git_ref": "master", "target_arch": "arm64", "kconfig": "defconfig", "toolchain": "gcc-9" }]' \
https://api.tuxbuild.com/v1/build
[{"git_repo": "https://github.com/torvalds/linux.git", "git_ref": "master", "target_arch": "arm64", "toolchain": "gcc-9", "kconfig": ["defconfig"], "kconfig_allconfig": null, "build_key": "PfHHehT4WzqbGCWFLOZ-Cg", "download_url": "https://builds.tuxbuild.com/PfHHehT4WzqbGCWFLOZ-Cg/"}]
The result of the POST is a json data structure describing the build. Printed with jq
, it looks like the following:
[
{
"git_repo": "https://github.com/torvalds/linux.git",
"git_ref": "master",
"target_arch": "arm64",
"toolchain": "gcc-9",
"kconfig": [
"defconfig"
],
"kconfig_allconfig": null,
"build_key": "4rolWYpvqW70LYGx9TIRcA",
"download_url": "https://builds.tuxbuild.com/4rolWYpvqW70LYGx9TIRcA/"
}
]
Use /status to monitor the status of the build.
Submit build set with curl
Using the same build endpoint, multiple builds can be requested. For example, build mainline with arm64 and arm.
$ curl -X POST --header "Authorization: $TUXBUILD_TOKEN" \
--header "Content-Type: application/json" \
--data '[{"git_repo": "https://github.com/torvalds/linux.git", "git_ref": "master", "target_arch": "arm64", "kconfig": "defconfig", "toolchain": "gcc-9" }, {"git_repo": "https://github.com/torvalds/linux.git", "git_ref": "master", "target_arch": "arm", "kconfig": "defconfig", "toolchain": "gcc-9" }]' \
https://api.tuxbuild.com/v1/build
Once again, it will return a json list.
[
{
"git_repo": "https://github.com/torvalds/linux.git",
"git_ref": "master",
"target_arch": "arm64",
"toolchain": "gcc-9",
"kconfig": [
"defconfig"
],
"kconfig_allconfig": null,
"build_key": "H8ITBrgWFjJ_NiriZFteEw",
"download_url": "https://builds.tuxbuild.com/H8ITBrgWFjJ_NiriZFteEw/"
},
{
"git_repo": "https://github.com/torvalds/linux.git",
"git_ref": "master",
"target_arch": "arm",
"toolchain": "gcc-9",
"kconfig": [
"defconfig"
],
"kconfig_allconfig": null,
"build_key": "yn1vuBZZF0nE2F-Vap7y-A",
"download_url": "https://builds.tuxbuild.com/yn1vuBZZF0nE2F-Vap7y-A/"
}
]
Use /status to monitor the status of the builds.
Check build status with curl
The status API endpoint will return the current status of a given build_key.
curl -s --header "Authorization: $TUXBUILD_TOKEN" --header "Content-Type: application/json" https://api.tuxbuild.com/v1/status/4rolWYpvqW70LYGx9TIRcA
It will return a json object such as the following:
{
"build_key": "4rolWYpvqW70LYGx9TIRcA",
"tuxbuild_status": "building",
"build_status": "building",
"git_sha": "d3dca69085e94e52a1d61a34b8e5f73a9f3d7eed",
"git_describe": "v5.6-rc5-270-gd3dca69085e9",
"git_short_log": "d3dca69085e9 (\"Merge branch 'i2c/for-current' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux\")",
"warnings_count": null,
"errors_count": null,
"download_url": "https://builds.tuxbuild.com/4rolWYpvqW70LYGx9TIRcA/",
"status_message": " "
}
The key field is "tuxbuild_status". It will show "queued", then "building", and finally "complete". The result of the build will be in "build_status", and may be "pass" or "fail".
Projects and Developers using tuxbuild
- LKFT - Linaro's Linux Kernel Functional Testing uses tuxbuild with gitlab-ci to continuously build upstream Linux kernels. The kernels are then functionally tested on a variety of hardware using LAVA.
Support
If you have any questions or concerns, please email them to tuxbuild@linaro.org. Please include the build ID with any build-specific questions.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.