Distribution compose tool
A tool to gather build artifacts and assemble them into a set of repositories.
rhcephcompose is a distribution compose tool, similar to Red Hat’s Pungi (open-source). In contrast to simply throwing all builds together into a single package repository, these tools give the user fine-grained control over the selection of builds and the layout of the final product’s output.
Composes are release snapshots that contain release deliverables such as installation trees with RPMs and Yum repodata. rhcephcompose creates an installation tree for Ubuntu packages. In Red Hat we use it for developing and shipping the RH Ceph Enterprise product for Ubuntu.
In the RHEL world, el6 and el7 repositories are typically separated into two entirely different trees in the filesystem. In Debian, a repository can mix several distribution versions together.
Chacra has an API for distro_version. Since we combine both Trusty and Xenial for some packages, this introduces complexity. So I store all binaries in “distro_version” of “all”, and then use the XML and builds to determine what should be tagged with what.
In a Brew world, we would be able to tag one build with multiple -candidate tags, but in Chacra there is only a one-to-one relationship between builds and distro_versions. So with Brew, we’d be able to tag “ceph-deploy-1.2.3” as both “ceph-1.3-trusty-candidate” and “ceph-1.3-xenial-candidate”, but in Chacra I can’t easily do that for individual builds.
In dist-git for Ubuntu, I store the branches as “-ubuntu” in order to combine the codebase for “-trusty” and “-xenial”. The reason for this is that I always ended up keeping “ceph-1.2-rhel-6” and “ceph-1.2-rhel-7” identical, and it was a pain to do that manually. As described above, I also keep all the builds “combined” within Chacra.
To save time when accessing chacra, rhcephcompose downloads all build artifacts to a local cache by default. This cache location is $XDG_CACHE_HOME/rhcephcompose/. If the XDG_CACHE_HOME environment variable is unset, rhcephcompose defaults this to ~/.cache (so builds are written to ~/.cache/rhcephcompose).
rhcephcompose never evicts items from this cache so it can grow without bound. It’s a good idea to clean it out occasionally. If you are running rhcephcompose with Jenkins, you can do this automatically by setting $XDG_CACHE_HOME to a location within the job’s workspace, and then have Jenkins simply clean up the workspace.
This is more of a python-requests thing, but if you receive an SSL warning, it’s probably because you don’t have the Red Hat IT CA set up for your Python environment. Particularly if you’re running this in a virtualenv, you’ll want to set the following configuration variable:
Where “RH-IT-Root-CA.crt” is the public cert that signed the Chacra server’s HTTPS certificate.