Buildbot-Phorge integration

Buildbot is a wonderful automation framework, Phorge is a community fork of Phabricator, a wonderful review and CVS hosting tool.

Sadly they don't talk too well to eachother. This aims to fix that.

Setting up Buildbot

Install this package from pypi into the same environment that your buildbot-master runs on.

Change Hook

This component is required in order to tell Buildbot about a new changeset.

Similarly to other Buildbot Change Hooks, we configure it in c["www"].

To set up the hook, we need the Phorge host, a secret to authenticate such host against Buildbot and a Conduit API token to authenticate Buildbot against Phorge in order to fetch additional data.

Example

# We use this to fetch additional data form Phorge's Conduit API
phorge_host : str = "we.phorge.it"
# Phorge uses HTTP Basic Authentication, the value of phorge_host will be
# username and the value of bb_token (may be a secret) will be the password
# This has to be set up in Phorge
bb_token : Union[str, Secret] = util.Secret("we.phorge.it.bb_token")
# Phorge doesn't send us all necessary data for the build, so we have to fetch
# it using the Conduit API, for which we need this token. It may be a secret
conduit_api_token: Union[str, Secret] = util.Secret("we.phorge.it.api_token")

# And the actual bits, when setting up the 'www' plugin, change_hook_dialects
# must be set up in a similar fashion to this.
c["www"] = dict(
	[...]
	change_hook_dialects=dict(phorge={
		"credentials": {
			phorge_host: (bb_token, conduit_api_token)
		}),
	[...]
)

Reporter

In order to notify Phorge about the result of builds, we need a Reporter.

Currently due to Conduit API limitations, no information other than PASS and FAILED can be sent.

Example

# Import the necessary class
from buildbot_phorge import PhorgeReporter

# Here we only need phorge_host and conduit_api_token, as for the hook
# Remember conduit_api_token may be a Secret.
c["services"].append(PhorgeReporter(token=conduit_api_token, host=phorge_host))

Setting up Phorge

In Phorge you have to setup a Build Plan for Harbormaster. Depending on your setup, you may only need one Build Step that triggers buildbot, as Buildbot is much better at automation.

This is Build Step of type Make HTTP Request.

As far as we can see, there is no option to pass arguments over POST requests, so we pick HTTP Method GET.

Add some secure Credentials which will be checked on Buildbot.

Remember that the User must match your Phorge hostname!

Setup When Complete as Wait For Message, Buildbot should tell Phorge about the result of the build afterwards.

The tricky bit is the URI setting, we have to pass all the information that Buildbot will need to trigger the build and let Phorge know about its result.

The URI should look like this (change ${CI_HOST} for your Buildbot hostname)

# Set this Build Step URI on Phorge
https://${CI_HOST}/change_hook/phorge?id=${build.id}&t=${target.phid}&ts=${step.timestamp}&u=${repository.uri}

Future work

This can still be improved but it works wonderfully as-is for us.

Still, there are some things that you can hack / improve.

Use the ${buildable.diff} to trigger builds on Reviews / Diffs

Use the ${buildable.revision} to trigger builds on Revisions / Audits