Eclipse Otterdog

Introduction

Otterdog is a tool to manage GitHub organizations at scale using a configuration as code approach. It is actively developed by the Eclipse Foundation and used to manage its numerous projects hosted on GitHub.

Quickstart

To install and use the cli part of otterdog you have to install the following:

• git (mandatory): install using apt install git
• otterdog (mandatory): install using pipx install otterdog
• bitwarden cli tool (optional): install using snap install bw
• pass cli tool (optional): install using apt install pass

Otterdog Presentation @ Open Source Summit 2023

Default Configuration used @ Eclipse Foundation

Documentation

The documentation is available at otterdog.readthedocs.io.

Build instructions

System requirements:

• python3.11+ (mandatory): e.g. install using apt install python3 or use pyenv install 3.12
• git (mandatory): install using apt install git
• poetry >=2 (mandatory): install using pipx install poetry>=2.0.1
• bitwarden cli tool (optional): install using snap install bw
• pass cli tool (optional): install using apt install pass

Building Steps

• Create a virtual python environment and install necessary python dependencies using poetry:

$ make init

Running make init will also install poetry if it is not installed yet.

• Testing build

$ ./otterdog.sh -h

Quick Setup

To start using the cli part of otterdog right away on a specific organization you have to set up the following:

• define a default configuration to use, or use the following default config right away
• create a otterdog.json file that contains a list of GitHub organizations to manager and their respective credentials
• start managing your organizations using the cli

Default configuration

The example default config has all supported features enabled and can be used right away. However, it is advised to use a released tag instead of main to avoid incompatibilities.

Otterdog configuration

Create a otterdog.jsonnet or otterdog.json file with the following content (replace bracketed values according to your setup):

{
  "defaults": {
    "jsonnet": {
      "base_template": "https://github.com/eclipse-csi/otterdog#examples/template/otterdog-defaults.libsonnet@main",
      "config_dir": "orgs"
    }
  },
  "organizations": [
    {
      "name": "<project-name>",
      "github_id": "<github-id>",
      "credentials": {
        "provider": "plain",
        "api_token": "<GitHub PAT>",
        "username": "<Username>",
        "password": "<Password>",
        "twofa_seed": "<2FA TOTP seed>"
      }
    }
  ]
}

The name of the configuration file can be freely chosen (can be overridden with the -c flag). However, when named otterdog.jsonnet or otterdog.json, the cli tool will automatically detect and use that file if it is in the current working directory.

[!IMPORTANT] In this example the plain provider is being used to access credentials to avoid setting up a real credential provider (see below) for a quick setup. However, the plain provider should NOT be used for anything else to avoid leakage of data in case the otterdog.json file is shared with other users.

Environment Variables

OTTERDOG_CONFIG_ROOT

The OTTERDOG_CONFIG_ROOT environment variable allows you to specify a custom root directory for Otterdog configuration files and organization data. E.g: https://github.com/EclipseFdn/otterdog-configs

When set, Otterdog will:

• Search for configuration files (otterdog.jsonnet or otterdog.json) in this directory
• Use this directory as the base path for the config_dir (default: orgs/) containing organization configurations

Usage:

# Set the configuration root directory
export OTTERDOG_CONFIG_ROOT=/path/to/config

# Run otterdog commands - will automatically use the specified directory
otterdog fetch-config eclipse-csi
otterdog apply eclipse-csi

Credentials

Otterdog needs certain credentials to access information from an organization and its repositories on GitHub:

• username / password / 2FA seed
• API token

The login / username / 2FA seed are required to access the web interface of GitHub in order to retrieve certain settings that are not accessible via its rest / graphql API.

The GitHub api token needs to have the following scopes enabled:

• repo
• workflow
• admin:org
• admin:org_hook
• delete_repo

The credentials can be stored in different providers (bitwarden, pass).

Bitwarden

When using bitwarden to store the credentials, you need to enter a valid item id as additional credential data:

{
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "bitwarden",
        "item_id" : "<bitwarden item id>"
      }
    }
  ]
}

The item stored in bitwarden needs to contain the following information (a sample json output of such an item):

{
  "object": "item",
  "id": "<bitwarden item id>",
  "name": "<item name>",
  "fields": [
    {
      "name": "api_token_admin",
      "value": "<github API token>"
    }
  ],
  "login": {
    "username": "<github username>",
    "password": "<github password>",
    "totp": "<2FA TOTP seed>"
  }
}

Mandatory items:

• Field with name "api_token_admin" and as value the GitHub token to access the organization
• login.username of a user that can access the organization with enabled 2FA
• login.password the password of that user
• login.totp the 2FA TOTP seed

Pass

When using pass to store the credentials, you need to enter fully qualified pass names to access the various required credential data:

{
  "organizations": [
    {
      "name": "<org name>",
      "github_id": "<github org id>",
      "credentials": {
        "provider": "pass",
        "api_token": "<path/to/api_token>",
        "username": "<path/to/username>",
        "password": "<path/to/password>",
        "twofa_seed": "<path/to/2fa_seed>"
      }
    }
  ]
}

In case your password storage dir is not located at the default location, you can configurate that in the defaults:

{
  "defaults": {
    "pass": {
      "password_store_dir": "path/to/storage/dir"
    }
  }
}

As the password_store_dir might be different on different machines, you can also customize that in a separate .otterdog-defaults.json file:

{
  "pass": {
    "password_store_dir": "path/to/storage/dir"
  }
}