deb-constrictor 0.9

Build DPKGs natively with Python.

deb-constrictor

Build Debian Packages (.deb/DPKGs) natively in Python. No dependencies on Java, Ruby or other Debian packages.

Install

Using pip:

pip install deb-constrictor

Usage

Define directories, links, scripts and dependencies:

from constrictor import DPKGBuilder, BinaryControl

dirs = [
    {
        'source': '~/python/beneboyit/frontend/src',
        'destination': '/srv/python/bbit-web-frontend',
        'uname': 'www-data'
    }
]

maintainer_scripts = {
    'postinst': '~/python/beneboyit/frontend/scripts/after-install',
    'preinst': '~/python/beneboyit/frontend/scripts/before-install'
}

links =  [
    {
        'path': '/etc/nginx/sites-enabled/bbit-web-frontend',
        'target': '../sites-available/bbit-web-frontend'
    },
    {
        'path': '/etc/uwsgi/apps-enabled/bbit-web-frontend.ini',
        'target': '../apps-available/bbit-web-frontend.ini'
    },
]

depends = ('nginx', 'uwsgi')

output_directory = '~/build'

c = BinaryControl('bbit-web-frontend',  '1.5', 'all', 'Ben Shaw', 'BBIT Web Frontend')

c.set_control_field('Depends', depends)

c.set_control_fields({'Section': 'misc', 'Priority': 'optional'})

d = DPKGBuilder(output_directory, c, dirs, links, maintainer_scripts)
d.build_package()

Output file is named in the format <packagename><version><architecture>.deb and placed in the destination_dir. Alternatively, provide a name for your package as the output_name argument, and the package will be created with this name in the output_directory.

constrictor-build tool

constrictor-build is a command line tool that will build a package based on information in a JSON file. By default, this file is in the current directory and called "build-config.json".

It loads the following fields and expects them to be in the same format as above:

• package (string, required)
• version (string, required)
• architecture (string, required)
• maintainer (string, required)
• description (string, required)
• extra_control_fields (dictionary of standard DPKG control field pairs, optional)
• directories (array of dictionaries as per example above, optional)
• links (array of dictionaries as per example above, optional)
• configuration_files (array of strings, can be paths or globs to match multiple)
• maintainer_scripts (dictionary as per example above, optional)
• parent (string, optional, see parent section below)
• deb_constrictor (dictionary, optional, see deb_constrictor section below). Valid keys are:
  • ignore_paths (array of string, optional)
  • environment_variables (array of two-element arrays, optional)
  • variables (array of two-element arrays, optional)
  • commands (dictionary)

Examples of configuration files and how you might use constrictor-build in conjunction with other build steps are included in the examples directory.

Environment variables in the form ${var_name} or $var_name will be substituted.

parent

You can also provide a "parent" field, which is a path to another build JSON file (path is relative to the config file) from which to read config values. For example, you might want to define the sections only in a parent config rather than in each child config. The parent lookup is recursive, so a parent can have a parent, and so on. constrictor-build also attempts to load a base configuration file as the root of the configuration tree. The default location of this file is ~/constrictor-build-config.json, but can be overridden by setting the CONSTRICTOR_BUILD_BASE_CONFIG_PATH environment variable.

Child values will replace parent values. Fields that are lists or dictionaries will be appended to/updated as appropriate. Items in child configuration lists will not be added to the parent configuration if they already exists; this means that if a parent and child both define the same Depends, or directory to include (for example), they won't be included twice in the computer configuration,

For example, a parent with this configuration:

{
  "extra_control_fields": {
      "Depends": ["some-package"]
  }
}

Could be overridden with a child with this configuration:

{
  "extra_control_fields": {
      "Depends": ["some-other-package"],
      "Provides": ["this-package"]
  }
}

Creating a computed configuration like this:

{
  "extra_control_fields": {
      "Depends": ["some-package", "some-other-package"],
      "Provides": ["this-package"]
  }
}

deb_constrictor

Provides a dictionary of metadata to configure build options such as file exclusion, pre/post build actions or variables. Valid keys are:

• ignore_paths (array of strings, optional)
• environment_variables (array of two-element arrays, optional)
• variables (array of two-element arrays, optional)
• commands (dictionary of arrays, optional)

ignore_paths

List of glob patterns of files to exclude when assembling data tar. Files are compared with their name relative to the included directory, and have a leading slash.

For example, on the file system, you have directory layout like so:

• src
• src/media/
• src/media/123.jpg
• src/media/456.jpg

And your build-config.json specifies a directory with source src/. To exclude all the jpg files in the media directory, set you ignore_paths to this:

"ignore_paths": ["/media/*.jpg"]

In this case though, the media directory will be empty (as it only contained .jpg files) and so would not be included in the archive at all. This might not be desirable if you want an empty directory to be deployed.

The solution to this is to add a placeholder file in the directory that would otherwise be ignored - it should be called either .gitkeep or .depkeep. If this file is found its containing directory will be added to the archive (as it is not empty) however the placeholder file will not be included.

environment_variables, variables

An array of two-element arrays in the format [key, value]; this format is used instead of a dictionary to preserve order so that values may depend on values that have been defined earlier.

environment_variables and variables both behave in the same way in that any values they define can be used to interpolate variables throughout the configuration, however if calling external scripts (e.g. pre/post build scripts) then only environment_variables will be passed to the sub-process.

Here's an example using variables.

{
  "package": "${VENV_NAME}",
  "deb_constrictor": {
    "variables": [
      ["BUILD_DIR", "build"]
    ],
      "environment_variables": [
        ["PYTHON_VERSION", "3.6"],
        ["VENV_NAME", "example-virtualenv"],
        ["VENV_DIR", "${BUILD_DIR}/${VENV_NAME}"],
        ["VENV_BIN_DIR", "${VENV_DIR}/bin"]
      ]
  },
  "directories": [
    {
      "source": "${BUILD_DIR}/virtualenvs/${VENV_NAME}",
      "destination": "/var/virtualenvs/${VENV_NAME}"
    }
  ]
}

After the variables are interpolated the configuration will be like this:

{
  "package": "example-virtualenv",
  "directories": [
    {
      "source": "build/virtualenvs/example-virtualenv",
      "destination": "/var/virtualenvs/example-virtualenv"
    }
  ]
}

Variable resolution order is variables, then environment_variables, then os.environ, i.e. a key will first be looked up in variables, if it does not exist then environment_variables, and so on to os.environ.

The variables values will only be used to interpolate the configuration while the environment_variables values will be exported to any sub processes being called, so in this example, PYTHON_VERSION, VENV_NAME, VENV_DIR and VENV_BIN_DIR will be added to os.environ, while BUILD_DIR will not.

commands

Commands can be supplied to be run before and after building. For example, to setup a virtualenv for packaging, and to upload the built .deb to an apt repository afterwards.

The supported keys for the commands are prebuild and postbuild, which will be called before and after (respectively) the DPKG being created. The commands should be supplied as an array (as would be sent to subprocess.call).

When using parent config files, commands defined in children override those in parents (as opposed to appending).

Two special variables are set (in addition to other defined variables) which are interpolated into the commands and set in the environment:

• DEB_CONSTRICTOR_WORKING_DIR: the directory containing the current config file being used (e.g. for /foo/bar/build-config.json, this value is /foo/bar)
• DEB_CONSTRICTOR_OUTPUT_PATH: the output path of DPKG, relative to the cwd. This variable is only set for postbuild. This can be combined with the working dir path to get absolute path.

An example of using these:

{
  "deb_constrictor": {
    "commands": {
      "prebuild": ["build-virtualenv.sh"],
      "postbuild": ["scp", "${DEB_CONSTRICTOR_OUTPUT_PATH}", "[e-mail]/srv/apt/incoming/"]
    }
  }
}

The prebuild command build-virtualenv.sh has access to the DEB_CONSTRICTOR_WORKING_DIR environment variable (as well as other environment_variables that have been defined) and can refer to all of these to execute tasks.

This example shows that postbuild command that will be interpolated before being executed, so the actual command that is called might be:

scp build/example-1.0_amd64.deb [e-mail]/srv/apt/incoming/

remove_after_build

If True, then the DPKG will be deleted after building, or more specifically, after the postbuild command has been run (it will be deleted even if a postbuild command does not exist though).

This is intended to be used to clean up a DPKG that is no longer needed, for example, if the postbuild script sends it to a remote server.

Known Issues

• Can only make Binary packages
• As with any tar based archive, ownership of files based on uname/gname can be wrong if the user does not exist. Use with caution or create postinst scripts to fix.