Skip to main content
Help us improve Python packaging – donate today!

A utility for automating multi-host, multi-environment software builds and deployments.

Project Description

batou

batou is a multi-(component|host|environment|…) deployment utility.

Deployments are described in a model using “components” and can be deployed locally and remotely. Models are written as Python code.

See the batou documentation for details on how to use it.

Changelog

1.0b25 (2014-07-31)

  • Improve gpg handling: don’t do –no-tty or –batch any longer as we actually only run the gpg code in (local) terminal environments usually.

1.0b24 (2014-07-04)

  • Actually fix unzipping.
  • Remove environment option connect_method, and use autodetection instead: If we’re not the configured service_user after the plain ssh connect, run sudo.
  • Refactor attributes and overrides: attributes allow template expansion now and also converting default values. The Attribute class has moved to the batou component module.
  • Yield nonzero returncode on remote exceptions.
  • Add convenience converter for attributes “list” to handle comma separated lists of strings.
  • Force safety of Nagios service description field: prohibit “,” as its not safe for Nagios business processes.
  • Allow specifying custom rsync options for SyncDir: the ‘–inplace’ option is a good idea in most cases, but not all. Fixes #11846.
  • Make supervisor restart/reload more robust towards non-running supervisors with lying PID files. Fixes #12744.
  • Remove harmful decoding step in File verify that caused wrong results.

1.0b23 (2014-06-30)

  • Allow unzip to overwrite existing files if the archive has newer copies.

1.0b22 (2014-06-30)

  • Fix uncaught import-error on mysql module.

1.0b21 (2014-06-30)

  • Fix bug in bundle shipping that made batou depend on the locale (#13465).
  • Add MySQL components.
  • Introduce batou.utils.Attribute() to avoid imperative type conversions in configure().

1.0b20 (2014-05-13)

  • Start using python-requests for HTTP/S downloads to support authentication (and other goodies).
  • Improve error logging.

1.0b19 (2014-04-07)

  • Allow specifying alternate find-links in requirements.txt
  • Remove remnants of the buildout component’s extends option.

1.0b18 (2014-03-31)

  • Introduce environment option connect_method, either ‘ssh+sudo’ (default) or plain ‘ssh’.

  • Improve update code: existing installations would not update cleanly to a new version due to wrong use of pkg_resources API. Existing installations need to be cleaned manually. :/

  • Archive: recognize .tar.xz files and extract them with tar.

  • Fix bug in CMMI verify that would always raise UpdateNeeded due to a path mismatch (#13145).

  • Component.cmd() now performs expand() on the given command by default. Saves typing:

    self.cmd(‘rm -rf {{component.workdir}}’)

    instead of

    self.cmd(self.expand(‘rm -rf {{component.workdir}}’))

  • File component: apply character encoding when reading and writing files. (#13165)

  • Mercurial clone: changed update behaviour yet again: unless configured differently, always achieve target state, even by losing uncommitted changes.

  • Allow unnamed access to the last added/ored component using the ‘self._’ attribute:

    def configure(self):

    self += Download(…) self += Extract(self._.target)

1.0b17 (2013-12-06)

  • Mercurial clone: refuse to update if there are uncommitted changes or outgoing changesets.
  • Mercurial clone: added an option to avoid updating existing clones, which is often desirable during development.

1.0b16 (2013-11-29)

  • Improve the bootstrap/upgrade fix for #13049: the bootstrap template needed a second improvement.

1.0b15 (2013-11-29)

  • Try to fix #13049 - will only work once the dev version is already at least at b15. Not sure whether it really does the job, though. Hard to test.

  • Make environment and host arguments for running batou local optional. dev and localhost are the defaults now. Fixes #13034

  • Allow specifying secrets with [component:xy] in addition to [xy] to make it easier to discover when used to the environment overrides. Both options are supported now. Fixes #13003

  • Refactor Python virtualenv and package management. Provide explicit knowledge about Python versions 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, and 3.3 to use compatible virtualenv/pip/setuptools versions and commands. Fixes #12941

    NOTE: The API for batou.lib.python.Package has changed in backwards-incompatible ways:

    Package() needs to be added to a VirtualEnv component. Before:

    self += VirtualEnv(...)
    self += Package(...)
    

    Now:

    venv = VirtualEnv(...)
    self += venv
    venv += Package(...)
    

    Package parameter pip_install_options has been renamed to install_options.

  • Use urllib module instead of wget command for cross platform compatibility.

1.0b14 (2013-11-26)

  • Make Buildout component install zc.buildout without dependencies (which is just setuptools), since that can break the version pinning of setuptools.
  • Check whether packages required via a -e expression in the local requirements.txt are already installed.
  • #12946: Make full path to python executable configurable for virtualenv (and buildout) component.
  • Remove cleanup code that removes batou-0.2-style virtualenv directories (bin, lib, include etc.) from remote deployments.
  • Add mailto setting to crontab component.
  • Added some documentation.

1.0b13 (2013-11-19)

  • Introduce a plugin mechanism: if a local requirements.txt file is present, install its contents into the .batou virtualenv.
  • Load components in alphabetical order. This enables importing from other components (when naming the directories appropriately).
  • #12986: Only try to unbundle if there actually are changes to unbundle.
  • Raise an error if a cron job is configured without a timing value.
  • Add platform support for Debian (crontab, logrotate, supervisor).
  • Added some documentation.

1.0b12 (2013-11-04)

  • Added branch argument to mercurial.Clone. Setting a branch automatically updates to the branch head on deploy. This is mostly useful for development environments.
  • Create the ‘secrets’ directory if it doesn’t exist, yet. Also, disallow editing secret files for non-existing environments.
  • Support continuing remote bootstrapping if we failed after creating the initial remote directory but were unable to use Mercurial.
  • #12898: build.Configure component was broken when using the default prefix.

1.0b11 (2013-10-17)

  • #12897: Use non-SSL pypi mirror for downloading virtualenv to fix tests failing randomly on machines that (for some reason) can’t validate PyPI’s certificate.

  • #12911: Ensure that we can configure file owners when they don’t exist during configure phase yet.

  • #12912: Fix untested and broken file ownership management.

  • #12847: Clean up unicode handling for File and Content components and templating.

  • #12910: Remote deployments failed when using bundles for transfers if no changes needed bundling.

  • #12766: Allow bootstrapping a batou project in an existing directory to support migration from 0.2.

  • #12283: Recognize files as ‘is_template’ by default. Auto-detect source files in the definition directory if they have the same basename. This is what you want in 99% of all cases. Explicitly stating either the ‘content’ or ‘source’ parameter disables auto-detection.

    Now you can write this:

    File(‘foo’)

    and have components/x/foo recognized as the source file and handled as a template.

  • Use ConfigParser instead of configobj which is effectively unmaintained (see https://twitter.com/voidspace/status/380699699042213889) and support lists separated by newlines in addition to commas.

1.0b10 (2013-09-27)

  • Package our own virtualenv instead of depending on the system-installed one. This should alleviate troubles due to old virtualenv versions that package distribute, which causes conflicts with recent setuptools versions (#12874).
  • Update supervisor version to 3.0.

1.0b9 (2013-08-22)

  • Update Package component so it ignores installed packages when installing. This way, we actually install setuptools even when distribute is installed. (Otherwise it’s a no-op since distribute tells pip that setuptools is already satisfied).
  • Fix update process: wrong call to old ‘.batou/bin/batou’ failed and early bootstrapping would downgrade temporarily which is confusing and superfluous. Fixes #12739.

1.0b8 (2013-08-17)

  • Remove superfluous mkdir call during remote bootstrap.
  • Make batou init print that it’s working. Bootstrapping can take a while, so at least signal that something’s going on.

1.0b7 (2013-08-17)

  • Depend on Python2.7 to be available on the PATH during early bootstrap. Otherwise our chances to get a 2.7 virtualenv are pretty small, too.
  • Improve project template: ignore the work/ directory by default.

1.0b6 (2013-08-17)

  • More MANIFEST inclusions: bootstrap-template.

1.0b5 (2013-08-17)

  • Improve MANIFEST so we actually package the init template and other generated files, like version.txt and requirements.txt.

1.0b4 (2013-08-17)

  • Provide a simple project-creation command, both for pip-installed batou’s as well as spawning new projects from existing ones. Fixes #12730
  • Fix #12679: make timeouts configurable.
  • Removed re-imports from batou main module to support light-weight self-installation and bootstrapping. I.e. ‘from batou import Component’ no longer works.
  • Provide a single main command together with a ‘bootstrap’ wrapper that you can check into your project and that is maintained during updates automatically. It also provides fully automatic bootstrapping, installation, upgrading and other maintenance.
  • Fix Python package installation version check.
  • Don’t use bin/buildout bootstrap command anymore. PIP installs a sufficient bin/buildout so buildout can do the rest internally.
  • Install zc.buildout during bootstrapping phase using PIP to avoid bootstrap.py problems.
  • Shorten URLs in the Build component to their basename.
  • Add ‘assert_cmd’ API to support simpler assertions for verify when needing to check the result of an external command.
  • Switch to asking pip installing eggs instead of flat installations as namespaces seem to collide otherwise.
  • Remove non-functional deprecated ‘md5sum’ attribute.
  • Components are context managers now. If you provide __enter__ it will be called before verify() and if you provide __exit__ this will be called after update (always - even if update isn’t actually called). This allows you to manage temporary state on the target system more gracefully. See the DMGExtractor for an example.
  • Major refactoring of internal data structures to simplify and improve test coverage. Some breakage to be expected:
    • Components do not have a ‘service’ attribute any longer. If you need access to the service base directory use ‘environment.base_dir’.
  • Another attempt at getting convergency for package installation: actually leverage pkg_resources to check what version of a package is installed. PIP does not necessarily list or freeze setuptools, even if forced to reinstall.
  • Remove superfluous ‘–update’ from pip calls that would temporarily install the incorrect package version.
  • Improve convenience of the ‘secrets-edit’ wrapper script to allow re-encrypting without re-entering the editor.
  • Consistently switch to using setuptools.
  • Fix #12399: incorrect stat attributes for Owner and Group
  • Add exclude parameter to Directory component.
  • Add env parameter to Component.cmd() (and corresponding build_environment parameter to the Build component) to allow adding/overriding environment variables.

1.0b3 (2013-07-09)

  • Enable logging in the remote core to see what’s going on on the remote side.
  • Try to better format exceptions from the remote side.
  • Try harder to get virtualenv back into a working state.
  • Allow remote deployments from root of repository.
  • Make PIP management more robust.

1.0b2 (2013-07-09)

  • Add component to manage PIP within a virtual env.
  • Add component to manage packages with PIP within a virtual env.
  • Restructure buildout component to make it more robust regarding setuptools/distribute preparation. Also remove usage of bootstrap completely as we rely on virtualenv anyway.

1.0b1 (2013-07-09)

  • Apply semantic versioning: initial development is over, so this is 1.0 now.
  • Major revamp of secrets management:
    • switch to GPG (instead of aespipe)
    • turn secrets into a core feature, removing the need for a special component
  • Add ‘–single’ to suppress parallel bootstrapping.

Release history Release notifications

History Node

1.7.5

History Node

1.7.4

History Node

1.7.3

History Node

1.7.2

History Node

1.7.1

History Node

1.7.0

History Node

1.6.1

History Node

1.6.0

History Node

1.6.0a3

History Node

1.6.0a2

History Node

1.6.0a1

History Node

1.5.0

History Node

1.5.0a6

History Node

1.5.0a5

History Node

1.5.0a4

History Node

1.5.0a3

History Node

1.5.0a2

History Node

1.5.0a1

History Node

1.4.0

History Node

1.4.0b7

History Node

1.4.0b6

History Node

1.4.0b5

History Node

1.4.0b4

History Node

1.4.0b3

History Node

1.4.0b2

History Node

1.4.0b1

History Node

1.3.5

History Node

1.3.4

History Node

1.3.3

History Node

1.3.2

History Node

1.3.1

History Node

1.3.0

History Node

1.3b3

History Node

1.3b2

History Node

1.3b1

History Node

1.2

History Node

1.2b2

History Node

1.2b1

History Node

1.1.2

History Node

1.1.1

History Node

1.1

History Node

1.1.0a12

History Node

1.1.0a11

History Node

1.1.0a10

History Node

1.1.0a9

History Node

1.1.0a8

History Node

1.1.0a7

History Node

1.1.0a6

History Node

1.1.0a5

History Node

1.1.0a4

History Node

1.1.0a3

History Node

1.1.0a2

History Node

1.1.0a1

History Node

1.0.10

History Node

1.0.9

History Node

1.0.8

History Node

1.0.7

History Node

1.0.6

History Node

1.0.5

History Node

1.0.4

History Node

1.0.3

History Node

1.0.2

History Node

1.0.1

History Node

1.0

History Node

1.0b33

History Node

1.0b32

History Node

1.0b31

History Node

1.0b30

History Node

1.0b29

History Node

1.0b28

History Node

1.0b27

History Node

1.0b26

This version
History Node

1.0b25

History Node

1.0b24

History Node

1.0b23

History Node

1.0b22

History Node

1.0b21

History Node

1.0b20

History Node

1.0b19

History Node

1.0b18

History Node

1.0b17

History Node

1.0b16

History Node

1.0b15

History Node

1.0b14

History Node

1.0b13

History Node

1.0b12

History Node

1.0b11

History Node

1.0b10

History Node

1.0b9

History Node

1.0b8

History Node

1.0b7

History Node

1.0b6

History Node

1.0b5

History Node

1.0b4.dev0

History Node

1.0b3

History Node

1.0b2

History Node

1.0b1

History Node

0.2.15

History Node

0.2.14

History Node

0.2.13

History Node

0.2.12

History Node

0.2.11

History Node

0.2.10

History Node

0.2.9

History Node

0.2.8

History Node

0.2.7

History Node

0.2.6

History Node

0.2.5

History Node

0.2.4

History Node

0.2.3

History Node

0.2.2

History Node

0.2.1

History Node

0.2

History Node

0.1.6

History Node

0.1.5

History Node

0.1.4

History Node

0.1.3

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
batou-1.0b25.zip (112.7 kB) Copy SHA256 hash SHA256 Source None Jul 31, 2014

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page