Migration tool for Odoo
Project description
🐜🐜🐜 Marabunta 🐜🐜🐜
Marabunta is a name given to the migration of the legionary ants or to the ants themselves. Restless, they eat and digest everything in their way.
Marabunta is used to provide an easy way to create Updates for Odoo fast and run easily. It also allows to differentiate between different environment to provide for instance demodata.
Usage
After installing marabunta, it will be available as a console command. To run properly it requires a migration file (which defines what has to updated/executed) and odoos connection parameters (view options in the options section.
At each run marabunta verifies the versions from the migration file and and processes new ones. It is very much recommended to configure it, so that marabunta is ran automatically if odoo is started. For instance adding it to your docker entrypoint.
Features
backup: Marabunta allows for a backup command to be executed before the migration.
addon upgrades: Marabunta is able to install or upgrade odoo addons.
operations: Allows to execute commands before or after upgrading modules.
modes: Modes allow the user to execute commands only on a certain environment. e.g. creation of demodata on a dev system.
maintenance page: publish an html page during the migration.
Versioning systems
Currently Marabunta allows for two different Versioning systems: The classic Major.Minor.Bugfix and the Five digits long versions for OdooMajor.OdooMinor.Major.Minor.Bugfix. Although the first marabunta version must be setup for the initial setup of your instance. (Find out more about the rationale here <https://github.com/camptocamp/marabunta/commit/9b96acaff8e7eecbf82ff592b7bb927b4cd82f02>)
Options
option
shortcut
envvar
purpose
–migration-file
-f
MARABUNTA_MIGRATION_FILE
Definition file for the migration.
–database
-d
MARABUNTA_DATABASE
Database we want to run the migration on.
–db-user
-u
MARABUNTA_DB_USER
Database user.
–db-password
-w
MARABUNTA_DB_PASSWORD
Database password.
–db-port
-p
MARABUNTA_DB_PORT
Database port (defaults to 5432).
–db-host
-H
MARABUNTA_DB_HOST
Database port (defaults to None).
–mode
MARABUNTA_MODE
Mode marabunta runs in for different envs.
–allow-serie
MARABUNTA_ALLOW_SERIE
Allow multiple versions to be upgraded at once.
–force-version
MARABUNTA_FORCE_VERSION
Force the upgrade to a version no matter what.
–web-host
MARABUNTA_WEB_HOST
Interface to bind for the maintenance page. (defaults to 0.0.0.0).
–web-port
MARABUNTA_WEB_PORT
Port for the maintenance page. (defaults to 8069).
–web-custom-html
MARABUNTA_WEB_CUSTOM_HTML
Path to custom maintenance html page to serve.
YAML layout & Example
Here is an Example migration file:
migration: options: # This includes general options which are used everytime marabunta is called. # --workers=0 --stop-after-init are automatically added install_command: odoo #Command which starts odoo install_args: --log-level=debug # additional Arguments backup: # Defines how the backup should be done before the migration. command: echo "backup command on ${DB_NAME}" stop_on_failure: true ignore_if: test "${RUNNING_ENV}" != "prod" versions: - version: setup # Setup is always the initia. version< operations: pre: # executed before 'addons' - echo 'pre-operation' post: # executed after 'addons' - anthem songs::install addons: upgrade: # executed as odoo --stop-after-init -i/-u ... - base - document modes: prod: operations: pre: - echo 'pre-operation executed only when the mode is prod' post: - anthem songs::load_production_data demo: operations: post: - anthem songs::load_demo_data addons: upgrade: - demo_addon - version: 0.0.2 backup: false # nothing to do this can be used to keep marabunta and gittag in sync - version: 0.0.3 operations: pre: # we also can execute os commands - echo 'foobar' - ls - bin/script_test.sh post: - echo 'post-op' - version: 0.0.4 backup: false addons: upgrade: - popeye
Run the tests
To run marabunta tests, it is a good idea to do an editable install of it in a virtualenv, and then intall and run pytest as follows:
$ git clone https://github.com/camptocamp/marabunta.git Cloning into 'marabunta'... $ cd marabunta $ virtualenv -p YOUR_PYTHON env $ source env/bin/activate $ pip install '.[test]' $ py.test tests
Release History
Unreleased
Features
Bugfixes
Improvements
Build
0.12.0 (2024-01-09)
Improvements
allow connecting to a local database server without password (using ident)
0.11.0 (2023-09-12)
Bugfixes
Fix the build and release workflow
Build
Deprecate python 2.7, 3.5 and 3.6
Support python 3.9, 3.10 and 3.11
0.10.7 (2022-05-24)
Bugfixes
Fix parsing of command line arguments (and related environment variables): --db-port, --web-port, --web-resp-status and --web-resp-retry-after
Improvements
Switch to Psycopg2 binary wheel
Build
Test for python 3.6, 3.7 and 3.8
0.10.6 (2021-09-14)
Improvements
Web server returns status 503 instead of 200
New option ‘web-healthcheck-path’
0.10.5 (2020-12-08)
Bugfixes
Fix backup operation if force_version is set
Improvements
Prevent text from bouncing in maintenance page
raise an exception if there is duplicate keys in migration file
0.10.4 (2019-02-15)
Bugfixes
Fix BoolEnvDefault when envvar is not defined
0.10.3 (2019-02-13)
Bugfixes
ALLOW_SERIES shouldn’t be true if a false value is given.
0.10.2 (2018-12-12)
Bugfixes
Crash when forcing upgrade of a version and no backup command is configured
0.10.1 (2018-11-09)
Build
The lib is now automaticaly published to Pypi by Travis when a tag is added
0.10.0 (2018-11-06)
Backward incompatible change
In the migration yaml file, the command and command_args options are now all merged into command
Features
Backup command and backup’s ignore_if are now run in a ‘sh’ shell so we can inject environment variables in the commands
Backup command can now use $database, $db_host, $db_port, $db_user, $db_password that will be substituted by the current configuration values
Bugfixes
When starting 2 concurrent marabunta process and the first fail, it releases the lock and the second would start odoo without actually run the migration. Now, when the migration lock is released, the other process(es) will recheck all versions as well before running odoo.
Documentation
Add some high-level documentation
0.9.0 (2018-09-04)
Features
1st version it’s always “setup”
In all projects’ usecases the 1st version is always to setup the initial state. Adopting setup as the 1st version makes also easier to squash intermediate version all in one as the project and its releases grow up.
Support 5 digits version
Now you can use 5 digits versions as per odoo modules. For instance: 11.0.3.0.1. This give us better numbering for patches and makes versionig withing Odoo world more consistent. Old 3 digits versions are still supported for backward compat.
Full rationale for above changes available here
https://github.com/camptocamp/marabunta/commit/9b96acaff8e7eecbf82ff592b7bb927b4cd82f02
Backup option
Migration allows using a backup command in order to perform specific commands (unless explicitly opted-out) before the migration step.
No backup machinery provided as you are suppose to run your own command to execute the backup.
Bugfixes
Build Py3 wheel on release
0.8.0 (2017-11-16)
Python3!
0.7.3 (2017-11-01)
Bugfixes
Support special chars (such as +) in Postgres passwords. The passwords were incorrectly passed through unquote_plus, which transform the + char to a space.
0.7.2 (2017-09-15)
Bugfixes
Use –no-xmlrpc option when running odoo as the new web server use the same port, it’s not needed anyway
0.7.1 (2017-09-11)
Bugfixes
Include maintenance html file in the distribution
0.7.0 (2017-09-08)
Features
Publish a maintenance web page during migration. The host and port are configurable with new options. By default the port match odoo’s (8069). A default maintenance is provided, but it can be configured as well.
When a migration fails, the log alongside the traceback are logged in the marabunta_version table.
Bugfixes
Commands with unicode chars make the migration fail
Build
Removed python3 from tox, it doesn’t run on py3 and we can’t make them run now. Odoo is still python2, py3 compat will come when it’ll switch.
0.6.3 (2016-12-12)
Bugfixes
The new connection opened in 0.6.2 might suffer from the same issue of timeout than before 0.6.0: the connection is long-lived but there is no keep-alive for this connection. Open a new connection for each update in marabunta_version, which might be spaced between long subprocess operations
0.6.2 (2016-12-12)
Bugfixes
Autocommit the operations done in the marabunta_version table. Previously, after an exception, the changes to marabunta_version were rollbacked, which is not the expected behavior (it makes the migration restart ceaseless). As a side effect, Marabunta now opens 2 connections. The connection opened for the adsivory lock cannot commit before the end because it would release the lock.
0.6.1 (2016-11-25)
Important bugfix! The changes in the marabunta_version were never committed, so migration would run again.
Bugfixes
Commit the connection so changes are not rollbacked.
0.6.0 (2016-11-21)
Improvements
Rework of the database connections:
The advisory lock is opened in a cursor in a thread, this cursor periodically executes a dummy ‘SELECT 1’ to be sure that the connection stay alive (not killed with a timeout) when a long-running subprocess is run.
The operations in database are executed in short-lived cursors. This prevents an issue we had when the open cursor was locking ‘ir_module_module’, preventing odoo to install/update properly.
Try to disable colors in output if the term does not support colors
0.5.1 (2016-10-26)
Fix: marabunta processes run concurrently all tried to run the migration, this is better handled with a PostgreSQL advisory lock now
0.5.0 (2016-10-12)
Odoo 10 Support
Features
Switch the default command line for running odoo to odoo instead of odoo.py (renamed in Odoo 10). For usage with previous version, you must specify the install_command in the migration.yml file.
0.4.2 (2016-08-17)
Bugfixes
Prevent error (25, ‘Inappropriate ioctl for device’) when stdout is not a tty by disabling the interactive mode.
0.4.1 (2016-07-27)
Bugfixes
Do not print on stdout the result of operations twice
0.4.0 (2016-07-26)
Improvements
New dependency on pexpect. Used to create a pseudo-tty to execute the operations. It enables line buffering and interactivity for pdb in the children processes.
Fixes
Noop operations are really considered as such
0.3.3 (2016-07-12)
Fixes
Encode print’s outputs to the stdout’s encoding or to utf8 by default
0.3.2 (2016-07-08)
Fixes
Failure when there are no version to process
0.3.1 (2016-07-07)
Fixes
Fix decoding issues with output of subprocesses
0.3.0 (2016-07-06)
Introducing modes.
Backward incompatible changes
--demo is replaced by a more general --mode argument, the equivalent being --mode=demo
MARABUNTA_DEMO is replaced by MARABUNTA_MODE
the configuration file has now operations and addons by “modes”, allowing to load some different scripts or install different addons for different modes (the addons list are merged and the operations of the modes are executed after the main ones):
- version: 0.0.1 operations: pre: # executed before 'addons' - echo 'pre-operation' post: # executed after 'addons' - anthem songs::install addons: upgrade: - base modes: prod: operations: pre: - echo 'pre-operation executed only when the mode is prod' post: - anthem songs::load_production_data demo: operations: post: - anthem songs::load_demo_data addons: upgrade: - demo_addon
--force renamed to --allow-serie
MARABUNTA_FORCE renamed to MARABUNTA_ALLOW_SERIE
--project-file renamed to --migration-file
MARABUNTA_PROJECT_FILE renamed to MARABUNTA_MIGRATION_FILE
Improvements
When ‘allow_serie’ is used, the same Odoo addon will not be upgraded more than one time when it is in the ‘upgrade’ section of more than one version
Fixes
Fix error when there is no db version in the database
Fix error AttributeError: 'bool' object has no attribute 'number' when there is an unfinished version
Fix error when the db version is above the unprocessed version
0.2.2 (2016-06-23)
Improvements
Adapted the README so that it is rendered as ReST on pypi.
0.2.1 (2016-06-23)
Bugfixes
Fixed the version information of the package and release date.
0.2.0 (2016-06-23)
Features
Added support for Python 3.4 and 3.5 in addition to 2.7.
Bugfixes
Fixed a crash with empty install args
Improvements
Use YAML safe_load for added security.
Documentation
Bootstrapped the Sphinx documentation.
Build
Switched to tox for the build. This allow to run the same tests in all environment locally like in travis. The travis configuration just calls tox now.
Added runtime dependencies to the package, kept separate from the build and test dependencies (installed separately by tox).
0.1.1 (2016-06-08)
Fixed problems with packaging so that now marabunta can be installable from pypi.
0.1.0 (2016-06-08)
Initial release. This corresponds to the initial work of Guewen Baconnier.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for marabunta-0.12.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84fa252e2869a2e2c31eed7e88e20fa60ffdc51a8a008591300b02bda93d431a |
|
MD5 | 0135e8ec3ec6cae1459d59b6fd98dac6 |
|
BLAKE2b-256 | 27ca7d296357798fb655add4a29913b5c34bd483857003e99624fbd40b31454a |