perfact-dbschematools 26.5.0

Tools for deploying necessary schema changes for a perfactema database

PerFact DBSchemaTools

Tools for deploying necessary schema changes for a perfactema database.

It aims to gradually replace DatabaseModifications by a schema definition delivered in the form of yaml files in a file system structure, so we don't define the steps to perform to arrive at some required schema, but the target schema itself. It also allows introspecting the current database and checking which (additive) change are to be performed to arrive at the correct schema.

The maintainers of this project are:

• Alexander Rolfes [e-mail]
• Ján Jockusch [e-mail]
• Viktor Dick [e-mail]

Technical details

The DBSchemaTools come bundled with an executable perfact-dbschema which is available in /usr/bin. This allows several actions to be performed (check out --help):

• patch: Compare the current DB against the definitions provided in the schema definition paths and extend the current DB as required.
• add: Adds an entity to a schema definition path
• dump: Dumps the full database into a schema definition path. Currently only for debugging.

Additionally, a wrapper perfact-dbschema-add is provided with a sudoers file that allows the user zope to execute it as user perfact on a typical PerFact system. This is intended to be used by the table manager while we transition more and more functionality over.

On a PerFact template system, the package perfact-dbutils-zope4 additionally provides a wrapper perfact-dbschema-patch. This is also used by the zodbsync configuration to make sure that any code deployment runs through the three steps:

1. Check the schema definitions as recognized by perfact-dbschematools and execute them, before changing anything inside Zope.
2. Play back DatabaseModifications and related paths, and trigger their application.
3. Play back any remaining code changes, i.e. the code that depends on the new schema.

A schema definition path is usually something like /opt/perfact/dbutils-zoperepo/__schema__ or a similar path somewhere in /var/lib/perfact/zodbsync/layers for an activated layer package. It contains a folder structure mostly oriented along tables, containing YAML files (and in the future probably also other file formats, when we include function definitions) that describe the entities that we require to be present in the database.

In preparation for a future situation where we can develop for multiple layers on the same system, the main executable perfact-dbschema requires the path to be explicitly named, making its arguments somewhat clumsy at times. Usually, you should not have to interact with it directly, relying on the table manager and on the wrapper perfact-dbschema-patch instead.

Usage from other packages

As of version 26.2.0, perfact-dbschematools no longer uses a virtual environment and can therefore be imported in other Python packages. A package that deploys some schema definitions that it requires to be executed to run can therefore do something like this at startup:

from perfact.dbschematools import patch
patch('/path/to/my/schema/definition', my_connection_string)

Configuration

The default configuration file /etc/pefact/dbschematools/config.yml currently only describes the database connection string.

FAQs

I tried deploying a code state that uses schema definitions and it failed

• Is perfact-dbutils-zope4 up to date? If you deployed using the layer package perfact-zope-layers-complete, the dependency should be set correctly so the necessary versions are installed, but if you only deployed using a git checkout, you will need to update the packages accordingly.
• Does the configuration of perfact-dbschematools contain the correct connection string?
• If you skipped the failures and continued the deployment, but fixed the cause, you can execute perfact-dbschema-patch to retry.
• Afterwards, you want to retry applying the DatabaseModifications.

During development, a column I created should have a different type after all

• Edit the file in __schema__/tables/<table>/columns/<column>.yml
• Execute perfact-dbschema-patch.
• If the column can not be converted automatically due to completely different types, you will need to first delete the column manually.