Simple file-based migrations for clickhouse
Project description
Clickhouse Migrations
Python library for creating and applying migrations in ClickHouse database.
Development and Maintenance of large-scale db systems many times requires constant changes to the actual DB system. Holding off the scripts to migrate these will be painful.
Features:
- Supports multi statements - more than one query per migration file.
- Allow running migrations out-of-box
- Simple file migrations format: {VERSION}_{name}.sql
- Supports Cluster deployments, makes sure that migrations state is consistent on all cluster nodes
Known alternatives
This package originally forked from clickhouse-migrator.
Package | Differences |
---|---|
clickhouse-migrator | Doesn't support multistatement in a single file , to heavy because of pandas, looks like abandoned |
django-clickhouse | Need django |
clickhouse-migrate | Doesn't support multistatement |
Installation
You can install from pypi using pip install clickhouse-migrations
.
Usage
In command line
clickhouse-migrations --db-host localhost \
--db-user default \
--db-password secret \
--db-name test \
--migrations-dir ./migrations
In code
from clickhouse_migrations.clickhouse_cluster import ClickhouseCluster
cluster = ClickhouseCluster(db_host, db_user, db_password)
cluster.migrate(db_name, migrations_home, cluster_name=None,create_db_if_no_exists=True, multi_statement=True)
Parameter | Description | Default |
---|---|---|
db_host | Clickhouse database hostname | localhost |
db_port | Clickhouse database port | 9000 |
db_user | Clickhouse user | default |
db_password | Clichouse password | default |
db_name | Clickhouse database name | None |
migration_path | Path to list of migration files | <project_root> |
migrations | Explicit list of migrations to apply | [] |
cluster_name | Name of Clickhouse topology cluster from <remote_servers> | None |
create_db_if_no_exists | If the db_name is not present, enabling this will create the db |
True |
multi_statement | Allow multiple statements in migration files | True |
secure | Use secure connection | False |
fake | Marks the migrations as applied but without actually running the SQL to change your database schema | False |
Notes
The Clickhouse driver does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the multi_statement param. There are two important caveats:
- This mode splits the migration text into separately-executed statements by a semi-colon ;. Thus cannot be used when a statement in the migration contains a string with a semi-colon.
- The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for clickhouse_migrations-0.8.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | db854d6a39c2477c31da877fb095c7e9d63231ea83a0f5cbc546037960c44c57 |
|
MD5 | 439f4cb43353ee580abbdedafd805270 |
|
BLAKE2b-256 | 2745fd7970bb6e5ca4d4956c3fd2ba010c88c103e4a7e1c60a1861fcf61b9390 |
Hashes for clickhouse_migrations-0.8.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f6f2e3944c0a8222fb0b0a9f4558ac5016cd7faeb564ffa5e23de90e122da719 |
|
MD5 | 4bb2a361ff4da760ee809c7c1b20fadc |
|
BLAKE2b-256 | f1aa550446d89b76ed4ac5745d4265d856b6ae1b7110722e272a2fdf71c3bba6 |