Skip to main content

Migration library for Clickhouse

Project description

ClickHouse Migrate

codecov PyPI version

Python library for applying migrations in ClickHouse database.

Installation

Installation via PyPi:

pip install clickhouse-migrate

Configuration

.ini file

To configure and use clickhouse-migrate we can create a *.ini configuration file with the following content

[databases]
# Here we should provide a connection string to a database
# In case of using a replicated ClickHouse instance one should provide all connection strings
# related to this particular replicated instance
connection_strings = [
        "clickhouse+native://default:@localhost:9000/clickhouse_migrate"
    ]

[migration_path]
# Here we should provide a path to the selected migration folder
directory = "./migrations"

Default name that is used by clickhouse-migrate library is clickhouse_migrate.ini Configurator will search for this file by default.

Environment variables

Another configuration option is to use environment variables

CLICKHOUSE_MIGRATE_DATABASES='["<connection_string_1>", ... ,"<connection_string_n>"]'
CLICKHOUSE_MIGRATE_DIRECTORY=<path to directory with migrations files>

Command line arguments

If the above mentioned methods is also not applicable for the project, one can use command line arguments -db / --databases — Databases list -dir / --migration_dir — Migrations directory

Usage

Create migration file

One can create a new migration via calling clickhouse-migrate create_migration command.

Command parameters:

-n / --name <name of a new migration file> — this is a required parameter

-c / --config <path to the created config file> — this is an optional parameter, one may want to use it when the config file is not located in the root directory or if its' name is different from default one

-db / --databases — optional parameter for providing database connection strings

-dir / --migration_dir — optional parameter for providing path to directory with migration files

Example usage:

clickhouse-migrate create_migration -n test_migration -c /home/clickhouse-migrate/config/conf.ini
clickhouse-migrate create_migration -n test_migration -dir /home/my_project/migrations -db clickhouse+native://default:@localhost:9000/db -db clickhouse+native://default:@localhost:9001/db

After calling the above-mentioned command a blank migration file YYYY-MM-DD-HH-mm-ss_<migration_name>.py will be created in the directory which was defined in the *.ini configuration file.

The content of the created file will look like this

from clickhouse_migrate import Step

migrations = [Step(sql="")]

One should use Step dataclass to create migrations. Migrations would be applied in the same order that they are stated in the migrations list variable. All of listed changes in the migration list variable are by default applied to all databases listed in the *.ini config file, since usually changes are required for all replicated instances.

If an SQL query stated in the Step dataclass should not be executed for all DB replicas one may use Step(sql="<sql query>", replicated=False) statement.

This rule does not apply in case there is only one connection string in *.ini file.

Apply migrations

One can apply migrations created via clickhouse-migrate create_migration command by calling clickhouse-migrate migrate.

Command parameters:

--config <path to the created config file> — this is an optional parameter, one may want to use it when the config file is not located in the root directory or if its' name is different from default one

-db / --databases — optional parameter for providing database connection strings

-dir / --migration_dir — optional parameter for providing path to directory with migration files

Example usage:

clickhouse-migrate migrate -c /home/clickhouse-migrate/config/conf.ini
clickhouse-migrate migrate -dir /home/my_project/migrations -db clickhouse+native://default:@localhost:9000/db

After calling this command all changes from migration files will be applied step-by-step. Changes are stored in clickhouse_migrate table.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

clickhouse-migrate-0.0.3.tar.gz (10.0 kB view details)

Uploaded Source

Built Distribution

clickhouse_migrate-0.0.3-py3-none-any.whl (12.9 kB view details)

Uploaded Python 3

File details

Details for the file clickhouse-migrate-0.0.3.tar.gz.

File metadata

  • Download URL: clickhouse-migrate-0.0.3.tar.gz
  • Upload date:
  • Size: 10.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.7.9

File hashes

Hashes for clickhouse-migrate-0.0.3.tar.gz
Algorithm Hash digest
SHA256 6335f253e5ffb9ff664d4fe37de48cf30a71083084d49ca544cfd0f6d2d3be93
MD5 2f7916b325ffb963f13934e6a8197a65
BLAKE2b-256 4246a878345d2fb7628b03bd51daaa8dfc20806e52af42a0bad08d4a50850a9e

See more details on using hashes here.

File details

Details for the file clickhouse_migrate-0.0.3-py3-none-any.whl.

File metadata

File hashes

Hashes for clickhouse_migrate-0.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 2c248bee3750e12f2850a0f1e8aec2781637440db98fa450cc329a8482da5141
MD5 b32c4bbf93305f2100a6c138e60f1afb
BLAKE2b-256 4ad957efd000be6c3909ede15aa330e357a42642b6ea7ff2e89d03b1653ffe4d

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page