Skip to main content

TiDB schema migration and governance tool — part of Schemashifts

Project description

tidbchange

TiDB schema migration and governance tool — part of the Schemashifts platform.

Other migration tools treat TiDB as MySQL. They handle CREATE TABLE and ALTER TABLE just fine. But the features that make TiDB worth adopting — TiFlash columnar replicas, placement policies, resource groups, HTAP workload isolation — are completely outside what either tool can express or track.

tidbchange fills that gap.

What tidbchange does that other tools cannot

Feature Other tools tidbchange
TiFlash replica management Raw SQL only, no async monitoring Full lifecycle with replication progress polling
TiFlash backfill wait Fires DDL and moves on — AVAILABLE=0 silently Polls INFORMATION_SCHEMA.TIFLASH_REPLICA until PROGRESS=1.0
Placement policies Not expressible Full create/attach/detach/drop lifecycle in YAML
Resource groups Not expressible Full create/alter/drop + priority/burstable in YAML
Drift detection Schema only Tables + TiFlash replicas + placement policies + resource groups
HTAP governance None 12 analyser rules + policy-as-code contracts
Baseline generation Basic DDL Full cluster state including TiDB-native features

Quick start

pip install tidbchange        # or: pip install -e .
tidbchange deploy --profile local

tidbchange.yml

default_profile: local

profiles:
  local:
    host:     127.0.0.1
    port:     4000            # TiDB default port
    user:     root
    password: ""
    database: ecommerce
    root_folder: ./migrations
    tiflash_timeout: 120

  dev:
    host:     ${TIDB_DEV_HOST}
    port:     4000
    user:     tidbchange
    password: ${TIDB_DEV_PASSWORD}
    database: ecommerce
    ssl:      true
    tiflash_timeout: 1800
    environment: dev

Migration script formats

SQL (.sql) — standard MySQL-compatible DDL

-- V1.0.0__create_users.sql
CREATE TABLE `users` (
    `id`    BIGINT NOT NULL AUTO_RANDOM,  -- AUTO_RANDOM avoids hot spots
    `email` VARCHAR(255) NOT NULL,
    PRIMARY KEY (`id`),
    UNIQUE KEY `uq_email` (`email`)
) ENGINE=InnoDB;

YAML (.yaml) — TiDB-native features

# V1.2.0__add_tiflash_replicas.yaml
tiflash_replicas:
  - table: order_analytics
    replicas: 1
  - table: orders
    replicas: 1

# V1.3.0__resource_groups.yaml
resource_groups:
  - name: analytics_rg
    ru_per_sec: 5000
    priority: LOW
    burstable: false

# V1.4.0__placement.yaml
placement_policies:
  - name: eu_policy
    primary_region: eu-west-1
    regions: [eu-west-1, eu-central-1]
    schedule: EVEN

attach_placement:
  - table: users
    policy: eu_policy

Commands

tidbchange deploy   --profile local               # apply all pending migrations
tidbchange deploy   --profile local --target-version 1.2.0  # deploy up to V1.2.0
tidbchange rollback --profile local --tag release-2.0       # undo a tagged release
tidbchange status   --profile local               # show migration history
tidbchange validate                               # lint scripts offline (no connection)
tidbchange baseline --profile local               # generate V0.0.0 from live cluster
tidbchange audit    --profile local               # compliance audit log

TiFlash backfill monitoring

The key feature of tidbchange over other tools for TiDB:

[INFO] SET TiFlash REPLICA 1  on  order_analytics
[INFO] Waiting for TiFlash replicas on 'ecommerce.order_analytics' to be AVAILABLE (timeout=1800s)...
[INFO] TiFlash replication progress: 0.0%  (elapsed 0s)
[INFO] ⏳ TiFlash replication 23.4%  (elapsed 30s, timeout in 1770s)
[INFO] ⏳ TiFlash replication 61.8%  (elapsed 60s, timeout in 1740s)
[INFO] TiFlash replication progress: 100.0%  (elapsed 82s)
[INFO] ✓ TiFlash replicas AVAILABLE for 'ecommerce.order_analytics'  (took 82s)

Other tools fire ALTER TABLE SET TIFLASH REPLICA 1 and immediately moves to the next script. The replica is left in AVAILABLE=0 state, and any OLAP queries silently fall back to TiKV rowstore. tidbchange waits until AVAILABLE=1 and PROGRESS=1.0 before proceeding.

Requirements

  • Python 3.9+
  • TiDB 5.0+ (for TiFlash) or TiDB 6.0+ (for placement policies and resource groups)
  • TiDB Cloud or self-hosted TiDB

License

Apache 2.0

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tidbchange-1.0.1.tar.gz (31.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tidbchange-1.0.1-py3-none-any.whl (33.0 kB view details)

Uploaded Python 3

File details

Details for the file tidbchange-1.0.1.tar.gz.

File metadata

  • Download URL: tidbchange-1.0.1.tar.gz
  • Upload date:
  • Size: 31.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.10

File hashes

Hashes for tidbchange-1.0.1.tar.gz
Algorithm Hash digest
SHA256 dc8205bc7dc04381306a747bf954efbfc5b8755c882f9e68ac17b890626adb51
MD5 9f3991c9461b1f84aa9036d1430eb55e
BLAKE2b-256 ca2d7629d86d9cbdaa18f9781926adf7e04b5e88541af528302b387fa8c9661f

See more details on using hashes here.

File details

Details for the file tidbchange-1.0.1-py3-none-any.whl.

File metadata

  • Download URL: tidbchange-1.0.1-py3-none-any.whl
  • Upload date:
  • Size: 33.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.10

File hashes

Hashes for tidbchange-1.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 02cc040875d29eab335b42f599ec8b9127ff99f0ef7bec86e5f3d595375eaeb3
MD5 932f58979d6c125f704611f29e757e3f
BLAKE2b-256 2f7194804c85413862488b29b1a95f6685b3bc9f7545f49bfd7749857781746d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page