Skip to main content

CDK Construct for managing DB migrations

Project description

CDK DB Migration

Source Test GitHub Docs

npm package PyPI package

Downloads npm PyPI

AWS CDK L3 construct for managing DB migrations. Currently implemented DBMS:

  • Athena

I created this construct because CloudFormations Glue Table doesn't support TBLPROPERTIES. I needed an alternative to create a table. Since creating a table is a DB migration, I created a migration construct instead of a simple table construct, which would be hard to impossible to update.

Installation

This package has peer dependencies, which need to be installed along in the expected version.

For TypeScript/NodeJS, add these to your dependencies in package.json. For Python, add these to your requirements.txt:

  • cdk-db-migration
  • aws-cdk-lib (^2.0.0)
  • constructs (^10.0.0)

CDK compatibility

  • Version 2.x is compatible with the CDK v2.
  • Version 1.x is compatible with the CDK v1. There won't be regular updates for this.

Usage

import * as Migration from 'cdk-db-migration';

const m1 = new Migration.Athena(this, 'M1', {
  up: 'CREATE EXTERNAL TABLE foo ...;',
  down: 'DROP TABLE foo;',
 });

const m2 = new Migration.Athena(this, 'M2', {
  dependsOn: m1,
  up: 'ALTER TABLE foo ...;',
  down: 'ALTER TABLE foo ...;',
});

Every migration requires a query for up and down migrations. up is executed when the migration is created. down is executed when the migration is destroyed.

A full example including creating bucket, database, workgroup and permissions can be found in the test directory.

Notes

No modifications: The construct will refuse to update any existing migration, because this is not how migrations work. Add another migration or first delete the migration, then add the modified statement.

Dependencies: Since migrations (might) depend on one another, make sure to set dependencies where required. In CDK you usually add dependencies like this:

const m1 = new Migration.Athena(this, 'M1', {...});
const m2 = new Migration.Athena(this, 'M2', {...});
m2.node.addDependency(m1);

Since dependencies are a very common pattern for migrations, a migration also accepts dependencies directly:

const m1 = new Migration.Athena(this, 'M1', {...});
const m2 = new Migration.Athena(this, 'M2', {
  dependsOn: m1,
  ...
});

Permissions: The Lambda function which runs the migrations, is not authorized to do anything at all, because the required permissions are very custom to the use case (database, workgroup, S3 location, KMS etc). Instead of giving too wide permissions by default, none are given at all. The construct exposes the IAM role and you need to grant the required permissions.

Best solution for your use case?: While the construct is capable of managing the state of a database over time, have a good thought if you really want to do this with CDK/CloudFormation. CloudFormation can ony handle up to 500 resources in a stack, so this (minus all the other resources in your stack) is going to be your hard limit of migrations. Migrations are executed by a Lambda function. Since the maximum execution time of a Lambda function is 15 minutes, migrations cannot exceed this limit.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cdk-db-migration-2.0.0.tar.gz (161.8 kB view details)

Uploaded Source

Built Distribution

cdk_db_migration-2.0.0-py3-none-any.whl (160.4 kB view details)

Uploaded Python 3

File details

Details for the file cdk-db-migration-2.0.0.tar.gz.

File metadata

  • Download URL: cdk-db-migration-2.0.0.tar.gz
  • Upload date:
  • Size: 161.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.7.3

File hashes

Hashes for cdk-db-migration-2.0.0.tar.gz
Algorithm Hash digest
SHA256 8a8e4467f1562a5ddddedef0a8406c5c099dce977174a7710546c7d1d9c6fb04
MD5 afc7a40ff7de72bd2ae7f269b26bd1f1
BLAKE2b-256 8084ffd9871d23318977c7f4fea01672619703a08ab1b53b3380a8c2585f7e8b

See more details on using hashes here.

File details

Details for the file cdk_db_migration-2.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for cdk_db_migration-2.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 8c74b90ffca8abb8c102c40d11c622a0f9ea60c095e91a216f943c3e83085f1a
MD5 8e0c89413a82983473ddfdba0c36ce30
BLAKE2b-256 88cad543e8ed6d9b901a9e9ba04653619a527524aa60a130f58ce66cc21802fb

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page