Skip to main content

CDK Construct for managing DB migrations

Project description

CDK DB Migration

Source Test GitHub Docs

npm package PyPI package NuGet package

Downloads npm PyPI NuGet

AWS CDK L3 construct for managing DB migrations. Currently implemented DBMS:

  • Athena

I created this construct because CloudFormations Glue Table doesn't support TBLPROPERTIES. I needed an alternative to create a table. Since creating a table is a DB migration, I created a migration construct instead of a simple table construct, which would be hard to impossible to update.

Usage

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
import cdk_db_migration as Migration

m1 = Migration.Athena(self, "M1",
    up="CREATE EXTERNAL TABLE foo ...;",
    down="DROP TABLE foo;"
)

m2 = Migration.Athena(self, "M2",
    depends_on=m1,
    up="ALTER TABLE foo ...;",
    down="ALTER TABLE foo ...;"
)

Every migration requires a query for up and down migrations. up is executed when the migration is created. down is executed when the migration is destroyed.

A full example including creating bucket, database, workgroup and permissions can be found in the test directory.

Notes

No modifications: The construct will refuse to update any existing migration, because this is not how migrations work. Add another migration or first delete the migration, then add the modified statement.

Dependencies: Since migrations (might) depend on one another, make sure to set dependencies where required. In CDK you usually add dependencies like this:

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
m1 = Migration.Athena(self, "M1", ...)
m2 = Migration.Athena(self, "M2", ...)
m2.node.add_dependency(m1)

Since dependencies are a very common pattern for migrations, a migration also accepts dependencies directly:

# Example automatically generated without compilation. See https://github.com/aws/jsii/issues/826
m1 = Migration.Athena(self, "M1", ...)
m2 = Migration.Athena(self, "M2",
    depends_on=m1, ...
)

Permissions: The Lambda function which runs the migrations, is not authorized to do anything at all, because the required permissions are very custom to the use case (database, workgroup, S3 location, KMS etc). Instead of giving too wide permissions by default, none are given at all. The construct exposes the IAM role and you need to grant the required permissions.

Best solution for your use case?: While the construct is capable of managing the state of a database over time, have a good thought if you really want to do this with CDK/CloudFormation. CloudFormation can ony handle up to 500 resources in a stack, so this (minus all the other resources in your stack) is going to be your hard limit of migrations. Migrations are executed by a Lambda function. Since the maximum execution time of a Lambda function is 15 minutes, migrations cannot exceed this limit.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for cdk-db-migration, version 1.1.0
Filename, size File type Python version Upload date Hashes
Filename, size cdk_db_migration-1.1.0-py3-none-any.whl (84.6 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size cdk-db-migration-1.1.0.tar.gz (86.2 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page