Skip to main content

Helps manage migrations for databases and anything else

Project description

https://travis-ci.org/lieturd/migrate-anything.svg?branch=master https://img.shields.io/badge/code%20style-black-000000.svg https://codecov.io/gh/Lieturd/migrate-anything/branch/master/graph/badge.svg

Migrate anything - database (etc.) migration utility, especially for Python projects.

What is this?

It’s kinda annoying how often you run into the question of how to handle migrations in your project, and there hasn’t seem to emerged any good, DB -agnostic, framework-agnostic, and storage-agnostic tool to manage them.

This project is an attempt to change that.

Usage examples

Firstly you’ll need this package in your project. Pick one of these:

pip install -U migrate-anything
poetry add migrate-anything
pipenv install migrate-anything

Simply put, create a Python package, don’t be too clever and call it e.g. migrations. Then put files in that package:

# migrations/__init__.py
from migrate_anything import configure, CSVStorage

configure(storage=CSVStorage("test.csv"))
# migrations/01-initialize-db.py
# Please note that this is built for a completely hypothetical DB layer
from my_db import get_db

DB = get_db()

def up():
    DB.create_table("example")

def down():
    DB.delete_table("example")

This would configure your migrations’ status to be stored in a local folder called migration_status and set up your first migration script. If you have a my_db module that works like this you could just run this with the command

migrate-anything migrations

Now in the real world you might want something more durable and a realistic example, so here’s another (still hypothetical) version:

# __init__.py
from migrate_anything import configure, KubernetesConfigMap

configure(storage=KubernetesConfigMap("migrate-anything-status"))
# 01-initialize-db.py
from pymongo import MongoClient

client = MongoClient()  # Hope you're really not using defaults
db = client.testdb

def up():
    db.posts.insert_one({
        "id": "post-1",
        "title": "We're live!",
        "content": "This is our first post, yay."
    })
    db.posts.create_index("id")

def down():
    db.posts.drop()

This would configure storage to a Kubernetes ConfigMap (if that class existed), and run migrations to MongoDB. There’s also going to be storage modules to MongoDB available but this is way cooler.

Oh and your Kubernetes pods will likely require the necessary RBAC rules to manage their ConfigMap. It’s unfortunately kinda complex, but I’m sure you can figure it out e.g. with this guide. Alternatively you can just use the MongoDB storage, or write your own - it’s easy.

# __init__.py
from migrate_anything import configure


class CustomStorage(object):
    def __init__(self, file):
        self.file = file

    def save_migration(self, name, code):
        with open(self.file, "a", encoding="utf-8") as file:
            file.write("{},{}\n".format(name, code))

    def list_migrations(self):
        try:
            with open(self.file, encoding="utf-8") as file:
                return [
                    line.split(",")
                    for line in file.readlines()
                    if line.strip()  # Skip empty lines
                ]
        except FileNotFoundError:
            return []

    def remove_migration(self, name):
        migrations = [
            migration for migration in self.list_migrations() if migration[0] != name
        ]

        with open(self.file, "w", encoding="utf-8") as file:
            for row in migrations:
                file.write("{},{}\n".format(*row))


configure(storage=CustomStorage("test.txt"))

You can also check out the examples.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for migrate-anything, version 0.1.2
Filename, size File type Python version Upload date Hashes
Filename, size migrate_anything-0.1.2-py2.py3-none-any.whl (7.9 kB) File type Wheel Python version py2.py3 Upload date Hashes View hashes
Filename, size migrate-anything-0.1.2.tar.gz (5.8 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page