Skip to main content
Donate to the Python Software Foundation or Purchase a PyCharm License to Benefit the PSF! Donate Now

Data migration that can be configured using JSON

Project description

Magnivore

Pypi Travis

A data migration tool built to make migrating entire databases as simple as writing json migration rules. MySQL, Sqlite and Postgres are supported.

A quick look

A simple migration rule:

{
    "profiles": {
        "sources": [
            {"table": "users"},
            {"table": "addresses", "on":"user"}
        ],
        "transform": {
            "name": "username",
            "city": "addresses.city"
        }
    }
}

This would migrate data to the profiles table, using data from users and joined addresses. In the transform section, we specify which fields we actually want to migrate.

In this case users.username would be migrated to name and addresses.city to city.

Installing

  1. Ensure that you have the necessary database drivers. You will need psycopg for postgres and PyMySQL for MySql

  2. Install from pip:

    pip install magnivore
    
  3. Configure magnivore. You can generate a skeleton config file with:

    magnivore config-skeleton
    
  1. Initialize magnivore:

    magnivore init
    
  2. Say hello:

    magnivore hello
    

You have successfully installed magnivore! You can now write migration rules and execute them with:

magnivore run myrules.json

Documentation

You can find the full documentation at http://magnivore.readthedocs.io

Troubleshooting

Currently there is poor support for table names that contain dashes, so ensure that your table names have no dashes.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
magnivore-0.4.2-py3-none-any.whl (11.8 kB) Copy SHA256 hash SHA256 Wheel 3.5
magnivore-0.4.2.tar.gz (8.5 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page