Skip to main content
https://travis-ci.org/someones/exodus.svg?branch=master

Mass Migrations for all!

Exodus is a simple framework that allows you to define data migrations and apply them to an entire data store, or individual objects.

Exodus does not know anything about your data store, instead each migration provides the logic for accessing the data store.

The ability to migrate an entire data store, or a single object at a time, enable Exodus to update data at read time, or through a batch process.

Exodus reads python files from a specified directory, migration objects are automatically registered and sorted by version.

By not providing extensive abstractions, Exodus allows you to migrate your data the way you want to. You won’t have to fight Exodus.

How to Use

Create a database migration module - ‘./migrations’ by default - and add a migration file. A specific filename format is not required, but it is recommended to use either the current date, an incrementing number, or both, as the prefix followed by a short description.

Eg:

  • 2015_10_12_add_address_to_user.py

  • 01_add_address_to_user.py, 02_add_rating_to_movie.py

A migration should inherit from BaseMigration and must define a number of variables and methods:

from exodus import BaseMigration

class Migration(BaseMigration):
    # used to sort the migrations, recommended to use a string that matches
    # the filename's unique prefix.
    version = '2015_10_20'

    # an optional list of class name variables that are used to check if
    # an object should be migrated. These values should match the
    # class.__name__ value of the class / object to be migrated.
    # the logic that uses this can be modified by over-riding the
    # `can_migrate_object` method
    classes = ['TestObject']

    def can_migrate_database(self, adapter):
        '''Returns True if the database can be migrated, False otherwise.
        This function should use your own logic to check the database version
        or other 'signal' that the migration is appropriate for your database.
        '''
        return self.version > adapter['version']

    def migrate_database(self, adapter):
        '''Performs a full migration on the database.
        This can include moving or updating objects.
        For each object requiring a migration, the method `migrate_object` should
        be called.
        By default, migrate_object will look for a function matching `migrate_<class name>`.
        If the `classes` variable defines a class name, but there is no corresponding `migrate_<class name>`,
        an exception will be thrown on construction.
        Before calling `migrate_<class name>`, a check if made for a function matching
        the name `can_migrate_<class name>`, if found (it is entirely optional to define)
        it is called with the object as a parameter. If False, the migration is
        not performed.
        Ensure any migration version signals are set at the end of the function.
        '''
        adapter['objects'] = adapter['old_objects']
        del adapter['old_objects']

        for obj in adapter['objects']:
            self.migrate_object(obj)
        adapter['version'] = self.version

    def can_migrate_TestObject(self, obj):
        '''Called when an object of type TestObject is sent to `migrate_object`.
        Returns True if the migration should be applied to the object.
        This function is entirely optional. If not defined, the migration will be
        performed regardless.
        '''
        return obj.version < self.version

    def migrate_TestObject(self, obj):
        '''Performs a migration on the object.
        If the `classes` variable defines a class name, but there is no corresponding `migrate_<class name>`,
        an exception will be thrown on construction.
        Ensure any migration version signals are set at the end of the function.
        '''
        obj.my_value = obj.my_value + 1
        obj.version = self.version
        return obj

Overriding Migration Logic

At times the default migration logic is not appropriate. The majority of the intelligence is located in the BaseMigration class to allow you to over-ride logic whenever required.

For example, you could modify a Migration to work on serialised JSON data rather than Python classes like so:

class JsonMigration(BaseMigration):
    def can_migrate_object(self, obj):
        if not self.classes:
            return False
        # the class name is stored in the __class__ value of the dict
        obj = json.loads(obj)
        clsname = obj['__class__']
        return clsname in self.classes

    def migrate_object(self, obj):
        # load the string
        parsed = json.loads(obj)

        # get the object class name and dispatch to the appropriate function
        clsname = parsed['__class__']

        # check if we can migrate the object
        # this is an optional function
        func = self._can_migrate_object_func(clsname)
        if func:
            if not func(parsed):
                return obj

        # perform the migration
        func = self._migrate_object_func(clsname)
        if func:
            parsed = func(parsed)
            # convert back to a string
            return json.dumps(parsed)

    def can_migrate_TestObject(self, obj):
        return obj['version'] < self.version

    def migrate_TestObject(self, obj):
        obj['my_value'] = obj['my_value'] + 1
        obj['version'] = self.version
        return obj

Authors

Metadata

Release files for exodus 1.0.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for exodus 1.0.8
File Size Uploaded Provenance
exodus-1.0.8.tar.gz 4.4 kB Without provenance Details

Release files / exodus-1.0.8.tar.gz

Download URL exodus-1.0.8.tar.gz
Size 4.4 kB
Tags Source
SHA-256 checksum
How to use checksums
311664b847f5d70346356b8f7331aaf452e05e6c755662b83c0747b2dea33199
BLAKE2b-256 checksum
How to use checksums
9b429b9d593628dbbba36930530a13bcf39fbfb83e86841ce111990863a3700e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Security for exodus 1.0.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page