versionedobj 0.2.2

Easy object serialization & versioning framework

This package provides an easy way to define complex nested objects that can be saved/loaded to/from strings or JSON files, and easily migrated from older object versions.

See API documentation

Example– VersionedObject as a configuration file

from versionedobj import VersionedObject

class DisplayConfig(VersionedObject):
    display_mode = "windowed"
    resolution = "1920x1080"
    volume = 0.66

# Populate class attributes to build your object
class UserConfig(VersionedObject):
    version = "v1.0.0"
    username = "john smith"
    friend_list = ["user1", "user2", "user3"]
    display_config = DisplayConfig() # VersionedObjects can be nested

# Create an instance of your object (instance attributes will match class attributes,
# and the initial values will be whatever values you set on the class attributes)
cfg = UserConfig()

# Change some values on the object instance
cfg.display_config.volume = 1.0
cfg.username = "jane doe"

# Save object instance to JSON file
cfg.to_file('user_config.json', indent=4)

# Load object instance from JSON file
cfg.from_file('user_config.json')

You can also save/load object data as a JSON string:

>>> obj_as_json = cfg.to_json(indent=4) # Serialize to JSON string
>>> obj_as_json                         # Print JSON string

{
    "version": "v1.0.0",
    "username": "jane doe",
    "friend_list": [
            "user1",
            "user2",
            "user3"
    ],
    "display_config": {
        "display_mode": "windowed",
        "resolution": "1920x1080",
        "volume": 1.0
    }
}

>>> cfg.from_json(obj_as_json)          # Load from JSON string

Or, as a dict:

>>> obj_as_dict = cfg.to_dict()   # Serialize to dict
>>> obj_as_dict                   # Print dict

{'version': '1.0.0', 'username': 'jane doe', 'friend_list': ['user1', 'user2', 'user3'], 'display_config': {'display_mode': 'windowed', 'resolution': '1920x1080', 'volume': 1.0}}

>>> cfg.from_dict(obj_as_dict)    # Load from dict

Filtering output

Whitelisting output/input by field name

When serializing, if you only want to output certain fields, you can use the ‘only’ parameter to specify which fields should be output (effectively a whitelist by field name):

cfg.to_file('user_config.json', only=['version', 'username', 'display_config.resolution'])

# Output looks like this:
#
# {
#     "version": "v1.0.0",
#     "username": "jane doe",
#     "display_config": {
#         "resolution": "1920x1080",
#     }
# }

The same parameter can be used for de-serializing:

cfg.from_file('user_config.json', only=['display_config.display_mode'])

# Only the 'display_config.display_mode' field is loaded from the file

Blacklisting output/input by field name

When serializing, if you don’t want to output certain fields, you can use the ‘ignore’ parameter to specify which fields should be excluded from output (effectively a blacklist by field name):

cfg.to_file('user_config.json', ignore=['friend_list', 'display_config.volume'])

# Output looks like this:
#
# {
#     "version": "v1.0.0",
#     "username": "jane doe",
#     "display_config": {
#         "display_mode": "windowed",
#         "resolution": "1920x1080"
#     }
# }

The same parameter can be used for de-serializing:

cfg.from_file('user_config.json', ignore=['friend_list'])

# Every field except for the 'friend_list' field is loaded from the file

Migrations – making use of the version number

Any VersionedObject object can have a version attribute, which can be any object, although it is typically a string (e.g. "v1.2.3"). This version attribute can be used to support migrations for older objects, in the event that you need to change the format of your object.

Example scenario, part 1: you have created a beautiful versioned object

Let’s take the same config file definition from the previous example:

from versionedobj import VersionedObject

# Nested config object
class DisplayConfig(VersionedObject):
    display_mode = "windowed"
    resolution = "1920x1080"
    volume = 0.66

# Top-level config object with another nested config object
class UserConfig(VersionedObject):
    version = "v1.0.0"
    username = "john smith"
    friend_list = ["user1", "user2", "user3"]
    display_config = DisplayConfig()

Imagine you’ve already released this code out into the world. People are already using it, and they have JSON files generated by your UserConfig class sitting on their computers.

Example scenario, part 2: you update your software, modifying the versioned object

Now, imagine you are making a new release of your software, and some new features require you to make the following changes to your versioned object:

• remove the the DisplayConfig.resolution field entirely

• change the name of DisplayConfig.volume to DisplayConfig.volumes

• change the value of DisplayConfig.volumes from a float to a list

from versionedobj import VersionedObject

# Nested config object
class DisplayConfig(VersionedObject):
    display_mode = "windowed"
    # 'resolution' field is deleted
    volumes = [0.66, 0.1] # 'volume' is now called 'volumes', and is a list

# Top-level config object with another nested config object
class UserConfig(VersionedObject):
    version = "v1.0.0"
    username = "john smith"
    friend_list = ["user1", "user2", "user3"]
    display_config = DisplayConfig()

Uh-oh, you have a problem…

Right now, if you send this updated UserConfig class to your existing users, it will fail to load their existing JSON files with version v1.0.0, since those files will contain the DisplayConfig.resolution field that we deleted in v1.0.1, and DisplayConfig.volume will similarly be gone, having been replaced with DisplayConfig.volumes. This situation is what migrations are for.

Solution– migrations!

The solution is to:

1. Change the version number to something new, e.g. v1.0.0 becomes v1.0.1

2. Write a migration function to transform v1.0.0 object data into v1.0.1 object data

from versionedobj import VersionedObject

# Nested config object
class DisplayConfig(VersionedObject):
    display_mode = "windowed"
    # 'resolution' field is deleted
    volumes = [0.66, 0.1] # 'volume' is now called 'volumes', and is a list

# Top-level config object with another nested config object
class UserConfig(VersionedObject):
    version = "v1.0.1" # Version has been updated to 1.0.1
    username = "john smith"
    friend_list = ["user1", "user2", "user3"]
    display_config = DisplayConfig()

# Create the migration function for v1.0.0 to v1.0.1
def migrate_100_to_101(attrs):
    del attrs['display_config']['resolution']        # Delete resolution field
    del attrs['display_config']['volume']            # Delete volume field
    attrs['display_config']['volumes'] = [0.66, 0.1] # Add defaults for new volume values
    return attrs                                     # Return modified data (important!)

# Add the migration function for v1.0.0 to v1.0.1
UserConfig.add_migration("v1.0.0", "v1.0.1", migrate_100_to_101)

after you add the migration function and update the version to v1.0.1, JSON files loaded with version``v1.0.0`` will be migrated to version v1.0.1.

The downside to this approach, is that you have to manually udpate the version number, and write a new migration function, anytime the structure of your config data changes.

The upside, of course, is that you can relatively easily support migrating any older version of your config file to the current version.

If you don’t need the versioning/migration functionality, just never change your version number, or don’t create a version attribute on your VersionedObject classes.