Skip to main content

Local Migrator

Codecov Test Code Style Documentation Status PyPI version Conda-forge version

This support package simplifies data persistence between user sessions and software version updates.

The main idea of this package is simplify data migration between versions, and allow to define migration information next to data structure definition.

Basic usage (data serialization)

If You only need to serialize data, then you could use only JSON hooks

import json

from pydantic import BaseModel
from local_migrator import Encoder, object_hook


class SampleModel(BaseModel):
    field1: int
    field2: str


data = SampleModel(field1=4, field2="abc")

with open("sample.json", "w") as f_p:
    json.dump(data, f_p, cls=Encoder)

with open("sample.json") as f_p:
    data2 = json.load(f_p, object_hook=object_hook)

assert data == data2

Migrations

To register this information there is register_class decorator. It has 4 parameters:

  • version - version of data structure

  • migration_list - list of tuple (version. migration_function).

  • old_paths - list of fully qualified python paths to previous class definitions. This is to allow move class during code refactoring.

  • use_parent_migrations - if True, then parent class migrations will be used.

Lets imagine that we have such code

from local_migrator import Encoder, object_hook

class SampleModel(BaseModel):
    field1: int
    field_ca_1: str
    field_ca_2: float

with open("sample.json", "w") as f_p:
    json.dump(data, f_p, cls=Encoder)

But there is decision to move both ca field to sub structure:

class CaModel(BaseModel):
    field_1: str
    field_2: float

class SampleModel(BaseModel):
    field1: int
    field_ca: CaModel

Then with local_migrator code may look:

from local_migrator import object_hook, register_class

class CaModel(BaseModel):
    field_1: str
    field_2: float

def ca_migration_function(dkt):
    dkt["field_ca"] = CaModel(field1=dkt.pop("field_ca_1"),
                              field2=dkt.pop("field_ca_2"))
    return dkt

@register_class("0.0.1", [("0.0.1", ca_migration_function)])
class SampleModel(BaseModel):
    field1: int
    field_ca: CaModel

with open("sample.json") as f_p:
    data = json.load(f_p, object_hook=object_hook)

Assume that there is decision to rename field1 to id. Then code may look:

from local_migrator import object_hook, register_class, rename_key

class CaModel(BaseModel):
    field_1: str
    field_2: float

def ca_migration_function(dkt):
    dkt["field_ca"] = CaModel(field1=dkt.pop("field_ca_1"),
                              field2=dkt.pop("field_ca_2"))
    return dkt

@register_class("0.0.2", [("0.0.1", ca_migration_function), ("0.0.2", rename_key("field1", "id"))])
class SampleModel(BaseModel):
    id: int
    field_ca: CaModel

with open("sample.json") as f_p:
    data = json.load(f_p, object_hook=object_hook)

More examples could be found in examples section of documentation

Additional functions

  • rename_key(from_key: str, to_key: str, optional=False) -> Callable[[Dict], Dict] - helper function for rename field migrations.

  • update_argument(argument_name:str)(func: Callable) -> Callable - decorator to keep backward compatibility by converting dict argument to some class base on function type annotation

Contributing

Contributions are encouraged! Please create pull request or open issue. For PR please remember to add tests and documentation.

Additional notes

This package is originally named nme but was rename to clarify its purpose.

This package is extracted from PartSeg project for simplify reuse it in another projects.

Metadata

Release files for local-migrator 0.1.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for local-migrator 0.1.10
File Size Uploaded Provenance Actions
local-migrator-0.1.10.tar.gz 24.6 kB Without provenance Details for local-migrator-0.1.10.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for local-migrator 0.1.10
File Interpreter ABI Platform Actions
local_migrator-0.1.10-py3-none-any.whl Python 3 none any Details for local_migrator-0.1.10-py3-none-any.whl

Total release size: 36.3 kB

Release files / local-migrator-0.1.10.tar.gz

Download URL local-migrator-0.1.10.tar.gz
Size 24.6 kB
Tags Source
SHA-256 checksum
How to use checksums
b836adf9ff9c6bacf7922dfd16c26f236df47e5fb492c27c50b9b98ed9ba254c
BLAKE2b-256 checksum
How to use checksums
1eb44d6d5ef31e3de8c5f94e89b41e5c31f31cfaab9fd68bd1ba2a03ec0b67e1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/4.0.2 CPython/3.11.7

Release files / local_migrator-0.1.10-py3-none-any.whl

Download URL local_migrator-0.1.10-py3-none-any.whl
Size 11.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e91861ed84b6d35df165371f174f7ad6ae005bc61ce4aee1e87bd1384166339d
BLAKE2b-256 checksum
How to use checksums
06369a8f451d012549aa7e1782a95b61f482bd1001f6fc7fa7ac1bad3e399dab
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/4.0.2 CPython/3.11.7

Security for local-migrator 0.1.10

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page