shoobx.immutable 2.0.2

Immutable Types

shoobx.immutable – Immutable Types

This library provides a state-based implementation of immutable types, including lists, sets and dicts. It handles an arbitrarily deep structure of nested objects.

In addition, support for revisioned immutables is provided, which allows for full revision histories of an immutable. A sample implementation of a revisioned immutable manager is also provided.

Optional: A pjpersist-based storage mechanism for revisioned immutables is provided, which allows for easy storage of versioned immutables.

Using Immutables

Immutable objects can make certain complex systems more reasonable, because they tightly control when an object is modified and how. It also guarantees that an object can never change for another accessor in a different subsystem.

Introduction

Let’s start with a simple dictionary:

>>> import shoobx.immutable as im

>>> with im.create(im.ImmutableDict) as factory:
...     answer = factory({
...         'question': 'Answer to the ultimate question of life, ...',
...         'answer': 0
...     })

>>> answer['answer']
0

But no value can be changed anymore:

>>> answer['answer'] = 42
Traceback (most recent call last):
...
AttributeError: Cannot update locked immutable object.

Immutable objects are updated through a special context manager that creates a new version of the object that can be modified within the context manager block.

>>> orig = answer
>>> with im.update(answer) as answer:
...     answer['answer'] = 42

>>> answer['answer']
42

Note that the answer dictionary is a completely new object and that the original object is still unmodified.

>>> orig is not answer
True
>>> orig['answer']
0

Of course we can also create complex object structures, for example by adding a list:

>>> with im.update(answer) as answer:
...     answer['witnesses'] = ['Arthur', 'Gag']

>>> answer['witnesses']
['Arthur', 'Gag']

Of course, the list has been converted to its immutable equal, so that items cannot be modified.

>>> isinstance(answer['witnesses'], im.ImmutableList)
True
>>> answer['witnesses'].append('Deep Thought')
Traceback (most recent call last):
...
AttributeError: Cannot update locked immutable object.

However, updating from an child/sub-object is not allowed, since creating a new version of a child would sematically modify its parent thus violating the immutability constraint:

>>> with im.update(answer['witnesses']) as witnesses:
...     pass
Traceback (most recent call last):
...
AttributeError: update() is only available for master immutables.

The system accomplishes this by assigning “master” and “slave” modes to the immutables. The root immutable is the master and any sub-objects below are slaves.

Immutable sets are also supported as a core immutable:

>>> data = im.ImmutableSet({6})
>>> data
{6}

>>> with im.update(data) as data:
...     data.discard(6)
...     data.add(9)
>>> data
{9}

Custom Immutables

Creating your own immutable objects is simple:

>>> class Answer(im.Immutable):
...     def __init__(self, question=None, answer=None, witnesses=None):
...         self.question = question
...         self.answer = answer
...         self.witnesses = witnesses

>>> answer = Answer('The Answer', 42, ['Arthur', 'Gag'])
>>> answer.answer
42

Note how the list is automatically converted to its immutable equivalent:

>>> isinstance(answer.witnesses, im.ImmutableList)
True

Of course you cannot modify an immutable other than the update context:

>>> answer.answer = 54
Traceback (most recent call last):
...
AttributeError: Cannot update locked immutable object.

>>> with im.update(answer) as answer:
...     answer.answer = 54
>>> answer.answer
54

Revisioned Immutables

Since mutables create a new object for every change, they are ideal for creating systems that have to keep track of their entire history. This package provides support for such systems by defining a revision manager API and revisioned immutable that are managed within it.

Let’s start by creating a custom revisioned immutable:

>>> class Answer(im.RevisionedImmutable):
...
...     def __init__(self, question=None, answer=None):
...         self.question = question
...         self.answer = answer

A simple implementation of the revision manager API is provided to demonstrate a possible implementation path.

>>> data = im.RevisionedMapping()
>>> data['a'] = answer = Answer('Answer to the ultimate question')

The answer is the current revision and has been added to the manager.

>>> data['a'] is answer
True

In addition to the usual immutability features, the Revisioned immutable has several additional attributes that help with the management of the revisions:

>>> answer.__im_start_on__
datetime.datetime(...)
>>> answer.__im_end_on__ is None
True
>>> answer.__im_manager__
<shoobx.immutable.revisioned.SimpleRevisionedImmutableManager ...>
>>> answer.__im_creator__ is None
True
>>> answer.__im_comment__ is None
True

The update API is extended to support setting the creator and comment of the change:

>>> answer_r1 = answer
>>> with im.update(answer, 'universe', 'Provide Answer') as answer:
...     answer.answer = 42

We now have a second revision of the answer that has the comemnt and creator set:

>>> answer.answer
42

>>> answer.__im_start_on__
datetime.datetime(...)
>>> answer.__im_end_on__ is None
True
>>> answer.__im_creator__
'universe'
>>> answer.__im_comment__
'Provide Answer'

The first revision is now retired and has an end date/time (which equals the start date/time of the new revision):

>>> answer_r1.__im_start_on__
datetime.datetime(...)
>>> answer_r1.__im_end_on__ == answer.__im_start_on__
True
>>> answer_r1.__im_state__ == im.interfaces.IM_STATE_RETIRED
True

The manager has APIs to manage the various revisions.

>>> revisions = data.getRevisionManager('a')
>>> len(revisions.getRevisionHistory())
2

>>> revisions.getCurrentRevision(answer_r1) is answer
True

We can even roll back to a previous revision:

>>> revisions.rollbackToRevision(answer_r1)

>>> len(revisions.getRevisionHistory())
1
>>> answer_r1.__im_end_on__ is None
True
>>> answer_r1.__im_state__ == im.interfaces.IM_STATE_LOCKED
True

Optional pjpersist Support

A more serious and production-ready implementation of the revision manager API is provided in shoobx.immutable.pjpersist which utilizes pjpersist to store all data.

Notes

A technical discussion on the system’s inner workings is located in the doc strings of the corresponding interfaces. In addition, the tests covera a lot of special cases not dicsussed here.

CHANGES

2.0.2 (2020-04-27)

• Added enum.Enum as immutable type / constant. This allows assigning an Enum to a ImmutableBase attribute.

2.0.1 (2020-04-23)

• Allow defaultInfo() decorator to be nested.

2.0.0 (2020-04-21)

• IMPORTANT: Add immutable state as a column to the table. This will require a migration of your database schema and data.

• Introduced new IM_STATE_DELETED state which marks an object as deleted.

• Add new _pj_with_deleted_items flag that when set will change the container API to return deleted items as well.

• Added ImmutableContainer.withDeletedItems() method that will clone the container and set the _pj_with_deleted_items flag. That will by definition reset all caches to prohibit inconsistent results.

• The test_functional_deletionAndRevival() demonstrates the deletion and revivial functionality.

1.5.0 (2020-04-20)

• Honor the _pj_remove_documents flag in the pjpersist ImmutableContainer by simply marking the last version of the object as retired and assigning an end date. This way deletions can be undone. Also, audit logs can now be complete.

• Allow the creator and comment to be specified globally, so that APIs don’t have to carry that information through all the layers.

1.4.3 (2020-02-22)

• Make sure that ImmutableContainer does not accept transient objects. This is particularly important since objects can be initialized in transient state when not using the create() context manager. It also protects the object from being updated in a container before completing its update.

• Refactored __delitem__ tests to be more minimal and document the use cases more clearly.

1.4.2 (2020-02-15)

• 1.4.1 was a brown bag release.

1.4.1 (2020-02-15)

• Missed to re-export shoobx.immutable.immutable.create

1.4.0 (2020-02-14)

• Changed the pattern of creating an immutable object to a context manager. NOTE, just creating an object like Immutable() will give you a transient object. The preferred pattern is:

  >>> import shoobx.immutable as im
  >>> with im.create(im.Immutable) as factory:
  ...     imObj = factory()
  

  This makes it way easier to set initial attributes. See README.rst and docs and tests for details.

1.3.1 (2020-02-10)

• Fixing leftover _pj_get_resolve_filter occurrences in ImmutableContainer

1.3.0 (2020-02-06)

• Fix ImmutableContainer.__delitem__ : In order to delete all revisions of an object, the delete method used an internal super() call to get query filters. That ended up ignoring subclass filters causing deletes across contianer boundaries.

  As a solution, a new _pj_get_resolve_filter_all_versions method has been introduced to return a query for all versions within a container. The _pj_get_resolve_filter method now uses the other one and simply adds the “latest version” constraint. All sub-containers should now override _pj_get_resolve_filter_all_versions instead of _pj_get_resolve_filter.

1.2.1 (2020-02-02)

• Fix ImmutableContainer.__delitem__ : it did not remove revisions of the deleted object

• Fix ImmutableContainer.rollbackToRevision : it rolled back ALL objects to the given revision

1.2.0 (2020-01-20)

• Extended IRevisionedImmutableManager to support efficient version management.

  • Added getNumberOfRevisions(obj) method to return the number of revisions available for a given object. Note that this does not necessarily equal to the latest revision number.

  • Exended getRevisionHistory() with multiple new arguments to support filtering, sorting and batching:

    Filter Arguments:

    • creator: The creator of the revision must match the argument.

    • comment: The comment must contain the argument as a substring.

    • startBefore: The revision must start before the given date/time.

    • startAfter: The revision must start after the given date/time.

    Ordering Arguments:

    • reversed: When true, the history will be return in reverse

      chronological order, specifically the latest revision is listed first.

    Batching Arguments:

    • batchStart: The index at which to start the batch.

    • batchSize: The size the of the batch. It is thus the max length of

      the iterable.

• Provided an implementation of the new arguments for both the simple revision manage and the pjpersist container.

• Declare that ImmutableContainer implements IRevisionedImmutableManager.

• Increased test coverage back to 100%.

1.1.1 (2019-06-11)

• Added datetime classes as system immutable types.

1.1.0 (2019-05-31)

• Introduced __im_version__ to IRevisionedImmutable and use it instead of timestamps to create a chronological order of revisions. (Timestamps might be slightly different accross servers and cause bad history.)

• Do not duplicate implementation of __im_update__() in RevisionedImmutableBase. Use __im_[before|after]_update__() to do all revision-related tasks.

• Tweak copy() implementation for ImmutableList and ImmutableDict.

• Properly implement ImmutableDict.fromkeys().

1.0.5 (2019-05-31)

• Fix ImmutableList.copy() to just work when locked. This allows for only making a shallow clone, since any update will cause a deep copy and thus immutability is guaranteed.

• Implemented ImmutableDict.copy(). Raise error on ImmutableDict.fromkeys().

• ImmutableContainer also needs an updated _pj_column_fields list.

• Minor test fixes.

• Minor documentation fixes and code comment enhancements.

1.0.4 (2019-05-30)

• Add API documentation.

1.0.3 (2019-05-30)

• Moved documentation to Read the Docs.

1.0.2 (2019-05-30)

• Add some readable documentation.

• Added high-level shoobx.immutable.update(im, *args, **kw) function.

• Implemented __repr__() for ImmutableSet to mimic behavior of ImmutableDict and ImmutableList.

1.0.1 (2019-05-30)

• Fix package description.

1.0.0 (2019-05-30)

• Immutable Types, Immutable Dict, Immutable Set, Immutable List

• Revisioned Immutable with Revision Manager sample implementation

• Optional: pjpersist support for immutables. Requires pjpersist>=1.7.0.

• Initial Release