This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Utilities to access the historical state of objects stored in a (history aware) ZODB (Zope Object DataBase). They can be useful to find out what happened to objects in the past and to restore accidentally deleted or modified objects.

I have used them recently to analyse a case where colleagues reported about mysteriously lost objects. It turned out that the objects have not been lost at all but only became unindexed.

This version is tested against ZODB 3.4 and ZODB 3.6. It will not work with ZODB 3.2. It may (or may not) work with more recent ZODB versions. Beside ZODB, this version requires Zope’s DateTime package.

API

Currently, there are the utilities: getHistory(obj, first=0, last=20), getObjectAt(obj, time), generateHistory(obj) and generateBTreeHistory(btree).

time can be a Zope DateTime instance, a float in seconds since epoch or a serial/transaction id.

generateHistory is a generator version of getHistory.

While generateHistory(btree) reports only changes to the top level btree object, generateBTreeHistory combines the history for all (current) subtrees and subbuckets, i.e. it treats the btree as the application sees it and not on the persistence level only. Note that the history records generated by generateBTreeHistory may not be complete: some deletion operations may be missing. However, there will be at least one entry in the history refering to such deletions.

Usage example

In the example below, the ZODB contains a folder f. Let’s have a look at its history.

>>> from dm.historical import getHistory, getObjectAt
>>> from pprint import pprint as pp
>>> h = getHistory(f)
>>> pp(h)
[{'description': '',
  'obj': <Folder at /f>,
  'size': 157L,
  'tid': '\x03r\xc8\x1b\xf3hhU',
  'time': DateTime('2008/01/01 09:59:57.049 GMT+1'),
  'user_name': '',
  'version': ''},
 {'description': '',
  'obj': <Folder at /f>,
  'size': 124L,
  'tid': '\x03r\xc8\x1bX\x0e\xfb\xcc',
  'time': DateTime('2008/01/01 09:59:20.639 GMT+1'),
  'user_name': '',
  'version': ''},
 {'description': '',
  'obj': <Folder at /f>,
  'size': 48L,
  'tid': '\x03r\xc8\x15\x1c\x9f\xb03',
  'time': DateTime('2008/01/01 09:53:06.709 GMT+1'),
  'user_name': '',
  'version': ''}]

This tells us that the ZODB knows about 3 transactions affecting f. We can access its state after the first transaction.

>>> f1 = h[-1]['obj']
>>> f1.objectIds()
[]

f was empty at that time. Let’s see how f was changed by the other transactions:

>>> for hr in h:
...   print hr['time'].strftime('%H:%M:%S'), hr['obj'].objectIds()
...
09:59:57 ['x', 'y']
09:59:20 ['x']
09:53:06 []

This tells us that the second transaction added x and the third transaction y.

We can control which history records are retrieved with the optional parameters first and last.

>>> pp(getHistory(f,last=1))
[{'description': '',
  'obj': <Folder at /f>,
  'size': 157L,
  'tid': '\x03r\xc8\x1b\xf3hhU',
  'time': DateTime('2008/01/01 09:59:57.049 GMT+1'),
  'user_name': '',
  'version': ''}]
>>> pp(getHistory(f,first=2))
[{'description': '',
  'obj': <Folder at /f>,
  'size': 48L,
  'tid': '\x03r\xc8\x15\x1c\x9f\xb03',
  'time': DateTime('2008/01/01 09:53:06.709 GMT+1'),
  'user_name': '',
  'version': ''}]

getObjectAt can be used to retrieve the historical state at a given time. Say, we want to learn how f was at 9:55. The most easy way to specify a time is via a DateTime.

>>> from DateTime import DateTime
>>> dt=DateTime(2008,1,1,9,55)
>>> dt
DateTime('2008/01/01 09:55:00 GMT+1')
>>> getObjectAt(f, dt).objectIds()
[]

When we request the state beyond the ZODB’s maintained history, we get a POSKeyError.

>>> getObjectAt(f, DateTime(2008,1,1,9,50))
Traceback (most recent call last):
  ...
ZODB.POSException.POSKeyError: ('\x00\x00\x00\x00\x00j\x82\xd3', '\x03r\xc8\x12\x00\x00\x00\x00')

The time can also be specified via a float in the unit seconds since epoch or via a serial/tid. However, these will be used rarely.

>>> ts=dt.timeTime()
>>> ts
1199177700.0
>>> getObjectAt(f, ts).objectIds()
[]
>>> getObjectAt(f, '\x03r\xc8\x15\x1c\x9f\xb03').objectIds()
[]

History

Version 1.2

Made compatible with ZODB 3.10

Version 1.1

added generateHistory and generateBTreeHistory.

Release History

Release History

1.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
dm.historical-1.2.tar.gz (8.5 kB) Copy SHA256 Checksum SHA256 Source Jul 3, 2013

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting