Appengine-Fixture-Loader 0.1.9

Installing

For the less adventurous, Appengine-Fixture-Loader is available on PyPI at https://pypi.python.org/pypi/Appengine-Fixture-Loader.

Single-kind loads

Let’s say you have a model like this:

class Person(ndb.Model):
    """Our sample class"""
    first_name = ndb.StringProperty()
    last_name = ndb.StringProperty()
    born = ndb.DateTimeProperty()
    userid = ndb.IntegerProperty()
    thermostat_set_to = ndb.FloatProperty()
    snores = ndb.BooleanProperty()
    started_school = ndb.DateProperty()
    sleeptime = ndb.TimeProperty()
    favorite_movies = ndb.JsonProperty()
    processed = ndb.BooleanProperty(default=False)

If you want to load a data file like this:

[
    {
        "__id__": "jdoe",
        "born": "1968-03-03T00:00:00",
        "first_name": "John",
        "last_name": "Doe",
        "favorite_movies": [
            "2001",
            "The Day The Earth Stood Still (1951)"
        ],
        "snores": false,
        "sleeptime": "23:00",
        "started_school": "1974-02-15",
        "thermostat_set_to": 18.34,
        "userid": 1
    },

...

    {
        "born": "1980-05-25T00:00:00",
        "first_name": "Bob",
        "last_name": "Schneier",
        "favorite_movies": [
            "2001",
            "Superman"
        ],
        "snores": true,
        "sleeptime": "22:00",
        "started_school": "1985-08-01",
        "thermostat_set_to": 18.34,
        "userid": -5
    }
]

All you need to do is to:

from appengine_fixture_loader.loader import load_fixture

and then:

loaded_data = load_fixture('tests/persons.json', kind=Person)

In our example, loaded_data will contain a list of already persisted Person models you can then manipulate and persist again.

The __id__ attribute, when defined, will save the object with that given id. In our case, the key to the first object defined will be a ndb.Key(‘Person’, ‘jdoe’). The key may be defined on an object by object base - where the __id__ parameter is omitted, an automatic id will be generated - the key to the second one will be something like ndb.Key(‘Person’, 1).

Multi-kind loads

It’s convenient to be able to load multiple kinds of objects from a single file. For those cases, we provide a simple way to identify the kind of object being loaded and to provide a set of models to use when loading the objects.

Consider our original example model:

class Person(ndb.Model):
    """Our sample class"""
    first_name = ndb.StringProperty()
    last_name = ndb.StringProperty()
    born = ndb.DateTimeProperty()
    userid = ndb.IntegerProperty()
    thermostat_set_to = ndb.FloatProperty()
    snores = ndb.BooleanProperty()
    started_school = ndb.DateProperty()
    sleeptime = ndb.TimeProperty()
    favorite_movies = ndb.JsonProperty()
    processed = ndb.BooleanProperty(default=False)

and let’s add a second one:

class Dog(ndb.Model):
    """Another sample class"""
    name = ndb.StringProperty()

Now, if we wanted to make a single file load objects of the two kinds, we’d need to use the __kind__ attribute in the JSON:

[
    {
        "__kind__": "Person",
        "born": "1968-03-03T00:00:00",
        "first_name": "John",
        "last_name": "Doe",
        "favorite_movies": [
            "2001",
            "The Day The Earth Stood Still (1951)"
        ],
        "snores": false,
        "sleeptime": "23:00",
        "started_school": "1974-02-15",
        "thermostat_set_to": 18.34,
        "userid": 1
    },
    {
        "__kind__": "Dog",
        "name": "Fido"
    }
]

And, to load the file, we’d have to:

from appengine_fixture_loader.loader import load_fixture

and:

loaded_data = load_fixture('tests/persons_and_dogs.json',
                           kind={'Person': Person, 'Dog': Dog})

will result in a list of Persons and Dogs (in this case, one person and one dog).

Multi-kind, multi-level loads

Anther common case is having hierarchies of entities that you want to reconstruct for your tests.

Using slightly modified versions of our example classes:

class Person(ndb.Model):
    """Our sample class"""
    first_name = ndb.StringProperty()
    last_name = ndb.StringProperty()
    born = ndb.DateTimeProperty()
    userid = ndb.IntegerProperty()
    thermostat_set_to = ndb.FloatProperty()
    snores = ndb.BooleanProperty()
    started_school = ndb.DateProperty()
    sleeptime = ndb.TimeProperty()
    favorite_movies = ndb.JsonProperty()
    processed = ndb.BooleanProperty(default=False)
    appropriate_adult = ndb.KeyProperty()

and:

class Dog(ndb.Model):
    """Another sample class"""
    name = ndb.StringProperty()
    processed = ndb.BooleanProperty(default=False)
    owner = ndb.KeyProperty()

And using __children__[attribute_name]__ like meta-attributes, as in:

[
    {
        "__kind__": "Person",
        "born": "1968-03-03T00:00:00",
        "first_name": "John",
        "last_name": "Doe",

        ...

        "__children__appropriate_adult__": [
            {
                "__kind__": "Person",
                "born": "1970-04-27T00:00:00",

                ...

                "__children__appropriate_adult__": [
                    {
                        "__kind__": "Person",
                        "born": "1980-05-25T00:00:00",
                        "first_name": "Bob",

                        ...

                        "userid": 3
                    }
                ]
            }
        ]
    },
    {
        "__kind__": "Person",
        "born": "1999-09-19T00:00:00",
        "first_name": "Alice",

        ...

        "__children__appropriate_adult__": [
            {
                "__kind__": "Person",

                ...

                "__children__owner__": [
                    {
                        "__kind__": "Dog",
                        "name": "Fido"
                    }
                ]
            }
        ]
    }
]

you can reconstruct entire entity trees for your tests.

Parent/Ancestor-based relationships with automatic keys

It’s also possible to set the parent by using the __children__ attribute.

For our example classes, importing:

[
    {
        "__kind__": "Person",
        "first_name": "Alice",

        ...

        "__children__": [
            {
                "__kind__": "Person",
                "first_name": "Bob",
                ...

                "__children__owner__": [
                    {
                        "__kind__": "Dog",
                        "name": "Fido"
                    }
                ]
            }
        ]
    }
]

should be equivalent to:

alice = Person(first_name='Alice')
alice.put()
bob = Person(first_name='Bob', parent=alice)
bob.put()
fido = Dog(name='Fido', parent=bob)
fido.put()

You can then retrieve fido with:

fido = Dog.query(ancestor=alice.key).get()

History

0.1.0 (2014-10-13)

• First release on GitHub.

0.1.1 (2014-12-4)

• Add support for multi-kind JSON files

0.1.2 (2014-12-4)

• Minor fixes

0.1.3 (2014-12-5)

• Added support for PropertyKey-based child entities

0.1.4 (2015-2-4)

• Fixed bug in which post-processor was called on every property change

• Added section on development to README.rst

0.1.5 (2015-2-11)

• Added __children__ support

• Added manual key definition through the __id__ attribute

0.1.6 (2015-8-30)

• Builds if you don’t have curl installed

• Minor documentation improvements

0.1.7 (2015-11-3)

• Syntax highlighting on the documentation

• Coverage analysis using Coveralls

0.1.8 (2016-02-05)

• New resources/Makefile

0.1.9 (2016-12-19)

• Replace pep8 with pycodestyle

• Update current SDK version detection to latest version