NoSQL db join
Project description
db-join
I realized during my personal journey using Google Datastore that I was doing something very often on a set of DB entities. That is, I had foreign key references on various fields in an entity and wanted to load the entities refenced by those keys. Additionally, I wanted to control which entities are loaded with some syntactic sugar and I wanted to do it efficienntly.
Hence the birth of db-join -- a NoSQL version of join semantics you get with a SQL db.
Basics
First, create an instance of a joiner. Right now, only DatastoreJoin
class exists
but I hope overtime other NoSQL dbs wrappers can be added.
>>> import join
>>> joiner = join.api.DatastoreJoin()
Next, given a iterable (typically via query), join against all the fields you wish.
>>> iterable = joiner(client, iterable, ('field1', 'field2'))
What this does is discovers if the DB keys referenced by field1
and field2
(if any)
and does a get_multi
on this keys and the mutates the db object with those discovered
entities. Thus, after the join both field1
and field2
will refer to entities (instead
of keys) assuming they are in fact keys and those keys do in fact refer to db entities.
Dotted notation
These fields may actually be dotted patterns as documented in the dotted-notation
package. Dotted notation permits you to fetch an item inside a deeply nested
datastructure.
>>> d = {'hello': {'there': [{'a': 1, 'b': 2}, {'a': 7, 'b': 8}]}}
>>> dotted.get(d, 'hello.there[1].b') == 8
Thus, if your DB entity has a list of keys OR something nested you can specify how to fetch it. For example,
>>> joiner(client, iterable, 'list_of_keys[*]')
This will join on all keys contained in a list referenced by list_of_keys
.
Chaining
But that's not all. A pattern may also use chaining notation:
>>> joiner(client, iterable, 'field1->another_field')
This will fetch the object at field1
and then fetch that object's the object at
another_field
.
Replacing
Turns out sometimes you want to replace the object with a referenced object. The !
operator lets you do this:
>>> joiner(client, iterable, '!field1')
This will replace the object at yielded by iterable with whatever object was found
at field1
. Note that this works with chaining as well.
>>> joiner(client, iterable, 'field1->!symlink')
This will replace field1
with whatever was found in symlink
.
Recursive chaining
Similar to replacing, sometimes you want to recursively expand objects that are linked
via the same field. The +
operator lets you do this:
>>> joiner(client, iterable, '+field')
This will fetch the object at field
. If that object also has field
, then it will
fetch the object at that object's field
and so on until there's no more work.
Internals
A Join
class has a number of abstractions to help you out. The two most important are
the getter
and the setter
. These methods are called whenever you're getting a value
of a field that matches pattern and when you're setting that value.
The default behavior is to just use dotted.get
and dotted.update
respectively.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.