Skip to main content

NoSQL db join

Project description

db-join

I realized during my personal journey using Google Datastore that I was doing something very often on a set of DB entities. That is, I had foreign key references on various fields in an entity and wanted to load the entities refenced by those keys. Additionally, I wanted to control which entities are loaded with some syntactic sugar and I wanted to do it efficienntly.

Hence the birth of db-join -- a NoSQL version of join semantics you get with a SQL db.

Basics

First, create an instance of a joiner. Right now, only DatastoreJoin class exists but I hope overtime other NoSQL dbs wrappers can be added.

>>> import join
>>> joiner = join.api.DatastoreJoin()

Next, given a iterable (typically via query), join against all the fields you wish.

>>> iterable = joiner(client, iterable, ('field1', 'field2'))

What this does is discovers if the DB keys referenced by field1 and field2 (if any) and does a get_multi on this keys and the mutates the db object with those discovered entities. Thus, after the join both field1 and field2 will refer to entities (instead of keys) assuming they are in fact keys and those keys do in fact refer to db entities.

Dotted notation

These fields may actually be dotted patterns as documented in the dotted-notation package. Dotted notation permits you to fetch an item inside a deeply nested datastructure.

>>> d = {'hello': {'there': [{'a': 1, 'b': 2}, {'a': 7, 'b': 8}]}}
>>> dotted.get(d, 'hello.there[1].b') == 8

Thus, if your DB entity has a list of keys OR something nested you can specify how to fetch it. For example,

>>> joiner(client, iterable, 'list_of_keys[*]')

This will join on all keys contained in a list referenced by list_of_keys.

Chaining

But that's not all. A pattern may also use chaining notation:

>>> joiner(client, iterable, 'field1->another_field')

This will fetch the object at field1 and then fetch that object's the object at another_field.

Replacing

Turns out sometimes you want to replace the object with a referenced object. The ! operator lets you do this:

>>> joiner(client, iterable, '!field1')

This will replace the object at yielded by iterable with whatever object was found at field1. Note that this works with chaining as well.

>>> joiner(client, iterable, 'field1->!symlink')

This will replace field1 with whatever was found in symlink.

Recursive chaining

Similar to replacing, sometimes you want to recursively expand objects that are linked via the same field. The + operator lets you do this:

>>> joiner(client, iterable, '+field')

This will fetch the object at field. If that object also has field, then it will fetch the object at that object's field and so on until there's no more work.

Internals

A Join class has a number of abstractions to help you out. The two most important are the getter and the setter. These methods are called whenever you're getting a value of a field that matches pattern and when you're setting that value.

The default behavior is to just use dotted.get and dotted.update respectively.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

db-join-0.1.8.tar.gz (9.1 kB view details)

Uploaded Source

Built Distribution

db_join-0.1.8-py3-none-any.whl (10.1 kB view details)

Uploaded Python 3

File details

Details for the file db-join-0.1.8.tar.gz.

File metadata

  • Download URL: db-join-0.1.8.tar.gz
  • Upload date:
  • Size: 9.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.0 requests/2.25.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.9.12

File hashes

Hashes for db-join-0.1.8.tar.gz
Algorithm Hash digest
SHA256 f6205a88859446bb82d3b2231d4b88e31918ccaf74b18d3404782d15367b49f2
MD5 97ef50560716fe55d684aac39818ddc4
BLAKE2b-256 f37764a19638a6e1867f5ba5997c266f0515427e1bd80f28d94c54ffccc9d219

See more details on using hashes here.

File details

Details for the file db_join-0.1.8-py3-none-any.whl.

File metadata

  • Download URL: db_join-0.1.8-py3-none-any.whl
  • Upload date:
  • Size: 10.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.0 requests/2.25.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.9.12

File hashes

Hashes for db_join-0.1.8-py3-none-any.whl
Algorithm Hash digest
SHA256 a76906dd1f2d366f2d4b287fd3eb7114134090b8a6f6470af1a39f14a2ac79a1
MD5 6a2a34215ba2033aa7bf36066afdd05b
BLAKE2b-256 b15898af7924512231bdb51ba250637aa3e5d65b97ccbd189d119920b278b4d5

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page