json-merge-tree

JSON Merge Tree (built on top of json-multi-merge) merges JSON objects in a hierarchy, allowing customization of how objects are merged.

The hierarchy is defined by the caller's parents module, which specifies a function for each resource type that returns the parent ID and another function to get its parents (unless it's the top level).

It relies on a SQLAlchemy Table containing the fields id (UUID), resource_id (UUID), resource_type (string), slug (string), inherits (boolean), draft_set (string), created_at (timestamp), and a field that stores the JSON object you want to merge.

Installation

Install this package, e.g., pip install json-merge-tree.

## merge_tree

Main entrypoint for merging JSON objects in a table up a hierarchy defined by `parents`.

### Example
```python
from json_merge_tree import merge_tree

import widget_service.parents
from widget_service.db import widgets

# Merge widgets of all the requested resource's ancestors and return the result
merged_json = merge_tree(
    table=widgets, 
    id='228135fb-6a3d-4551-93db-17ed1bbe466a', 
    type='brand', 
    json_field='widget', 
    parents=widget_service.parents, 
    slugs=None, 
    debug='annotate'
)

Selected columns

The query selects only resource_id, slug, inherits, and the json_field column, which the merge reads, and id, which callbacks commonly read. When debug is set, it also selects resource_type and draft_set for the debug annotations.

The records passed to callback contain only these columns. If your callback reads any other column, add it to the query with buff_query:

def buff_query(C, query, filters):
    return query.add_columns(C.updated_at), filters

def callback(record):
    print(record.updated_at)

merged_json = merge_tree(
    table=widgets,
    id='228135fb-6a3d-4551-93db-17ed1bbe466a',
    type='brand',
    json_field='widget',
    parents=widget_service.parents,
    buff_query=buff_query,
    callback=callback,
)

async_merge_tree

The asyncio version of merge_tree, taking the same arguments and returning the same result.

• db is an AsyncConnection, or an AsyncEngine. Given an engine, it connects only for the one query, after all the parents have been fetched, so no connection is held while the parent getters call other services.
• The getters in parents may be coroutine functions that return the (id, type, getter) tuples, or plain functions/generators like those merge_tree takes.
• Each resource's parents are fetched concurrently (e.g. a page's site and section chains), but merged in the order their getter returned them, so the result is the same as merge_tree's.

from json_merge_tree import async_merge_tree

merged_json = await async_merge_tree(
    db=read_engine,
    table=widgets,
    id='228135fb-6a3d-4551-93db-17ed1bbe466a',
    type='brand',
    json_field='widget',
    parents=widget_service.parents,
)

AsyncTreeMerger is the class behind it, as TreeMerger is behind merge_tree.

merge

Generic function to merge two JSON objects together. Largely the same as jsonmerge.Merger.merge but with the added ability to customize how objects are merged with the annotations below.

Merge key annotations

You can append the annotation listed below to a key at any level to customize how its value affects the merged json.

-- Unset

Unset this key in the merged json. The value of this key does not matter - you can set it to null.

• E.g. 1

{
  "colors": {
    "content": "#000000", 
    "section--": null
  }
}

merged into

{
  "colors": {
    "section": "#000001"
  }
}

results in

{
  "colors": {
    "content": "#000000"
  }
}

• E.g. 2

{
  "styles": {
    "h1--": null, 
    "h2": {
      "fontSize": 3
    }
  }
}

merged into

{
  "styles": {
    "h1": {
      "fontWeight": "heading"
    }
  }
}

results in

{
  "styles": {
    "h2": {
      "fontSize": 3
    }
  }
}

! Replace

Replace this key's value with this value.

• E.g. 1

{
  "colors": {
      "content": "#000000", 
      "section!": "#000002"
  }
}

merged into

{
  "colors": {
    "section": "#000001"
  }
}

results in

{
  "colors": {
    "content": "#000000", 
    "section": "#000002"
  }
}

• E.g. 2

{
  "styles": {
    "h1!": {
      "fontFamily": "heading",
      "fontSize": 5
    }
  }
}

merged into

{
  "styles": {
    "h1": {
      "fontWeight": "heading"
    }
  }
}

results in

{
  "styles": {
    "h1": {
      "fontFamily": "heading",
      "fontSize": 5
    }
  }
}

Slugs

Slugs are kind of a weird feature – and getting weirder as we use them to solve more use cases.

Originally, A slug was a named json object scoped under a resource in the hierarchy to be merged. In our case, a custom-page theme merged under one of the resources in our resource hierarchy.

The use of slugs has been extended to included named json object mixins not associated with any one resource at the bottom of the hierarchy, but scoped to a resource at any level. For example, a site wants to have a "dark" theme that could be applied to any page within the site.

When merging, the library first merges the json objects without slugs, then merges json objects with slugs at each level. Multiple slugs can be included when merging.

slugs_only

When slugs_only=True is passed to merge_tree, the resource JSON at every level of the hierarchy is skipped — only slug JSON objects are merged. This is useful when you only need the slug data without any of the base resource JSON mixed in.

merged_json = merge_tree(
    table=widgets,
    id='228135fb-6a3d-4551-93db-17ed1bbe466a',
    type='brand',
    json_field='widget',
    parents=widget_service.parents,
    slugs=['dark-theme'],
    slugs_only=True,
)

Note that the underlying query still fetches all resource records (to check inherits and maintain correct cache invalidation), it just excludes them from the merge.