Skip to main content

sqlalchemy-json

SQLAlchemy-JSON provides mutation-tracked JSON types to SQLAlchemy:

  • MutableJson is a straightforward implementation for keeping track of top-level changes to JSON objects;

  • NestedMutableJson is an extension of this which tracks changes even when these happen in nested objects or arrays (Python dicts and lists).

Examples

Basic change tracking

This is essentially the SQLAlchemy mutable JSON recipe. We define a simple author model which list the author’s name and a property handles for various social media handles used:

class Author(Base):
    name = Column(Text)
    handles = Column(MutableJson)

Or, using the declarative mapping style:

class Category(Base):
    __tablename__ = "categories"

    id = mapped_column(Integer, primary_key=True)
    created_at: Mapped[DateTime] = mapped_column(DateTime, default=datetime.now)
    updated_at: Mapped[DateTime] = mapped_column(
        DateTime, default=datetime.now, onupdate=datetime.now
    )
    keywords: Mapped[list[str]] = mapped_column(MutableJson)

The example below loads one of the existing authors and retrieves the mapping of social media handles. The error in the twitter handle is then corrected and committed. The change is detected by SQLAlchemy and the appropriate UPDATE statement is generated.

>>> author = session.query(Author).first()
>>> author.handles
{'twitter': '@JohnDoe', 'facebook': 'JohnDoe'}
>>> author.handles['twitter'] = '@JDoe'
>>> session.commit()
>>> author.handles
{'twitter': '@JDoe', 'facebook': 'JohnDoe'}

Nested change tracking

The example below defines a simple model for articles. One of the properties on this model is a mutable JSON structure called references which includes a count of links that the article contains, grouped by domain:

class Article(Base):
    author = Column(ForeignKey('author.name'))
    content = Column(Text)
    references = Column(NestedMutableJson)

With this in place, an existing article is loaded and its current references inspected. Following that, the count for one of these is increased by ten, and the session is committed:

>>> article = session.query(Article).first()
>>> article.references
{'github.com': {'edelooff/sqlalchemy-json': 4, 'zzzeek/sqlalchemy': 7}}
>>> article.references['github.com']['edelooff/sqlalchemy-json'] += 10
>>> session.commit()
>>> article.references
{'github.com': {'edelooff/sqlalchemy-json': 14, 'zzzeek/sqlalchemy': 7}}

Had the articles model used MutableJson like in the previous example this code would have failed. This is because the top level dictionary is never altered directly. The nested mutable ensures the change happening at the lower level bubbles up to the outermost container.

Non-native JSON / other serialization types

By default, sqlalchemy-json uses the JSON column type provided by SQLAlchemy (specifically sqlalchemy.types.JSON.) If you wish to use another type (e.g. PostgreSQL’s JSONB), your database does not natively support JSON (e.g. versions of SQLite before 3.37.2/), or you wish to serialize to a format other than JSON, you’ll need to provide a different backing type.

This is done by using the utility function mutable_json_type. This type creator function accepts two parameters:

  • dbtype controls the database type used. This can be an existing type provided by SQLAlchemy or SQLALchemy-utils, or an augmented type to provide serialization to any other format;

  • nested controls whether the created type is made mutable based on MutableDict or NestedMutable (defaults to False for MutableDict).

import json

from sqlalchemy import JSON, String, TypeDecorator
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy_json import mutable_json_type

class JsonString(TypeDecorator):
    """Enables JSON storage by encoding and decoding on the fly."""

    impl = String

    def process_bind_param(self, value, dialect):
        return json.dumps(value)

    def process_result_value(self, value, dialect):
        return json.loads(value)


postgres_jsonb_mutable = mutable_json_type(dbtype=JSONB)
string_backed_nested_mutable = mutable_json_type(dbtype=JsonString, nested=True)

Dependencies

  • sqlalchemy

Development

Here’s how to setup your development environment:

python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
# run tests
pytest

Changelog

0.7.0

0.6.0

0.5.0

0.4.0

  • Adds a type creation function to allow for custom or alternate serialization types. This allows for a way around the regression in SQLite compatibility introduced by v0.3.0.

0.3.0

  • Switches JSON base type to sqlalchemy.types.JSON from deprecated JSON type provided by SQLAlchemy-utils.

0.2.2

0.2.1

  • Fixes a typo in the README found after uploading 0.2.0 to PyPI.

0.2.0 (unreleased)

  • Now uses JSONType provided by SQLAlchemy-utils to handle backend storage;

  • Backwards incompatible: Changed class name JsonObject to MutableJson and NestedJsonObject to NestedMutableJson

  • Outermost container for NestedMutableJson can now be an array (Python list)

0.1.0 (unreleased)

Initial version. This initially carried a 1.0.0 version number but has never been released on PyPI.

Metadata

Release files for sqlalchemy-json 0.7.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sqlalchemy-json 0.7.0
File Size Uploaded
sqlalchemy-json-0.7.0.tar.gz 8.8 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for sqlalchemy-json 0.7.0
File Interpreter ABI Platform
sqlalchemy_json-0.7.0-py3-none-any.whl Python 3 none any Details

Total release size: 16.5 kB

Release files / sqlalchemy-json-0.7.0.tar.gz

Download URL sqlalchemy-json-0.7.0.tar.gz
Size 8.8 kB
Tags Source
SHA-256 checksum
How to use checksums
620d0b26f648f21a8fa9127df66f55f83a5ab4ae010e5397a5c6989a08238561
BLAKE2b-256 checksum
How to use checksums
3e5773ba3d0ee5efbec5a0d15ee3e21606edd33b1d1fd11b5d64e581c8b8a3f6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.12

Release files / sqlalchemy_json-0.7.0-py3-none-any.whl

Download URL sqlalchemy_json-0.7.0-py3-none-any.whl
Size 7.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
27881d662ca18363a4ac28175cc47ea2a6f2bef997ae1159c151026b741818e6
BLAKE2b-256 checksum
How to use checksums
e1791fc7309ecf75756e9ad9280f19cd83ca7b79a0dae36cd025f668e8e2741f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.12

Release history Release notifications | RSS feed

This release

0.7.0 This release

2 release files

0.6.0

2 release files

0.5.0

1 release file

0.4.0

2 release files

0.3.0

1 release file

0.2.3

1 release file

0.2.2

1 release file

0.2.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page