graphty is a Python library for materializing Pydantic object graphs from relational data.

WARNING: This project is in an early stage of development and should be used with caution.

The core idea of graphty is to utilize Pydantic models as a declarative DSL for building Polars query plans.

Instead of writing imperative data transformation code - group this, aggregate that, nest this inside that - users should be able to define the shape of what they want as a Pydantic model hierarchy; and graphty figures out the respective materialization from flat tabular data.

graphty interprets Pydantic models as DataFrame transformation specifications.

Introduction

The graphty library addresses the structural impedance mismatch between flat relational data representations and hierarchical object models. It extends Pydantic with a small declarative DSL for expressing grouping, aggregation, and deduplication operations. These transformations are compiled into Polars expressions, yielding records that are subsequently validated and materialized as Pydantic model objects.

Although originally developed for implementing typed REST APIs over SPARQL endpoints, the model materializer can be applied to any tabular data representation, including SQL query results, CSV files, dataframes, etc.

Installation

graphty is a PEP 621-compliant package and available on PyPI.

Usage

As mentioned, graphty uses Pydantic model definitions as declarative data transformation instructions, extending Pydantic with a small DSL for grouping and aggregation.

Nested models are resolved recursively, list types are interpreted as aggregation targets and require a group_by definition in ConfigDict.

Basic Example

Given simple relational Author/Work data

data = [
    {"name": "Tolkien", "title": "The Hobbit", "year": 1937},
    {"name": "Tolkien", "title": "The Lord of the Rings", "year": 1954},
    {"name": "Tolkien", "title": "The Silmarillion", "year": 1977},
    {"name": "Orwell", "title": "Animal Farm", "year": 1945},
    {"name": "Orwell", "title": "1984", "year": 1949},
]

one can define and materialize a Pydantic model like so:

from collections.abc import Iterator
from pydantic import BaseModel
from graphty import ConfigDict, ModelMaterializer

class Work(BaseModel):
    title: str
    year: int

class Author(BaseModel):
    model_config = ConfigDict(group_by="name")

    name: str
    works: list[Work]
 
models: Iterator[Author] = ModelMaterializer(model=Author, data=data).generate_models()

Here, the Author model defines a model aggregation target for the Author.works field; the graphty planner will therefore partition the underlying data according to the "name" key and aggregate Work objects into a list.

Note that graphty is recursive on all code paths and ergo enables materialization of arbitrarily nested and aggregated object graphs.

The above validates against the Author model and serializes to the following JSON representation:

[
    {
        "name": "Tolkien",
        "works": [
            {
                "title": "The Hobbit",
                "year": 1937
            },
            {
                "title": "The Lord of the Rings",
                "year": 1954
            },
            {
                "title": "The Silmarillion",
                "year": 1977
            }
        ]
    },
    {
        "name": "Orwell",
        "works": [
            {
                "title": "Animal Farm",
                "year": 1945
            },
            {
                "title": "1984",
                "year": 1949
            }
        ]
    }
]