json-projection

Strip the parts of a JSON document you do not want before a parser or validator sees them. Kept members are copied byte-for-byte; everything else is skipped by jiter's cursor without ever being parsed. Zero runtime dependencies. Optional pydantic integration that excludes fields on any model, including models from libraries you do not control.

Why

pydantic parses the whole document into a tree before validating, then discards what the model does not need. For payloads whose bulk sits under unwanted keys that is most of the cost:

shape (~1 MB) Model.model_validate_json with json-projection
big array under a key the model ignores (S1) 4.363 ms / 25.90 MiB 0.364 ms / 0.01 MiB
big nested field excluded inside a list (S3) 13.223 ms / 41.33 MiB 0.744 ms / 0.24 MiB
kubernetes-like list, drop spec/status (S12) 9.794 ms / 18.52 MiB 1.363 ms / 0.51 MiB
everything kept, nothing to skip (S7) 5.922 ms / 8.50 MiB 7.006 ms / 9.91 MiB

Full table and method in benchmarks/results.md.

Install

pip install json-projection              # projector only
pip install 'json-projection[pydantic]'  # plus the pydantic integration (pydantic >= 2.13)

Usage

import json_projection as jp

jp.project(raw, {"name", "id"})  # keep two root keys
p = jp.Projection({"name": True, "items": {"__all__": {"id": True}}})  # compile once
p(raw)  # keep items[*].id and name

A spec is a set of root keys, or a mapping where a key maps to True (keep the whole value), a nested mapping (descend into an object) or {"__all__": spec} (apply to each array element). Everything not named is dropped.

With pydantic

from json_projection.pydantic import Projected

thin = Projected(Log, exclude={Log: {"debug"}, Sample: {"events"}})
log = thin.validate_json(raw)  # real Log/Sample instances; excluded fields are never parsed

Excluded fields that have a default keep their default. Excluded required fields are absent from the instance and from model_fields_set. exclude is keyed by class so nested third-party models can be targeted; a plain set means the root model.

What you give up

  • Error payloads show the projected document. A ValidationError on a whole object reports the input without the dropped members.
  • UTF-8 inside dropped data is not validated. Escapes, control characters, numbers and nesting still are.
  • Nesting depth is not accounted globally. Every dropped or kept member starts a fresh nesting budget of 200 below the member that holds it, and the containers the projector descends through cost nothing, so a document can be accepted at any total depth where pydantic alone rejects it for exceeding its recursion limit. What comes out is still bounded by pydantic's own parse of the kept members.
  • Kept data is copied once. When little can be dropped, or a huge string sits under a dropped key, the copy costs more than it saves (see the last benchmark row).
  • Partial instances: a model with excluded required fields should not also appear inside a union; the serializer warns about the missing fields there.
  • The pydantic integration relies on the core-schema layout and on SchemaValidator(..., _use_prebuilt=False), which pydantic does not promise to keep. CI tests the latest release and pre-release; a RuntimeError is raised if pydantic-core ignores the schema edit.
  • Refused configurations. projected_validator on its own refuses, with ValueError, models whose config would still consume an excluded key: extra='forbid' or extra='allow', and populate_by_name/validate_by_name when an excluded field has a default. Projected lifts that restriction only when its derived projection provably reaches every occurrence of every excluded class; where it cannot (see "Kept-whole subtrees"), the guards stay on.
  • Before/wrap/plain validators are not projected through. Their input is not the shape the schema they wrap describes, so a model or field behind one is kept whole and nothing inside it is projected. A root model whose own fields sit behind a model_validator(mode='before'), 'wrap' or 'plain' cannot be projected at all: Projected and projection_spec raise TypeError. Use projected_validator alone.
  • After validators and excluded required fields. A model_validator(mode='after') that touches an excluded required field raises AttributeError from validate_json, not a ValidationError.
  • Extras. A model with extra='allow' that has no excluded field of its own is kept whole (nothing below it is projected). A model with extra='allow' that does have excluded fields loses all its extras: the projection keeps only declared fields.
  • Standalone projected_validator. Excluded fields with a default are redirected to the alias \x00excluded:<name>; a document that contains that literal key still populates the field. Projected strips it. Validating with by_name=True or by_alias=False on the returned SchemaValidator looks the field up under its own name again, so a document that carries the excluded key populates the field and the default is not applied. Projected.validate_json refuses both flags with ValueError whenever its projection is incomplete, and forwards them when it is not (the projection stripped alias and name alike).
  • Kept-whole subtrees. The derived projection describes models, lists, sets and variable-length tuples; everything else is kept whole -- dict values, unions, fixed tuples, dataclasses, TypedDicts, Any, a class that appears inside itself, an extra='allow' model with no excluded fields of its own, the object named by a multi-segment alias path, a JSON key that two fields describe differently. Nothing inside a kept subtree is projected, so when an excluded class is reachable in one, Projected keeps the standalone guards on for every class in the schema.
  • RootModel is unsupported in 0.1: projection_spec and Projected raise TypeError, and projected_validator reports the excluded names as not found. A RootModel declares one field, root, and the class it wraps is reached through it like any other nested model.
  • Custom __init__. A model that defines its own __init__ is refused with ValueError when exclusion would touch it or anything below it: pydantic-core calls that __init__, which validates through the class's original validator and ignores the exclusion entirely.
  • A field aliased __all__ cannot be expressed: __all__ is the array wildcard in a mapping spec, so a derived spec containing it is rejected with TypeError.
  • Data-dependent defaults and computed fields. A default_factory that takes the validated data raises a plain KeyError from validate_json when it reads an excluded field, and a computed field that reads an excluded attribute raises AttributeError from model_dump. Neither becomes a ValidationError.

Errors

On invalid JSON, or a root that is not an object, project returns the input unchanged so that whatever parses it next reports the error at the original position. Pass strict=True to raise ValueError instead.

Supported

CPython 3.9 to 3.14 (one abi3 wheel per platform); Linux x86_64 and aarch64, macOS, Windows. Free-threaded builds are not supported yet.

Releasing

  1. Bump version in Cargo.toml, move the changelog entry from "unreleased" to the date, commit.
  2. git tag vX.Y.Z && git push --tags. The release workflow builds wheels for Linux x86_64/aarch64, macOS arm64/x86_64 and Windows x64 plus the sdist, and publishes via PyPI trusted publishing.

One-time setup before the first tag: create the json-projection project on PyPI, add a trusted publisher for rasmusfaber/json-projection with workflow release.yml and environment pypi, and create the pypi environment in the GitHub repository settings.

Project links

Data verified by PyPI on Sep 7, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for rasmusfaber from gravatar.com rasmusfaber

Credits

Author: Rasmus Faber-Espensen

GitHub Statistics

Data verified by PyPI on Sep 7, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.9

Provides Extra

pydantic

Tags

json pydantic jiter performance projection

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

json_projection-0.1.0.tar.gz (94.7 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

json_projection-0.1.0-cp39-abi3-win_amd64.whl (175.7 kB view details)

Uploaded CPython 3.9+Windows x86-64

json_projection-0.1.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (291.8 kB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ x86-64

json_projection-0.1.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (290.1 kB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64

json_projection-0.1.0-cp39-abi3-macosx_11_0_arm64.whl (268.2 kB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

json_projection-0.1.0-cp39-abi3-macosx_10_12_x86_64.whl (273.4 kB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file json_projection-0.1.0.tar.gz.

File metadata

  • Download URL: json_projection-0.1.0.tar.gz
  • Upload date:
  • Size: 94.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for json_projection-0.1.0.tar.gz
Algorithm Hash digest
SHA256 46c503a829c1e359551927589c4c9876f2d55d6723700d8e25dd04af2d1401a2
MD5 85356db99381f4abbf6424f30ca89831
BLAKE2b-256 0948057ffa98a1e80ba11bf3529c4db090b219b4af0d65905f44817a96b3bbe8

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_projection-0.1.0.tar.gz:

Publisher: release.yml on rasmusfaber/json-projection

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file json_projection-0.1.0-cp39-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for json_projection-0.1.0-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 47068948e3bdee7c6d7391cda5076f6f85a5a9ad980ad79083542ab2531bbc70
MD5 1372672cc25a6c7be364ee7ec83bf216
BLAKE2b-256 d13611df9aee6e7d0e3bdf1ed2a87b68ba2b9f020e97a76f828c1b49d7a5b997

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_projection-0.1.0-cp39-abi3-win_amd64.whl:

Publisher: release.yml on rasmusfaber/json-projection

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file json_projection-0.1.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for json_projection-0.1.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 02f56d2f833bb51122cea43f6d5ae97895da638a538176cc491a88793a6ed96b
MD5 ff440f9f7bd846746c491db5b40e1f7c
BLAKE2b-256 de1509d024a88e492fa4b423bf54b203351bc0282ed31d37adb67296d572083c

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_projection-0.1.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on rasmusfaber/json-projection

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file json_projection-0.1.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for json_projection-0.1.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 5dbecf07293a29337fe7eb63fb67ea08ab8e8d7c431e2ef3a81cdd14410dc04a
MD5 b9ec4c96447b48999c5a626e2581c3c9
BLAKE2b-256 769c38308914830ec9e99883e6638e887c0a86cc954be15d4a5cd707e6b93029

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_projection-0.1.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on rasmusfaber/json-projection

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file json_projection-0.1.0-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for json_projection-0.1.0-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0aa0794c5119fc8661e77637ae991b886eb8d969ce03677076041187e70c6e51
MD5 d3b13b6eb1273e35bd3109c2e4b0734e
BLAKE2b-256 162d318d5cfa31b2c1a8825c76d8db4e2f65552d3f02d83afd7d5a413cdab29c

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_projection-0.1.0-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on rasmusfaber/json-projection

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file json_projection-0.1.0-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for json_projection-0.1.0-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 f24f8e5555951df33a7250647ed035a2d753b947f977c75110c00418fb57ca0e
MD5 cfa27a834ce18e3e0da030b6ea56093f
BLAKE2b-256 99e3d383ec1e209948a6c2861bfee3ac86e39573825bdfee4f793f29ce6aa8dd

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_projection-0.1.0-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on rasmusfaber/json-projection

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.1.0 This release

6 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page