modict 0.4.2

A dict subclass with optional typed fields, validators, computed values, and deep nested ops

modict

modict is a Python dict subclass with an optional model-like layer: typed fields, defaults, factories, validators, computed values, and a full set of deep nested operations.

It stays a real dict throughout — every standard dict method works, and modict instances are accepted everywhere a dict or Mapping is expected, without conversion.

Why not just use…

dict — Great for free-form data, but no types, no validation, no computed fields, no deep ops. You end up reimplementing the same helpers everywhere.

dataclass — Clean syntax for typed containers, but not a dict: you need dataclasses.asdict() at every boundary, no runtime type checking, no coercion, no computed fields with cache invalidation, no extra keys, no nested ops.

TypedDict — A static typing annotation, not a runtime object. No behavior, no defaults, no validators. Only useful for type checkers.

Pydantic BaseModel — Excellent for strict data contracts and API modeling, but model-first: not a dict subclass, requires explicit .model_dump() conversion at boundaries, and is designed around validation-as-contract rather than mutable data manipulation.

attrs — Similar expressiveness to dataclass for modeling, but again not a dict. Adds descriptors and slots but no nested ops, no JSONPath, no coercion pipeline.

modict occupies a different position: it's a dict that can be progressively enriched. Start with raw data, add structure as it stabilizes, keep full dict compatibility throughout. No conversion, no boundaries, no paradigm switch.

When to use modict

• Config and settings: typed defaults, computed derived values, merge/diff/patch between configs.
• JSON/API payloads: parse directly with modict.loads(), navigate with JSONPath, validate selectively — no schema required upfront.
• ETL and data pipelines: transform nested structures with walk/unwalk/merge, track changes with diff/diffed.
• Typed events and messages: a modict subclass with extra="forbid" and required=True fields behaves like a TypedDict with runtime enforcement — and is still a plain dict you can pass directly to any serializer or bus.
• Prototyping: start free-form, progressively add hints/validators/computed as your data shape stabilizes.

Contents

• Installation
• Quick Start
• Path-Based Tools
• Core Concepts
• Field Definition
• Factories
• Validators
• Computed Fields
• Validation Pipeline
• Configuration (Deep Dive)
• Type Checking & Coercion
• Serialization
• Deep Conversion & Deep Ops
• Package Tour (Internal Modules)
• Public API Reference
• Development
• Contributing
• License

Installation

pip install modict

Requirements:

• Python 3.10+
• JSONPath support relies on jsonpath-ng (the only external dependency)

Quick Start

Use it as a dict (default)

from modict import modict

m = modict({"user": {"name": "Alice"}, "count": 1})

assert m["count"] == 1
assert m.user.name == "Alice"       # attribute access for keys

m.extra = 123                       # extra keys are allowed by default
assert isinstance(m, dict)          # True: modict is a real dict

Define a typed modict class with defaults

from modict import modict

class User(modict):
    name: str            # annotation-only: not required, but validated/coerced if provided
    age: int = 25        # default value
    country: str = "FR"  # another default (not passed at init)

u = User({"name": "Alice", "age": "30"})
assert u.age == 30                  # best-effort coercion unless strict=True
assert u.country == "FR"

Tip: Annotated fields without defaults are not required by default; they provide type validation/coercion when the key is present. Plain annotations and class defaults are enough for most fields. Use modict.field(...) when you need more control (required flag, explicit hint).

repr(modict) shows the live user-facing view of the structure. Computed fields are evaluated for display and rendered as Computed(current_value) so they stay identifiable in console output.

Path-Based Tools

modict comes with a consistent path system for reading/writing nested structures (including inside lists), plus a Path object for disambiguation and introspection.

Supported path formats

You can target nested values using:

• JSONPath strings (RFC 9535): $.users[0].name
• Tuples of keys/indices: ("users", 0, "name")
• Path objects: Path("$.users[0].name")

The Path object

Path is a parsed, strongly-typed representation of a nested path.

from modict import Path

p = Path("$.users[0].name")
assert tuple(p) == ("users", 0, "name")
assert str(p) == "$.users[0].name"      # __str__ renders back to JSONPath
assert repr(p) == "Path($.users[0].name)"

assert p.starts_with(("users", 0))
assert p.relative_to(("users",)) == Path((0, "name"))

bound = p.with_root({"users": [{"name": "Alice"}]})
assert bound.resolve() == "Alice"

Internally, a Path is a tuple of PathNode components. Each node carries:

• the key/index ("users", 0, "name")
• an optional reference to the origin container instance when it can be inferred, which lets helpers distinguish Mapping vs Sequence structure during reconstruction.

All nested helpers accept JSONPath strings, tuples, or Path objects interchangeably.

Nested operations

from modict import modict

m = modict({"user": {"name": "Alice"}})

m.get_nested("$.user.name")           # "Alice"
m.set_nested("$.user.age", 30)
m.has_nested("$.user.age")            # True
m.pop_nested("$.user.age")            # 30
m.del_nested("$.user.name")           # removes the key

m.set_nested(
    "$.prefs.theme",
    "dark",
    create_missing=True,
    container_factory=lambda path: {},
)

Paths in deep traversal

• walk() yields (Path, value) leaf pairs.
• walked() returns a {Path: value} mapping.
• unwalk(walked) reconstructs a nested structure from a {Path: value} mapping using plain dict / list containers.
• unwalk(..., kind_resolver=...) can refine the inferred structure per container path.
• ignore_types=True remains available as a legacy mode to ignore Path hints and use only local key-shape heuristics.
• modict.unwalk(...) then retypes the root mapping through the target class, so model validation/coercion can re-establish the desired root type.

Core Concepts

• modict is a real dict: it supports standard dict operations and behaves like a mutable mapping.
• A modict class can declare fields with type annotations and modict.field(...).
• The model-like behavior is controlled by the _config class attribute (a modictConfig dataclass):

class User(modict):
    _config = modict.config(extra="allow")  # see "Configuration (Deep Dive)" for full reference

Field Definition

The recommended public entry-point is modict.field(...):

modict.field(
    default=MISSING,
    hint=None,       # None = use class annotation when provided
    required=False,  # only required when explicitly True
    validators=None, # internal: used by the metaclass when collecting @modict.validator(...)
)

Example (explicit defaults):

from modict import modict, MISSING

class User(modict):
    name: str = modict.field(default=MISSING, required=True)
    age = modict.field(default=25, hint=int)

u = User({"name": "Alice", "age": "30"})
assert u.age == 30

Note: if you pass hint=... explicitly in modict.field(...), it takes precedence over the class annotation.

Factories

Use modict.factory(callable) to define dynamic defaults (a fresh value per instance), similar to Pydantic's default_factory. This is essential for mutable defaults like lists or dicts — without it, all instances would share the same object.

from modict import modict

class User(modict):
    name: str
    tags: list[str] = modict.factory(list)  # new list per instance

u1 = User(name="Alice")
u2 = User(name="Bob")
u1.tags.append("python")
assert u2.tags == []

Validators

Field validators

Use @modict.validator(field_name, mode="before"|"after") to validate and/or transform a single field value.

• mode="before" (default): runs before coercion and type checking — receives the raw input value. This is the right place for type-changing transforms.
• mode="after": runs after coercion and type checking — receives the already-typed value. Should return a value of the same type (or raise). The type is not re-checked after this step, so returning a different type silently bypasses the hint.

from modict import modict

class User(modict):
    email: str

    @modict.validator("email")
    def normalize_email(self, value):
        return value.strip().lower()

u = User(email="  [e-mail] ")
assert u.email == "[e-mail]"

Model validators (cross-field invariants)

Use @modict.model_validator(mode="before"|"after") for checks that span multiple fields.

In mode="after", the instance is already fully populated and all field validators have run. The method receives (self, values) where values is the final dict of field values.

from modict import modict

class Range(modict):
    start: int
    end: int

    @modict.model_validator(mode="after")
    def check_order(self, values):
        if values["start"] > values["end"]:
            raise ValueError("start must be <= end")

Computed Fields

Computed fields are virtual values evaluated on access. They are stored as Computed objects inside the dict and evaluated via __getitem__.

from modict import modict

class Calc(modict):
    a: int
    b: int

    @modict.computed(cache=True, deps=["a", "b"])
    def sum(self) -> int:
        return self.a + self.b

c = Calc(a=1, b=2)
assert c.sum == 3
c.a = 10
assert c.sum == 12  # cache invalidated because "a" changed

Inline (non-decorator) form:

class Calc(modict):
    a: int
    b: int
    sum = modict.computed(lambda m: m.a + m.b, cache=True, deps=["a", "b"])

Inline "dict value" form (no subclass):

m = modict({"a": 1, "b": 2})
m["sum"] = modict.computed(lambda m: m.a + m.b)
assert m.sum == 3

Notes:

• Computed values still go through the validation pipeline (type checks, JSON serializability check, …) when enabled.
• Cache invalidation semantics:
  • deps=None (default): invalidate on any key change.
  • deps=[...]: invalidate only when one of the listed keys changes (can include other computed names).
  • deps=[]: never invalidate automatically.

Manual invalidation:

m = modict({"a": 1, "b": 2})
m["sum"] = modict.computed(lambda m: m.a + m.b, cache=True, deps=[])

_ = m.sum  # cached
m.a = 10   # deps=[] → no auto-invalidation
assert m.sum == 3

m.invalidate_computed("sum")
assert m.sum == 12

# or invalidate all at once:
m.invalidate_computed()

Validation Pipeline

The pipeline is controlled by _config.check_values:

• check_values="auto" (default): enabled when the class looks model-like (has hints, validators, or relevant config).
• check_values=True: always enabled.
• check_values=False: bypassed (pure dict behavior).

Related config flags:

• strict: when True, no coercion is attempted — values must already match the declared type.
• validate_assignment: when True (default), every assignment goes through the full pipeline (coercion, type check, validators). Set to False to only run validation at init time.
• enforce_json: when True, values must be JSON-serializable after processing.

When enabled, the pipeline runs:

• eagerly at initialization (__init__ → validate())
• on each assignment when validate_assignment=True
• on reads of computed fields (__getitem__ evaluates and then validates the returned value)

Order of operations for a field value:

1. field validators in mode="before" (receive raw input)
2. coercion (only when strict=False)
3. type check against hint (if the field has a type annotation)
4. field validators in mode="after" (receive coerced, typed value)
5. JSON-serializability check (enforce_json=True, with optional encoders)

If any step raises, the whole assignment is rejected.

Configuration (Deep Dive)

All model-like behavior is controlled by the class attribute _config, a modictConfig dataclass created via modict.config(...). Only modict-supported options are accepted — unknown keys raise TypeError.

class User(modict):
    _config = modict.config(
        check_values="auto",
        extra="allow",
        strict=False,
        validate_assignment=True,  # default
        auto_convert=True,
    )

Config reference

• check_values: True/False/"auto".
  • "auto" enables the pipeline when the class looks model-like: it has hints, validators, model validators, or config constraints (e.g. extra != "allow", enforce_json=True, strict=True, …).
• check_keys: True/False/"auto".
  • Key-level constraints are structural checks (presence/allowed-keys/invariants), separate from value validation.
  • "auto" enables key constraints when the model declares them (e.g. extra != "allow", require_all=True, computed fields, or any field with required=True).
  • When False, modict behaves more like a plain dict regarding keys: required=True, require_all=True, extra="forbid"/"ignore", and computed overwrite/delete protection are all skipped.
  • frozen=True is always enforced regardless of check_keys.

Example: keep structure strict but skip value coercion/type checking:

class Msg(modict):
    _config = modict.config(check_values=False, check_keys=True, extra="forbid")
    role: str = modict.field(required=True)
    content: str = modict.field(required=True)

• extra: "allow" (default) / "forbid" / "ignore".
  • "forbid" raises on unknown keys at init and on assignment.
  • "ignore" drops unknown keys silently.
• strict: when True, disables coercion (type checking still applies when hints exist).
• validate_assignment: when True (default), every assignment re-runs the full pipeline. Set to False to only validate at init.
• frozen: when True, __setitem__ / __delitem__ raise — effectively read-only instances.
• auto_convert: when True (default), values stored in nested mutable containers are lazily upgraded on access:
  • nested plain dict → modict (plain modict, not your subclass)
  • applies recursively inside lists, tuples, sets, and dicts as you touch them.
• enforce_json: when True, values must be JSON-serializable after the pipeline runs.
  • allow_inf_nan: controls whether NaN/Infinity pass the JSON check (default: True).
  • json_encoders: a {type: callable} mapping used as fallback encoders by dumps()/dump().
• validate_default: when True, default field values are type-checked at class creation time (skips Factory/Computed).
• from_attributes: when True, MyModict(obj) can read declared fields from obj.field attributes (when obj is not a mapping).
• override_computed: when False (default), computed fields are protected at runtime: you cannot overwrite or delete them on an existing instance. During model construction/casting, class-declared computed fields still win over incoming values so the target model contract is preserved.
• require_all: when True, all declared class fields must be present at initialization; annotation-only fields become required and cannot be deleted.
• evaluate_computed: when True (default), computed fields are evaluated on access. When False, the Computed object itself is returned (pure storage mode, no evaluation).

Required vs defaults (dict-first semantics):

• A class default (e.g. age: int = 25) is an initializer: injected once at construction if missing, but still removable later when require_all=False.
• A field is an invariant only when you opt in: set required=True on the field (or require_all=True on the model) to enforce presence.

Performance / dict-like mode

If you want modict to behave as close as possible to a plain dict (minimal overhead), opt out of most advanced features:

class Fast(modict):
    _config = modict.config(
        check_values=False,      # skip validation/coercion pipeline
        check_keys=False,        # skip structural key constraints (required/extra/...)
        auto_convert=False,      # skip lazy conversion of nested containers on access
        evaluate_computed=False  # treat Computed as raw stored objects
    )

Config inheritance / merging

modict merges configs across inheritance in a Pydantic-like way:

• config values explicitly set in a subclass override inherited values
• when using multiple inheritance, the left-most base wins (for explicitly-set config keys)

class Base(modict):
    _config = modict.config(extra="forbid")
    x: int

class Child(Base):
    y: int = 0
    # inherits extra="forbid"

class Override(Child):
    _config = modict.config(extra="allow")
    # extra is now "allow", rest inherited from Base/Child

Type Checking & Coercion

modict relies on its internal runtime type system (modict/typechecker/) for:

• type checking against annotations (check_type(hint, value))
• best-effort coercion (coerce(value, hint)) when strict=False
• the @typechecked decorator for runtime checking of function arguments/return values

This subsystem supports common typing constructs (Union, Optional, list[str], dict[str, int], tuple[T, ...], ABCs from collections.abc, …).

When coercion fails, the original value is kept; the subsequent type check then decides whether it's accepted (based on hints and strict mode). This means strict=False is permissive but not silent — type mismatches still raise if the hint doesn't match.

Serialization

modict uses a JSON-like API backed by json.dumps/json.dump/json.loads/json.load.

dumps / dump (output)

Serialize a modict instance to a JSON string or file. Both methods accept:

• exclude_none: drop keys with None values
• encoders: {type: callable} mapping for custom serialization (overrides json_encoders from config)

from datetime import datetime
from modict import modict

class Event(modict):
    _config = modict.config(json_encoders={datetime: lambda d: d.isoformat()})
    name: str
    ts: datetime

e = Event(name="launch", ts=datetime(2024, 1, 1))
print(e.dumps())
# {"name": "launch", "ts": "2024-01-01T00:00:00"}

json_encoders in _config serves as the default encoder table for all dumps()/dump() calls on that class. You can override it per-call by passing encoders=... directly.

e.dumps(encoders={datetime: lambda d: d.timestamp()})
# {"name": "launch", "ts": 1704067200.0}

dump() writes to a file path or file-like object:

e.dump("event.json")

loads / load (input)

Class-level methods that parse JSON and return a modict instance:

m = modict.loads('{"name": "launch", "ts": "2024-01-01T00:00:00"}')
m = modict.load("event.json")

These are thin wrappers around json.loads/json.load — no custom deserialization logic is applied. For typed deserialization with coercion, construct from the parsed dict directly:

data = modict.loads('{"name": "launch", "ts": "2024-01-01T00:00:00"}')
event = Event(**data)  # pipeline runs: coercion, type checks, validators, ...

to_jsonable

For serialization beyond JSON (e.g. YAML, MessagePack), to_jsonable(obj, encoders) from modict.collections_utils recursively converts a structure to plain JSON-safe types (dicts, lists, primitives). This is what dumps/dump use internally.

Deep Conversion & Deep Ops

Deep conversion

modict ships with conversion utilities that preserve container identity as much as possible:

• modict.convert(obj): recursively upgrades dict nodes to modict (and walks into mutable containers).
  • The root dict becomes your class; nested dicts become plain modict unless they were already instances.
  • recurse=False stops recursion when reaching a modict node (used internally for lazy auto_convert).
• m.to_modict(): deep conversion of an instance in-place (calls convert(self)).
• m.to_dict(): deep un-conversion back to plain containers.

Deep operations on nested structures

• walk() / walked(): flatten a nested structure to (Path, value) pairs.
• unwalk(walked, *, kind_resolver=None): reconstruct a nested structure from a {Path: value} mapping using structural dict / list containers, with an optional hook to refine inferred mapping / sequence kinds per path. The root can then be recast through modict.unwalk(...).
• merge(mapping): deep, in-place merge (mappings merge by key; sequences merge by index). Returns None — modifies in place.
• diff(mapping): deep diff — returns {Path: (left, right)} with MISSING for absent values.
• diffed(mapping): minimal nested patch — returns a plain modict containing only the changes needed so that self.merge(self.diffed(other)) equals other. Keys removed in other are set to MISSING.
• deep_equals(mapping): deep equality by comparing walked representations (container types are ignored — a modict and a plain dict with the same leaves are equal). Use this when cross-type equality is needed; == uses the native dict comparison (type-sensitive).

Package Tour (Internal Modules)

This section is an overview of the main internal modules and what functionality they implement. If you only need the user-facing API, skip to Public API Reference.

modict/core/ (the modict class)

Core behaviors implemented here:

• dict subclass with attribute access (m.key ↔ m["key"]) while keeping Python attributes working
• validation pipeline (validate(), assignment validation, extra handling)
• computed fields evaluation and dependency-based cache invalidation
• lazy nested conversion (auto_convert) implemented in __getitem__
• nested ops (get_nested, set_nested, pop_nested, …) backed by JSONPath/Path
• deep ops (walk, walked, unwalk, merge, update, diff, diffed, deep_equals, deepcopy)
• JSON helpers (loads, load, dumps, dump)
• modictMeta: collects declared fields from __annotations__ and assigned attributes
  • supports plain defaults, modict.field(...) (Field), modict.factory(...) (Factory), and @modict.computed(...) (Computed)
  • collects @modict.validator(...) into per-field validators
  • collects @modict.model_validator(...) into model-level validators
• modictConfig: configuration object with explicit-key tracking and inheritance merge semantics
• modictKeysView / modictValuesView / modictItemsView: dict views that read through __getitem__ (so computed fields, lazy conversion, and validation all apply during iteration)

modict/collections_utils/ (nested structure utilities)

This package is responsible for paths, nested operations, and deep traversal. See modict/collections_utils/README.md and modict/path_utils/README.md for more details.

• _path.py: Path / PathNode — JSONPath (RFC 9535) parsing and formatting via jsonpath-ng.
  • Path components can cache origin container references so walk() → unwalk() can distinguish Mapping vs Sequence structure without recreating arbitrary concrete container classes.
  • Path(...) accepts JSONPath strings, tuples/lists of keys, or another Path.
• _basic.py: container-agnostic get_key / set_key / has_key / keys / unroll.
• _advanced.py: get_nested / set_nested / pop_nested / del_nested / has_nested, walk / walked / unwalk, deep_merge / diff_nested / deep_equals.
• _view.py: View — base class for custom collection views over mappings or sequences.
• _missing.py: MISSING sentinel to distinguish "absent" from None.

modict/typechecker/ (runtime typing + coercion)

See modict/typechecker/README.md for more details.

• TypeChecker: checks values against typing hints and collection ABCs.
• Coercer: best-effort conversions for common hints/containers.
• Convenience API: check_type, coerce, can_coerce, and typechecked, coerced for decorator-based type checking and coercion on functions.

modict/model_api/ (field system)

• Field, Factory, Computed: field descriptor types used by modictMeta during class creation.
• Validator, ModelValidator: signature adapters for common call styles (allow flexible validator signatures).
• build_fields_and_model_validators: metaclass helper that collects fields and validators from a class dict.

Public API Reference

This section lists the public symbols exported by modict and the main methods on modict instances.

Exports

From from modict import ...:

• Data structure:
  • modict
• Field system:
  • Field (advanced; most users should prefer modict.field(...))
  • Factory (advanced; most users should prefer modict.factory(...))
  • Computed (advanced; most users should prefer @modict.computed(...))
  • Validator, ModelValidator (advanced; decorators are the typical entry-point)
• Config:
  • modictConfig (usually created via modict.config(...))
• JSONPath types:
  • Path, PathNode
• Sentinel:
  • MISSING
• Search:
  • Query(path=MISSING, value=MISSING) — combined path + value constraint; .find(root) returns matching (Path, value) pairs. MISSING means "no constraint"; None as value matches leaves whose value is literally None
• Type checking / coercion:
  • check_type(hint, value)
  • coerce(value, hint)
  • can_coerce(value, hint)
  • typechecked (decorator)
  • TypeChecker
  • Coercer
  • Exceptions: TypeCheckError, TypeCheckException, TypeCheckFailureError, TypeMismatchError, CoercionError

modict class methods

• modict.config(**kwargs) -> modictConfig
• modict.field(...) -> Field
• modict.factory(callable) -> Factory
• @modict.validator(field_name, mode="before"|"after")
• @modict.model_validator(mode="before"|"after")
• @modict.computed(cache=False, deps=None)
• JSON helpers:
  • modict.loads(s, **json_kwargs) -> modict
  • modict.load(fp_or_path, **json_kwargs) -> modict
• Conversion:
  • modict.convert(obj, seen=None) -> Any
  • modict.unconvert(obj, seen=None) -> Any
  • modict.unwalk(walked: dict[Path, Any], ignore_types: bool = False, *, kind_resolver=None) -> Any

modict instance methods

Instance methods keep standard dict behavior, plus:

• Validation:
  • validate()
• Conversion:
  • to_modict() -> modict (deep conversion)
  • to_dict() -> dict (deep unconvert)
• Serialization:
  • dumps(exclude_none=False, encoders=None, **json_kwargs) -> str
  • dump(fp_or_path, exclude_none=False, encoders=None, **json_kwargs) -> None
• Nested operations (JSONPath / tuple / Path):
  • get_nested(path, default=MISSING)
  • set_nested(path, value, *, create_missing=False, container_factory=None) where container_factory is called as factory(path)
  • del_nested(path)
  • pop_nested(path, default=MISSING)
  • has_nested(path) -> bool
• Key operations:
  • translate(mapping_or_kwargs) -> modict (returns a plain translated modict)
  • exclude(*keys) -> modict
  • extract(*keys) -> modict
  • find(query=MISSING, *, path_constraint=MISSING, value_constraint=MISSING) -> Generator — lazily yields (Path, value) pairs matching a Query or inline constraints (deep)
  • found(query=MISSING, *, path_constraint=MISSING, value_constraint=MISSING) -> modict — same, returned as a {Path: value} modict
• Deep operations:
  • merge(mapping) -> None (in-place)
  • update(other=(), /, **kwargs) -> None — like dict.update() but routes through validation
  • diff(mapping) -> dict[Path, tuple]
  • diffed(mapping) -> modict — minimal nested patch; self.merge(self.diffed(other)) equals other
  • deep_equals(mapping) -> bool
  • deepcopy() -> modict
• Walking:
  • walk(callback=None, filter=None, excluded=None) -> Iterable[tuple[Path, Any]]
  • walked(callback=None, filter=None) -> dict[Path, Any]
• Computed cache:
  • invalidate_computed(*names) -> None (no args = all)

Development

python3 -m pytest -q

Contributing

Contributions are welcome.

• Please open an issue to discuss larger changes.
• For pull requests: add/adjust tests under tests/ and keep python3 -m pytest -q green.
• Local setup: pip install -e ".[dev]".

See CONTRIBUTING.md for details.

License

MIT. See LICENSE.