Nadata

Nadata is a declarative data schema and object-graph framework with its own language. You describe your data model in .nastruct schema files, populate it in .nadata data files, and get a typed, queryable, persistent object graph — usable from the command line or as an embedded Python library.

# structs.nastruct — the model
class Human {
    name: Str;
    birthday: DateTime = DateTime();

    @age(now: DateTime = DateTime.now()) -> DateTime {
        return now - self.birthday;
    }
}

# data.nadata — the data
tom = Human(name="Tom", birthday=DateTime(d=12, m=4, y=2003));

print("Tom is an adult:", tom.age(DateTime.now()).year >= 18);
Array[Human] adults = Human.where((h: Human) -> h.age(DateTime.now()).year >= 18);

Features

• Typed schema language — structs, inheritance, attributes with defaults, const bind-once attributes, methods and computed properties, standalone functions; import "..."; shares schema libraries across projects
• String library and hash indexes — typed string members (split, join, replace, contains, ...), and opt-in per-attribute hash indexes (index(Human, "email") + get_by(Human, "email", x)) with automatic insert / update / removal maintenance — O(1) equality lookups instead of full scans
• Rich type system — sized integers/floats (Int32, UFloat16, ...), Str, Bool, DateTime, Color, Array[T], Dict[K, V], Set[T], Queue[T] / PriorityQueue[T, P], sparse spatial grids (GridSquare2D[T], GridHex2D[T], GridCube3D[T]), shape-checked Tensor[T, d1, ...] with symbolic dims bound to const attributes, entity references; indexing (arr[0], dict["k"]) and string concatenation
• Error handling — raise "message"; and try { ... } catch (err) { ... } in scripts and method bodies; runtime errors (indexing, I/O builtins) are catchable too, while return / break / continue pass through
• Declarative data files — forward and circular references, script statements, print, chained polymorphic queries
• Runtime introspection — elt.id, type(x), elt.list_attr() / list_methods(), schema-constrained getattr/setattr, and database-wide schema_classes() / schema_functions()
• Persistence — whole-database dumps, dirty-only delta appends, and removal tombstones to a single .ndb file
• Bounded memory — disk-backed lazy rows (~50 bytes/row index), a bounded LRU decode cache with clean-only eviction, pin/unpin, flush/clear_cache, and remove_from_db from scripts — stream millions of rows while the model stays hot
• Autograd and training — tape-based reverse-mode differentiation on Float tensors (arithmetic, matmul, reductions, shape ops, gather, slices, relu), seeded data partitioning for train/valid/test splits, and built-in SGD — train models end-to-end in .nadata scripts
• Sessions & identity — a project folder is a session: #init seeds once per world, #auto syncs every open, #manual scripts take parameters (#manual(lr: Float = 0.05)), and nid = "name" gives instances stable, updatable identity (get_by_nid, nids, first/last) — schema files re-merge on every open, and schema evolution is reconciled against persisted rows (removed attributes are dropped; type changes are reported, never fatal). I/O builtins (read_json, write_csv, file_hash, ...) resolve against the project root, with get_setting/set_setting freshness markers
• Interactive console — nadata console <path> REPL over any loaded database, with persistent state and expression echoing
• Python API — load, query, mutate, execute raw source, persist
• Zero dependencies — pure Python 3.10+

Installation

git clone <repository-url>
cd Nadata
pip install -e .
pip install -e .[dev]   # optional: pytest + mypy for development

Quick start

# Run a project (schema + data) and execute its scripts
nadata run examples/ex0_simple

# Print a summary of a project or a saved .ndb database
nadata info examples/ex0_simple
nadata info store.ndb

# Persist a project to a single database file
nadata dump examples/ex0_simple store.ndb

# Explore it interactively (works on folders and .ndb files)
nadata console examples/ex0_simple
nadata console store.ndb

Or from Python:

import nadata

app = nadata.Nadata.load("./examples/ex0_simple")

print(app.get_attribute("tom", "birthplace"))   # Paris
print(app.count("Human"))                        # 3

app.dump("store.ndb")                            # persist

Documentation

Examples

The examples/ folder contains self-contained projects, from basics to edge cases:

Editor support

Syntax highlighting for .nastruct and .nadata files ships as a VS Code / VSCodium extension in editors/vscode-nadata. .vsix packages are not stored in git — build one and install it:

cd editors/vscode-nadata
npx -y @vscode/vsce package --allow-missing-repository   # -> nadata-support-<version>.vsix
code --install-extension nadata-support-<version>.vsix

Packaged releases attach the current .vsix to the matching git tag.

A sync test (tests/test_grammar.py) keeps the grammar aligned with the language: it fails whenever a builtin type, keyword, or scope is missing.

Development

python -m pytest tests/ -q        # run the test suite
python -m mypy nadata/               # type check (strict)
python -m scripts.inspect_parse examples/ex0_simple   # inspect the parse IR

License

MIT — see LICENSE.