ddigraph

A modern Python toolkit that transforms DDI (Data Documentation Initiative) XML metadata into knowledge graphs. Supports DDI Codebook and DDI-L FragmentInstance formats with streaming parsing, batched writes, and full async I/O across multiple graph backends.

Documentation | Getting Started | PyPI | Source Code

Features

• Neo4j and RDF -- load into Neo4j, or read and write RDF with a documented, versioned vocabulary aligned to DISCO, SKOS and XKOS
• File export -- ddigraph export writes Turtle, JSON-LD, N-Triples, RDF/XML, JSON or CSV with no database involved
• SHACL shapes -- generated from the same schema that builds the Neo4j constraints, so they cannot drift
• Preview before you load -- ddigraph preview summarises a file as text, Mermaid, or one self-contained HTML page
• Streaming XML processing -- Memory-bounded iterparse for files of any size
• Batched writes -- UNWIND-based Cypher for 10-100x fewer database round trips
• Async I/O -- Concurrent parsing and writing with back-pressure control
• Format auto-detection -- identifies DDI Codebook, DDI-L and DDI-CDI input
• Unified schema -- Single source of truth for all node and relationship definitions
• Adapter pattern -- Plug in custom graph backends via GraphWriteAdapter protocol
• Production-ready -- Retry logic, observability hooks, pydantic-based configuration

Quick Start

Install

pip install ddigraph

Load DDI metadata (CLI)

# Set Neo4j connection
export DDIGRAPH_NEO4J_URI=bolt://localhost:7687
export DDIGRAPH_NEO4J_USER=neo4j
export DDIGRAPH_NEO4J_PASSWORD=secret

# Bootstrap schema and load data (format is auto-detected)
ddigraph bootstrap
ddigraph load survey.xml --dataset-id my-survey

Load DDI metadata (Python)

import ddigraph

ddigraph.bootstrap(target="bolt://localhost:7687")
result = ddigraph.load("survey.xml", target="bolt://localhost:7687")
print(result.nodes_written, "nodes,", result.relationships_written, "relationships")

Export without a database

import ddigraph

result = ddigraph.export("survey.xml", "survey.ttl", format="turtle")
print(result.triples, "triples")

Supported Formats

XSD Coverage

ddigraph ships with 100 % coverage of every concrete identifiable element declared in the bundled XSD schemas (schemas/). Coverage is enforced by the audit script and a pytest guardrail so new schema releases surface any gaps:

Run python scripts/xsd_coverage.py to regenerate the audit or python scripts/xsd_coverage.py --json for machine-readable output.

Supported Backends

demo/ scripts ship in neither the wheel nor the source distribution. They are worked examples of driving another store from the parser tier, not supported backends. To build your own, use ddigraph.iter_graph(), which yields backend-neutral nodes and relationships for any DDI flavor.

Docker Quick Start

docker run --rm --name neo4j-demo \
  -p 7474:7474 -p 7687:7687 \
  -e NEO4J_AUTH=neo4j/password \
  neo4j:5

export DDIGRAPH_NEO4J_URI=bolt://localhost:7687
export DDIGRAPH_NEO4J_USER=neo4j
export DDIGRAPH_NEO4J_PASSWORD=password

ddigraph bootstrap
ddigraph load your-file.xml --dataset-id demo

Documentation

Full documentation is available at pbisson44.github.io/ddigraph in English and French.

• Getting Started -- Installation, quick start, 10-minute tutorial
• User Guide -- Architecture, DDI formats, relationships, adapters
• Graph Backends -- Neo4j, RDF/SPARQL, Gremlin, NetworkX
• Reference -- CLI commands, configuration
• Advanced -- Performance tuning, AI readiness, standards interoperability
• Contributing -- How to contribute

Development

git clone https://github.com/pbisson44/ddigraph.git
cd ddigraph
pip install -e ".[dev,docs]"

ruff check . && ruff format .
mypy .
pytest
mkdocs serve

License

MIT -- see LICENSE for details.