┌─────────────────┐                                        ┌──────────────────┐
  │     Input       │                                        │     Output       │
  │                 │                                        │                  │
  │  XMI 2.1        │         ┌──────────────────┐           │  EA XMI          │
  │  EA XMI         │────────▶│   crunch_uml     │──────────▶│  JSON            │
  │  QEA (EA native)│         │                  │           │  Markdown        │
  │  JSON           │         │  Normaliseer     │           │  Excel           │
  │  Excel          │         │  Transformeer    │           │  RDF / TTL       │
  │  CSV            │         │  Valideer        │           │  JSON-LD         │
  │  i18n           │         └────────┬─────────┘           │  JSON Schema     │
  │                 │                  │                     │  OpenAPI         │
  └─────────────────┘                  │                     │  SQLAlchemy code │
                              ┌────────┴─────────┐           │  EA Repository   │
                              │  crunch_uml.db   │           │  Schema Diff     │
                              │  SQLite / PG     │           │  CSV / i18n      │
                              └──────────────────┘           └──────────────────┘

Key Features

• Multi-format import — Reads XMI 2.1, Enterprise Architect XMI, QEA, JSON, Excel, CSV and i18n files
• Normalised storage — Stores all UML entities (Package, Class, Attribute, Association, Generalization, Enumeration, Diagram) in a SQLAlchemy database
• Diagrams with layout — Diagram membership and geometry (positions, sizes, z-order, edge waypoints) are read from EA XMI and QEA files, stored in a canonical coordinate system, and written back out — including a full XMI round-trip and EA repository layout updates
• Multi-schema support — Load different versions or translations of the same model into separate schemas within one database, then compare or merge them
• 23 export formats — EA XMI, JSON, CSV, Excel, Markdown, Jinja2 templates, RDF/TTL/JSON-LD, JSON Schema, OpenAPI, SQLAlchemy code, EA Repository update, schema diff, and more
• Plugin architecture — Add custom parsers, renderers or transformers without changing core code
• Pipeline approach — Import, transform and export are independent steps that can be combined into automated workflows
• Upsert support — Import different datasets or incremental changes into the same database
• Safe database evolution — Every database carries a datamodel version; compatible upgrades are migrated in place (additive), incompatible ones recreate the database with a clear warning

Documentation

Documentation is avalailable on GitHub Pages

Install and Usage

Install using pip:

pip install crunch-uml

Quick start:

# Import an Enterprise Architect XMI file into a new database
crunch_uml import -f model.xmi -t eaxmi -db_create

# Copy a sub-model to a named schema
crunch_uml transform -ttp copy -sch_to my_model -rt_pkg EAPK_12345

# Export to Excel
crunch_uml -sch my_model export -t xlsx -f model.xlsx

Or clone the repository:

git clone https://github.com/brienen/crunch_uml.git
cd crunch_uml
pip install -e .

General Options

crunch_uml [-h] [-v] [-d] [-w] [-db_url DATABASE_URL] [-sch SCHEMA] {import,transform,export} ...

Commands

import

Import data into the Crunch UML database.

crunch_uml import -f <file> -t <type> [options]

Supported Input Types:

Supported tables: packages, classes, attributes, enumerations, enumerationliterals, associations, generalizations, diagrams and the diagram junction tables (diagram_class, diagram_enumeration, diagram_association, diagram_generalization) that carry membership and layout.

transform

Transform a model from one schema to another within the database.

crunch_uml transform -ttp <type> -sch_to <target_schema> [options]

export

Export data from the Crunch UML database.

crunch_uml [-sch SCHEMA] export -t <type> -f <file> [options]

Supported Export Types:

detect

Classify a model file by its content, never by its extension. Reads at most 64 KiB (plus the last 64 KiB of an XML file, and a prolog that has not ended there up to 1 MiB, so a DOCTYPE behind a very long comment is still seen), opens SQLite read-only, and prints one JSON line.

crunch_uml detect -f model.qea
# {"verdict": "qea", "format": "qea", "accepted": true, "code": null, "size": 51961856, ...}

Accepted: eaxmi (EA-XMI 2.1 from Enterprise Architect with its extension block), qea (EA repository in rollback-journal mode with the ten t_* tables and no views or triggers) and artifact (a row artifact written by pack). Anything else gets a verdict and a refusal code (file_type_unknown, xmi_not_ea, xml_malformed, xml_forbidden, qea_unreadable). Exit code 0 when accepted, 2 when refused.

pack

Parse an Enterprise Architect model into a row artifact (.cua.gz): gzip-compressed JSON with every crunch_uml table, readable with nothing but a standard library.

crunch_uml pack -f model.qea -o model.cua.gz        # format as detected
crunch_uml pack -f model.xml -t eaxmi -o model.cua.gz

pack runs detect, parses into a fresh temporary SQLite and writes the artifact: a header (format = semtk-crunch-artifact, format_version, datamodel_version, producer, source with format, exporter, SHA-256 and size but never the file name, capabilities, run) and tables with columns and rows sorted by primary key. It prints one JSON line; exit code 0 on success, 2 with a code for refused or empty input (including model_empty for a model without classes), 1 with parse_failed otherwise.

Multi-Schema: Version Comparison & Translations

One of the most powerful features is multi-schema support. Import the same model multiple times into different schemas to compare versions or create translations:

# Import version 2.0
crunch_uml import -f model_v2.xmi -t eaxmi -db_create
crunch_uml transform -ttp copy -sch_to v2 -rt_pkg ROOT_ID

# Import version 1.0 into a separate schema
crunch_uml -sch v1_raw import -f model_v1.xmi -t eaxmi
crunch_uml transform -ttp copy -sch_from v1_raw -sch_to v1 -rt_pkg ROOT_ID

# Generate a diff report
crunch_uml -sch v1 export -t diff_md -f changes.md \
    --compare_schema_name v2 --compare_title "Changes v1 → v2"

Local LLM translations via Ollama

The i18n exporter ships with three translation backends:

Set up Ollama

# Install Ollama: https://ollama.com
ollama pull mistral-small3.1:24b   # ~14 GB Q4_K_M — recommended default
ollama serve                       # if not already running

Activate

export CRUNCH_UML_TRANSLATE_BACKEND=ollama
export OLLAMA_NUM_PARALLEL=8       # server-side, match the worker count

crunch_uml export -t i18n -f out.json --language en --translate True --from_language nl

If Ollama is unreachable or returns an error the renderer transparently falls back to the translators API, so adding the env-var never breaks an existing pipeline.

Tuning knobs

Every setting is configurable both as an environment variable and as a CLI flag on crunch_uml export. The CLI flag wins when both are set; the env-var wins over the baked-in default.

One-liner with CLI flags

crunch_uml export -t i18n -f out.json --language en --translate True \
    --from_language nl \
    --translate_backend ollama \
    --ollama_model mistral-small3.1:24b \
    --translate_workers 8 \
    --translate_context

Identifier casing

Many model fields are camelCase, PascalCase or snake_case identifiers rather than prose. The prompt instructs the LLM to translate the words and rejoin them in the same style, and a deterministic post-processing step (reconcile_case) repairs any leftover whitespace. So "aanvangAanwezigheid" reliably comes back as "startAttendance" even when the model briefly drops the camelCase shape.

Opaque tokens are never translated

Some values have no meaningful translation and previously made the LLM hallucinate. The Ollama backend short-circuits these patterns: the source is returned verbatim without any model call.

• XML/HTML-like single tags: <memo>, <typing>, </br>, <UML:Class>
• EA-generated identifiers: EAID_…, EAPK_…, EAID_attr_…
• URLs (http://, https://, www.…)
• ISO dates and timestamps (2024-05-13, 2024-05-13T10:30:00Z)
• Pure punctuation / numbers

Model recommendation

Deterministic translation pipeline

The pipeline backend is the recommended route for authoritative, reproducible translations. Official EU/government terminology is the basis; the context from the source model decides the meaning. Same input + same sources + same (logged) model digests → same output. Full design: docs/technisch/vertaalpijplijn.md.

The cascade, per model element (name, definition and other fields translated together in one call, so they stay consistent):

1. i18n file as translation memory — existing translations per (section, GUID, field) are reused, never re-translated. Reviewed translations in the i18n file always win.
2. Termbanks — candidates from IATE (TBX), EuroVoc and any other Linked Open Data source (SKOS/RDFS/OWL, all rdflib serialisations, format auto-detected). An exact hit that is unambiguous after deterministic disambiguation (domain + source definition overlap) is taken without any model, concept URI logged.
3. Local LLMs via Ollama — two workhorse models translate independently with a binding glossary; agreement on the name accepts, disagreement escalates to the heavy model. Definitions are checked deterministically against the glossary instead. Hierarchical order (packages → classes → attributes) feeds each level's fixed name translations into the glossary of the level below.
4. NMT safety net (optional transformers dependency) below the LLM.
5. Online services (Google/Bing) only when explicitly allowed — they are not reproducible.

A preflight establishes what is available before anything is translated and logs a compact overview (resolved model tags + digests + pull dates, loaded termbanks with their version read from the data itself). Missing or outdated resources produce a warning and a skipped layer — never a crash, never silent wrong output. Model names are prefixes: mistral-small resolves to the highest locally installed tag, so minor model upgrades need no config change.

Configure

export CRUNCH_UML_TRANSLATE_BACKEND=pipeline

# Termbanks: comma-separated files or directories; order = priority.
# Formats are auto-detected (.ttl/.rdf/.jsonld/... via rdflib, .tbx as IATE).
export CRUNCH_UML_TERMBANKS="resources/gemma_begrippen.ttl,resources/lod/,resources/IATE_export.tbx"

# LLM roles: first = primary workhorse (definitions), second = second vote on names.
export CRUNCH_UML_LLM_WORKHORSES="qwen2.5:14b,mistral-small3.1:24b"
export CRUNCH_UML_LLM_HEAVY="qwen2.5:32b"          # escalation model (largest you can run)

crunch_uml export -t i18n -f out.json --language en --translate True --from_language nl

All pipeline settings (env-var, with an equivalent CLI flag on crunch_uml export):

Getting the termbanks

• IATE — download the TBX export from iate.europa.eu/download-iate and point CRUNCH_UML_TERMBANKS at the .tbx file. The release date in the header is reported by the preflight.
• EuroVoc — download the SKOS distribution from op.europa.eu (any RDF serialisation works) and add the file or its directory to the list.
• Own vocabularies — any SKOS/RDFS/OWL file (e.g. the GEMMA-begrippenkader as Turtle) drops in without code changes; labels, definitions and domains are queried generically.

Degradation behaviour

Development

# Get a comprehensive list of development tools
make help

Documentation

Full documentation (including architecture, examples and technical design) is available via MkDocs:

pip install mkdocs-material mkdocs-glightbox
mkdocs serve

Future Improvements

• Caching & validation engine for large models
• Streaming/chunked parsing for large files
• REST API interface (FastAPI)
• AI-based semantic indexing
• Snowflake and Azure database connectors

License

MIT