OLDAP tools

OLDAP tools is a CLI tool for managing parts of the OLDAP framework. It allows to

• dump all the data of a given project to a gzipped TriG file
• load a project from a gzipped TriG file created by oldap-tools
• load a hierarchical list from a YAML file
• dump a hierarchical list to a YAML file
• validate and add manually defined archive trees from YAML
• validate versioned instance-data YAML/JSON locally and preflight it against a live OLDAP model

Installation

The installation is done using pip: pip install oldap-tools

Documentation

Structured documentation is available in docs/:

• Installation and Connection
• Command Reference
• Ontology YAML
• Archive Structure YAML
• Instance Data YAML/JSON

The first real resumable media batch is examples/data/chama-photographs-batch-01.yaml: three Chama HEIC photographs, their owner annotations, and the reusable resources they reference.

• Hierarchical Lists
• Update Semantics
• Operations and Backups

Usage

The CLI tool provides the following commands:

• oldap-tools project dump: Dump all the data of a given project to a gzipped TriG file
• oldap-tools project load: Load a project from a gzipped TriG file created by oldap-tools
• oldap-tools lists dump: Dump a hierarchical list to a YAML file
• oldap-tools lists load: Load a hierarchical list from a YAML file
• oldap-tools ontology validate: Validate an ontology datamodel YAML file
• oldap-tools ontology load: Load or update an ontology datamodel from YAML
• oldap-tools ontology dump: Dump an ontology datamodel to YAML or TriG
• oldap-tools archive validate: Validate a manually defined archive structure YAML file
• oldap-tools archive load: Add a YAML-defined archive structure to an existing project
• oldap-tools data validate: Validate a versioned instance-data YAML/JSON document offline
• oldap-tools data prepare: Replace iri: auto placeholders once with stable UUID-based project-local identities
• oldap-tools data import --dry-run: Check instance data against a live project without writing
• oldap-tools data import --apply: Create one resource and process a declared local IIIF image without updates or overwrites
• oldap-tools data import --apply --batch: Resume a sequential multi-resource metadata-and-media import
• oldap-tools data media-attach: Attach or idempotently verify media for one existing resource
• oldap-tools staging ensure-mobile-folder: Add the application-managed Mobile folder to existing StagingAreas
• oldap-tools fasnacht taxonomy-inventory: Produce a read-only Fasnacht taxonomy and empty-event migration report
• oldap-tools fasnacht taxonomy-migration-plan: Expand the current Fasnacht data references into a digest-bound read-only migration manifest
• oldap-tools fasnacht taxonomy-migration-apply: Apply the reviewed Object/Event/Practice cutover after a full project backup; organisations and event resources remain untouched

Common options

• --graphdb, -g: URL of the GraphDB server (default: "http://localhost:7200")
• --repo, -r: Name of the repository (default: "oldap")
• --user, -u: OLDAP user which performs connected operations
• --password -p: OLDAP password for connected operations
• --graphdb_user: GraphDB user (default: None). Not needed if GraphDB runs without athentification.
• --graphdb_password: GraphDB password (default: None). Not needed if GraphDB runs without athentification.
• --verbose, -v: Print more information

The local ontology validate, archive validate, and data validate commands do not require OLDAP credentials.

Command

Project dump

This command dumps all the data of a given project to a gzipped TriG file. It includes user information of all users associated with the project. The command has the following syntax (in addition to the common options):

oldap-tools [common_options] [graphdb-options] project dump [-out <filename>] [--data | --no-data] [-verbose] <project_id>

The graphdb options see above. The other options are defined as follows:

• -out <filename>: Name of the output file (default: "<project_id>.trig.gz")
• --data | --no-data: Include or exclude the data of the project (default: include)
• -verbose: Print more information
• <project_id>: Project identifier (project shortname)

The file is basically a dump of the project specific named graphs of the GraphDB repository. This are the following graphs:

• <project_id>:shacl: Contains all the SHACL shapes of the project
• <project_id>:onto: Contains all the OWL ontology information of the project
• <project_id>:lists: Contains all the hierarchical lists of the project
• <project_id>:data: Contains all the resources (instances) of the project

The user information is stored as special comment in the TriG file and is interpreted by oldap-tools project load.

Project load

This command loads a project from a gzipped TriG file created by oldap-tools. It has the following syntax (in addition to the common options):

oldap-tools [common_options] [graphdb-options] project load --i <filename>

The options are as follows:

• --inf, -i: Name of the input file (required)
• -verbose: Print more information

If a user does not exist, then the user is created. If the User is already existing, then the user is replaced.

NOTE: This will change in the future in order to only update project specific permissions to the existing user.

List dump

This command dumps a hierarchical list to a YAML file. This file can be edited to add/remove or change list items. The command has the following syntax (in addition to the common options):

oldap-tools [common_options] lists dump [-out <filename>] <project_id> <list_id>

This command generates a YAML file which can be edited and contains the list and all it nodes

The options are as follows:

• -out , -o: Output file
• <project_id>: Project identifier (project shortname)
• <list_id>: List identifier

List load

This command loads a hierarchical list from a YAML file into the given project. The command has the following syntax (in addition to the common options):

oldap-tools [common_options] lists load --inf <filename> <project_id>

If a list already exists, loading is additive: nodes that are present in YAML but missing in the store are inserted, including their subtrees. Existing nodes are never deleted or moved; if the YAML would place an existing node below a different parent, the load aborts with an error.

The options are as follows:

• --inf, -i: Name of the input file (required)
• <project_id>: Project identifier (project shortname)

Ontology validate

This command validates an ontology YAML file against the bundled schema:

oldap-tools [common_options] ontology validate --inf <filename>

Ontology load

This command loads or updates a project datamodel from a YAML file:

oldap-tools [common_options] ontology load --inf <filename> [--mode update|replace] [--connectors create|replace] [--backup|--no-backup] [--backup-out <filename>]

By default, the command makes a TriG gzip backup of the model and list graphs before loading. The replace mode deletes the existing datamodel graphs (<project>:shacl and <project>:onto) and recreates them from YAML. The update mode compares the YAML classes and properties with the current datamodel and lets oldaplib perform the corresponding updates. Set an attribute to null in update mode to delete it, for example label, comment, name, description, min_count, or max_count.

Hierarchical lists can be referenced as external YAML files or defined inline. A property can point to a list node class with to_class: list:<ListId>. Existing lists are extended additively from YAML: missing nodes are inserted, while existing nodes are not deleted or moved.

Lucene connectors can be declared in the same YAML file. They are skipped by default and only applied when --connectors create or --connectors replace is passed. Single-property fields can use a QName directly and get their Lucene field name from the property fragment. Multi-step chains must be given an explicit Lucene field name.

Example:

ontology:
  project:
    shortname: fasnacht
    iri: https://fasnacht.digital
    namespace: http://fasnacht.digital/ns/
    start: 2025-06-01

  lists:
    CreativeCommons: CreativeCommons.yaml

  external_ontologies:
    schema:
      namespace: https://schema.org/
      label: schema.org
      proposedResourceClass:
        - Person
        - Organization

  classes:
    fasnacht:Person:
      label:
        en: Person
        de: Person
      superclass:
        - schema:Person
      properties:
        - iri: schema:familyName
          datatype: xsd:string
          name:
            en: Family name
            de: Nachname
          min_count: 1
          max_count: 1
          order: 1
          editor: TEXT_FIELD

  lucene_connectors:
    fasnacht:
      types:
        - fasnacht:Story
        - fasnacht:ArchiveObject
        - fasnacht:ArchiveMediaObject
      fields:
        - fasnacht:storyContent
        - schema:abstract
        - fasnacht:archiveObjectTitle
        - schema:description
        - fieldName: representedArchiveObjectTitle
          chain:
            - fasnacht:archiveMediaObjectOf
            - fasnacht:archiveObjectTitle

oldap-tools creates one GraphDB Lucene connector per project. The connector name is always the project short name. Multiple entries below lucene_connectors are treated as YAML grouping/specification blocks and are merged into that project connector.

Ontology dump

This command dumps an ontology datamodel either as YAML or as a TriG gzip file containing the <project>:shacl, <project>:onto, and <project>:lists graphs:

oldap-tools [common_options] ontology dump [-out <filename>] [--format yaml|trig] [--include-taxonomies] <project_id>

When dumping YAML, --include-taxonomies writes all project lists as separate <ListId>.yaml files next to the ontology YAML file and adds an ontology.lists block that references them. External ontology references are always included with their namespace, labels, comments, and proposed resource, datatype-property, and object-property names.

Ensure the Staging Mobile folder

This command validates an existing StagingArea and ensures that it contains one Mobile folder directly below its unique root folder named top. It is idempotent and defaults to a dry-run:

oldap-tools [common_options] staging ensure-mobile-folder --staging-area <staging-area-iri>

After checking the report, repeat the command with --apply to create the missing folder. Use --staging-area more than once for an explicit set or use --all to process every StagingArea in the project. The command aborts on ambiguous or misplaced system folders instead of guessing.