EduceLab Herculaneum Graph Database API

This API is considered a work in progress and can change at any moment.

Architecture

Setup Quick Start

For a visual overview of the full setup process (beyond just this repo), see the quick start guide:

For step-by-step server setup instructions, see docs/SERVER_SETUP.md.

Installation

This package supports two install modes:

Client only (lightweight)

For remote machines that only need to call the REST API:

pip install educelab-hercdb

This installs only the requests library. See src/educelab/hercdb/client/README.md for client usage.

Server (full)

For running the REST API server, loading data, or querying Neo4j directly:

pip install educelab-hercdb[server]

This adds fastapi, neo4j, numpy, pandas, prompt-toolkit, pydantic, rapidfuzz, and uvicorn.

Development Setup

# Install base dependencies
uv sync

# Or with server extras (fastapi, neo4j, etc.)
uv sync --extra server

# Run commands in the environment
uv run python -c "from educelab.hercdb.client import HercClient"

# Or activate the venv directly
source .venv/bin/activate

Client Library

from educelab.hercdb.client import HercClient

client = HercClient(host="api.example.com", token="my-token")
pherc = client.get_artifact_by_name("211")

See src/educelab/hercdb/client/README.md for the full API reference.

Direct Database Connection

For environments with the server extra installed, you can connect to Neo4j directly:

from educelab import hercdb

uri = "neo4j://localhost:7687"
user = "foo"
password = "bar"
db = hercdb.connect(uri, user, password)
if db.verify_connection():
  print("Connected!")

If credentials are not passed directly, the package reads them from ~/.educedb or environment variables. See docs/SERVER_SETUP.md for configuration details.

REST API

A FastAPI-based REST API is available for querying the database over HTTP. All endpoints require Bearer token authentication.

uv run uvicorn educelab.hercdb.rest.server:app --reload

Interactive API docs are available at /docs (Swagger UI) and /redoc (ReDoc) once the server is running.

See src/educelab/hercdb/rest/README.md for endpoint documentation and docs/SERVER_SETUP.md for production deployment.

Loading Data

Data loading is done in two steps using the loader scripts. Both read CSV files from input_data/.

1. Load metadata and UUIDs

uv run python src/educelab/hercdb/loader/metadata_loader.py

Reads (defaults):

• input_data/metadata_file.csv - Pre-processed metadata file. (PHerc, Cornice, Pezzo, Disegni nodes and properties.)
• input_data/uuid_file.csv - Pre-processed uuid file. (all EduceLabID added)

Optional arguments:

uv run python src/educelab/hercdb/loader/metadata_loader.py \
  --metadata path/to/metadata.csv \
  --uuid path/to/uuid.csv

2. Load scan data

uv run python src/educelab/hercdb/loader/scan_loader.py

Reads (defaults):

• input_data/negatives.csv - FlatbedScanDataset nodes
• input_data/pgs_datasets_20260601(in).csv - PGSRaw nodes
• input_data/spectral_datasets_20260609_final.csv - SpectralRaw nodes

By default (--replace) it deletes all existing PGSRaw/SpectralRaw nodes and reloads from scratch (FlatbedScanDataset is untouched); pass --no-replace to merge into existing data instead. Nodes are keyed on the scan uuid, so re-running is idempotent.

Optional arguments:

uv run python src/educelab/hercdb/loader/scan_loader.py \
  --negatives path/to/negatives.csv \
  --photogrammetry path/to/pgs.csv \
  --spectral path/to/spectral.csv \
  --no-replace

Note: Run metadata_loader first since scan data links to EduceLabID nodes.

Reporting Tools

Scan completeness report

el-hercdb-scan-report walks every PHerc and its hierarchy and writes two CSVs that flag artifacts needing first-time scans or re-scans.

# After `uv sync --extra server`, the entry point is on PATH:
uv run el-hercdb-scan-report --out-dir ./tmp

# Equivalent fallback without re-syncing:
uv run python -m educelab.hercdb.cli.scan_completeness --out-dir ./tmp

Outputs (default names):

• scan_completeness_full.csv — one row per (artifact, UUID) pair, plus blank-UUID sentinel rows for artifacts that have no EduceLabID assigned. Columns: PHerc, Cornice, Pezzo, UUID, PGS Status, PGS Latest Complete Date, PGS Path, Spectral Status, Spectral Latest Complete Date, Spectral Path, Institution.
• scan_completeness_issues.csv — same shape, filtered to rows where PGS or Spectral is missing/incomplete, or the artifact has no UUID at all.

Status values are complete / incomplete / missing when the artifact has a UUID, and blank when it does not (so "UUID assigned but no scan" stays distinguishable from "no UUID even assigned"). Files are written with utf-8-sig so Excel opens them with correct character encoding.

The report walks REPLACES relationships between EduceLabIDs, so pre-replacement scans on a retired predecessor UUID still count toward the artifact's coverage.

Temporary Scripts and Notes

The tmp/ directory contains temporary scripts, notes, and other informal resources shared among the team. Contents are version controlled but considered ephemeral — they may be rewritten or deleted at any time and should not be relied upon as stable code.

Delete all data

To clear the database before reloading:

from educelab.hercdb.loader import PhercGraphDatabaseLoader
loader = PhercGraphDatabaseLoader()
loader._delete_all_nodes()