terria-catalog

A Pythonic SDK for creating, modifying, validating and publishing TerriaJS initialization catalogs.

terria-catalog lets you build and edit Terria init files as strongly-typed Python objects — CatalogGroup, GeoJsonCatalogItem, WebMapServiceCatalogItem, and friends — instead of hand-editing JSON or juggling dictionaries. It mirrors Terria's own terminology and object model (derived directly from the TerriaJS source Traits classes), and produces valid Terria init JSON that is compatible with the official specification.

It is designed to be the canonical way our Digital Twin ecosystem creates and modifies catalogs stored in Amazon S3 — but it has no hard dependency on S3 (or even on any I/O): storage and serialization are cleanly decoupled and injectable.

from terria_catalog import Catalog, GeoJsonCatalogItem

catalog = Catalog.from_s3(bucket="my-bucket", key="init/catalog.json")

catalog.add(
    GeoJsonCatalogItem(
        name="Reservoirs",
        id="reservoirs",
        url="https://example.org/reservoirs.geojson",
    ),
    path="/Water Resources/Reservoirs",  # missing groups are auto-created
)

catalog.move("reservoirs", "/Water Resources")
catalog.update("reservoirs", description="Major reservoirs in the basin")
catalog.remove("some-old-dataset")

catalog.publish()  # validates, serializes, and writes back to S3

You never touch JSON. You never touch dictionaries. The JSON exists only inside the serializer.

Installation

pip install terria-catalog          # core (no cloud dependencies)
pip install 'terria-catalog[s3]'    # + boto3 for the S3 storage backend

Requires Python 3.10+.

Core concepts

Supported item types

Fully modeled (with type-specific traits):

GeoJsonCatalogItem · WebMapServiceCatalogItem · WebMapTileServiceCatalogItem · CsvCatalogItem · ShapefileCatalogItem · CogCatalogItem · CesiumTerrainCatalogItem · Cesium3DTilesCatalogItem · IonImageryCatalogItem · ArcGisMapServerCatalogItem · ArcGisFeatureServerCatalogItem

Placeholders (correct type, traits not yet modeled — data still round-trips): KmlCatalogItem, CzmlCatalogItem, GpxCatalogItem, GltfCatalogItem, WebFeatureServiceCatalogItem, OpenStreetMapCatalogItem, MapboxVectorTileCatalogItem.

Any unknown Terria type encountered when loading a catalog becomes a GenericCatalogMember so no data is ever lost on a load/save round-trip.

The public API

Loading

from terria_catalog import Catalog

Catalog.from_s3(bucket="b", key="init/catalog.json")  # needs [s3] extra
Catalog.from_file("wwwroot/init/catalog.json")
Catalog.from_json(json_string)
Catalog()  # a new, empty catalog

Editing (paths or ids everywhere)

catalog.add(item, path="/Water Resources/Reservoirs")  # returns the item
catalog.remove("dataset-id")  # id, name, or path
catalog.update("dataset-id", description="...", opacity=0.6)
catalog.move("dataset-id", "/Infrastructure/Dams")
catalog.reorder("/Water Resources", ["reservoirs", "gauges"])

Identifiers are resolved as a path if they contain /, otherwise by id (then by name). Unknown keyword arguments to update() are stored in the member's extra mapping, so you can set traits the SDK doesn't model yet.

Groups as first-class objects

water = catalog.group("/Water Resources")  # created if missing
water.add(GeoJsonCatalogItem(name="Reservoirs", id="r", url="..."))
water.create_group("Rivers")  # idempotent
water.group("Rivers/Perennial")  # relative, auto-creating
water.remove("r")

Validation & transactions

catalog.validate()  # raises ValidationFailedError on problems
errors = catalog.validate(raise_on_error=False)

with catalog.transaction():  # atomic: rolls back on error…
    catalog.add(...)
    catalog.move(...)
    catalog.remove(...)
# …or if validation fails at the end of the block

Built-in validators cover duplicate ids, duplicate paths, invalid hierarchy, missing required properties, and unknown Terria types.

Publishing

catalog.publish()  # to the bound storage backend
catalog.publish(LocalStorage("out.json"))
catalog.to_json()  # -> str  (no I/O)
catalog.to_dict()  # -> dict (no I/O)

Safe releases: versioning, rollback & concurrency

catalog.publish() is a plain overwrite (last-write-wins). When several pipelines edit the same S3 catalog and you need rollback + a CDN cache refresh, use ReleaseManager:

from terria_catalog import ReleaseManager, S3Storage

manager = ReleaseManager(
    S3Storage("my-bucket", "init/catalog.json"),  # enable S3 bucket versioning
    cache_invalidator=my_cloudfront_invalidator,  # your CacheInvalidator impl
)

# Concurrency-safe read-modify-write: reloads & retries if another pipeline wins.
manager.update(lambda cat: cat.add(item, path="/Climate"), message="add rainfall")

# Rollback to a previous version.
versions = manager.history()  # newest first
manager.rollback(to=versions[1].version_id)

ReleaseManager.publish() validates first and returns a Release handle (ETag, version id, checksum, timestamp). Optimistic concurrency uses conditional writes (If-Match), so competing pipelines retry instead of clobbering each other. The CacheInvalidator interface is where CloudFront (or any CDN) plugs in — the library itself has no CDN dependency. Environment specifics (bucket, IAM, the actual CloudFront call) stay in your deployment code.

Use it with your AI (no JSON, no code study)

Colleagues can manage the catalog in natural language through their assistant of choice. Three surfaces, one safe tool set (validate → preview → confirm → publish, with version history and rollback):

• MCP server — for Claude Desktop/Code, Cursor, etc. pip install 'terria-catalog[mcp]', point it at a catalog with env vars, run terria-catalog-mcp. Then just chat: "add the 2026 flood COG under Flood Forecasting and publish."
• CLI — terria-catalog --s3 bucket/init/catalog.json add --type cog … (with --json and --dry-run), ideal for shell-driving agents.
• Python — terria_catalog.agent.CatalogSession + run_tool / export_schemas for custom function-calling agents.

There's also a Claude Skill (skills/terria-catalog/) and a drop-in system prompt. Full guide: docs/AI_USAGE.md.

Design philosophy

• Objects, not JSON. Consumers manipulate typed Terria objects. JSON lives only in serializers/terria_json.py.
• Decoupled storage. The catalog talks to a StorageBackend interface; S3 is just one implementation and is imported lazily so core has no cloud deps.
• Injectable everything. Serializer, storage, config and validators are all passed in, never hard-wired.
• Open for extension. Adding a new item type is: create a dataclass that sets terria_type, export it, add a test. It self-registers — no serializer, registry, or facade changes required.

See docs/ARCHITECTURE.md for the full design and examples/ for runnable scripts.

Documentation

• Use it with your AI — MCP / CLI / Python for assistants.
• Coordinating multiple writers — the pipeline-vs-human plan.
• Architecture — how the library is built.
• System prompt · Claude Skill · AGENTS.md · llms.txt

Development

pip install -e '.[dev]'
pre-commit install

ruff check . && ruff format --check .
black --check .
mypy
pytest --cov=terria_catalog

See CONTRIBUTING.md.

License

MIT © International Water Management Institute (IWMI).