IsoMap

Isotopic and paleoecological data standardisation middleware.

IsoMap is an open-source desktop application that automates the standardisation, validation, and export of legacy isotopic and paleoecological datasets for submission to centralised repositories (Neotoma, IsoArcH, PANGAEA, NOAA NCEI, LiPD).

📖 User Guide — full walkthrough with screenshots, workflow steps, and troubleshooting. 📋 Changelog — version history and release notes.

Features

IsoMap provides a 6-tab wizard interface backed by 8 processing engines. Implementation status is marked next to each engine — several engines are partial stubs; see CHANGELOG.md for the full list of known limitations.

• Import Engine ✅: CSV and Excel spreadsheets with automatic encoding detection and type inference.
• Column Mapping Engine ✅: 6-stage fallback pipeline (exact → preferences → fuzzy → semantic embeddings → Valentine ensemble → distribution profiling) to map source columns to 5 target repository schemas. Valentine is now correctly wired against the ≥1.0 API; the semantic stage requires the optional [ml] extras (SciBERT ONNX, ~400 MB).
• Chronology Engine ✅: Auto-detects and normalises uncalibrated/calibrated dates to cal_BP with 1950 CE anchor. Note: this is format normalisation only — no radiocarbon calibration (IntCal20 etc.) is performed.
• Spatial Verification Engine ✅: Cleans coordinates, displays on interactive Leaflet map, exports bounding box. EPSG:4326 only — no UTM/other CRS transformation yet.
• Validation Engine ✅: Pandera constraint checking with row-level error tracking against schema definitions. Required-field presence is enforced.
• Export Engine ⚠️ partial: CSV, Excel, GeoJSON, JSON, PANGAEA, NOAA, RO-Crate, LaTeX data paper all produce output. LiPD export is non-conformant (missing required LiPD v1.3 fields; pylipd.load() rejects it). PANGAEA and NOAA exports are template stubs with [INSERT_*] placeholders, not submission-ready files.
• Provenance Engine ⚠️ partial: ProvenanceTracker class exists and is unit-tested but is not wired into the RPC pipeline — main.py doesn't instantiate it. Produces a JSON audit log, not PROV-O RDF as the original scope doc describes.
• Automated Data Papers ✅: Jinja2 ESSD-compliant LaTeX manuscript generation.

Technology Stack

Getting Started

Prerequisites

• Rust 1.70+ — rustup.rs
• Python 3.12+ — pip install -e ".[dev]"
• Node.js 20+ with npm

Development

cd ui && npm install        # Frontend dependencies
npm run tauri dev           # Full desktop app with hot-reload
npm run dev                 # Vite frontend only (no Tauri)
cd src-tauri && cargo build # Rust backend only

Testing

pytest tests/               # Python backend (36 tests)
cd ui && npx playwright test # E2E browser tests (6 tests)

Production Build

./scripts/build-sidecar.sh  # PyInstaller standalone binary
cd ui && npm run tauri build # Full desktop app bundle

Bundles output to ui/src-tauri/target/release/bundle/.

Supported Repositories

Add custom schemas in data/schemas/ following the JSON Schema format.

Architecture

React invoke() → Rust Tauri commands → sidecar_rpc() → shell wrapper → Python JSON-RPC

The Rust layer (ui/src-tauri/src/lib.rs) exposes 9 Tauri commands that serialise parameters to JSON-RPC and forward them to the Python sidecar (src/isomap/main.py) via stdin/stdout. The sidecar processes the request and returns a JSON-RPC response.

Key Files

License

MIT