geotech-report-extraction 0.5.0

Extract geotechnical data from PDF reports and output DIGGS XML

Geotech Report Extraction

Extract geotechnical borehole data from PDF reports or Azure Document Intelligence JSON exports and output DIGGS 2.6 XML.

Features

• Parse borehole logs from geotechnical reports (Langan, Schnabel, and generic formats)
• Extract soil layers, SPT blow counts, groundwater levels, and lab test results
• Azure Document Intelligence (DI) JSON input for cloud/serverless workflows
• Palantir Foundry integration with ready-to-use Spark transforms
• XGBoost page classifier for automatic boring log identification
• Template-based appendix cover page detection and report structure analysis
• Optional vision-based extraction using Anthropic Claude or GPT-4o via Palantir Funhouse
• Output DIGGS 2.6 XML for interoperability
• Geospatial utilities for coordinate conversion and boring location mapping

Installation

pip install geotech-report-extraction

Core dependencies include XGBoost, scikit-learn, and pandas for ML-based page classification.

Optional extras

# PDF parsing (PyMuPDF)
pip install geotech-report-extraction[pdf]

# OCR support (Tesseract)
pip install geotech-report-extraction[ocr]

# Vision LLM extraction (Anthropic Claude)
pip install geotech-report-extraction[vision]

# Geospatial utilities (coordinate conversion, mapping)
pip install geotech-report-extraction[geo]

# Everything
pip install geotech-report-extraction[all]

Quick Start

From PDF

from geotech_report_extraction import extract_report

result = extract_report("report.pdf")

# With vision LLM
result = extract_report("report.pdf", use_vision=True, vision_api_key="sk-...")

From Azure Document Intelligence JSON

from geotech_report_extraction.di_reader import extract_from_di_json

result = extract_from_di_json("report_di.json")

# Or with a pre-parsed dict
result = extract_from_di_json(di_data_dict, file_label="my_report")

Palantir Foundry

See foundry_transforms/boring_log_pipeline.py for a three-stage Spark pipeline:

1. Flatten raw DI JSON files into a per-page tabular dataset
2. Identify boring log pages and group by boring ID
3. Extract samples, soil layers, and water levels per boring

Report Structure Analysis

After page classification, the appendix cover page analyzer identifies report structure:

from geotech_report_extraction.report_structure import analyze_report_structure

# pages: list of dicts with 'text' and 'predicted_class' keys
structure = analyze_report_structure(pages)

for section in structure.sections:
    print(f"Appendix {section.letter}: {section.title} ({section.appendix_type})")
    print(f"  Pages {section.start_page}–{section.end_page}")

The analyzer builds a per-report template from classifier-predicted covers, then uses it to confirm predictions and find missed covers. Works across firm formats (Schnabel structured headers, Langan minimal covers).

CLI

geotech-extract report.pdf -o output.xml

License

MIT