laterite

A Rust-backed AGS4 toolchain for the AGS4 geotechnical data format — validate, read as typed data, query, build, fix, diff, certify, and convert ↔ Excel — with a modern, born-typed polars API.

Coming from python-ags4? laterite.compat is a faithful, faster stand-in for its AGS4, check, utils and data modules — one token changes: from python_ags4 import AGS4 becomes from laterite.compat import AGS4, and your code keeps working.

Install

pip install laterite                     # base AGS4 (polars + duckdb, pyarrow-free)
pip install "laterite[compat]"           # + pandas (python-ags4 drop-in) — still pyarrow-free
pip install "laterite[compat,pyarrow]"   # + the optional pyarrow accelerator (or [all])

Requires Python ≥ 3.12. The wheel is abi3, so one binary covers 3.12 / 3.13 / 3.14. Installing it also puts the lat CLI on your PATH.

The [compat] drop-in is pyarrow-free and fast on its own; adding pyarrow swaps the pandas step for pyarrow's to_pandas and unlocks the Arrow-backed string dtype — an accelerator, never a requirement.

Use

import laterite

# Validate — errors + warnings by default (FYI is opt-in)
report = laterite.validate("delivery.ags")
report.is_valid
for rule, findings in report.by_rule().items():
    print(rule, len(findings))

# Read born-typed columns: a 2DP heading is a float, a DT a datetime
ags = laterite.read("delivery.ags")
ags.groups                       # ['PROJ', 'LOCA', 'SAMP', …]
ags["LOCA"]["LOCA_GL"][0]        # → 12.3  (a polars DataFrame per group)

# SQL across groups, no conversion step
ags.sql("SELECT loca_id, count(*) FROM SAMP GROUP BY 1")

# Repair a dirty file into a fresh handle, then keep working with it
fixed = ags.fix(risky=True)      # pads short rows, transliterates non-ASCII, …

# Typed graph: PROJ → LOCA → SAMP → …
from laterite.ags4 import read_typed
for loca in read_typed("delivery.ags").locas:
    print(loca.loca_id, loca.loca_gl)

# python-ags4 drop-in — swap the import, keep your code
from laterite import compat as AGS4
tables, headings = AGS4.AGS4_to_dataframe("delivery.ags")
AGS4.dataframe_to_AGS4(tables, headings, "round-trip.ags")

read returns born-typed polars frames by default (or pandas with read(..., backend="pandas")) — both pyarrow-free, read back from a Python-owned in-memory DuckDB engine.

More than a faster python-ags4

python-ags4 is the reference Python library for AGS4 — validation plus pandas read/write — and it inspired this project. laterite matches that surface and adds a toolchain on top:

Performance

Synthetic, spec-valid AGS4 from ags4-forge — the wide scaffold: 123 groups, realistic type mix, zero findings. macOS arm64, hot files, mean of 5 warm runs, python-ags4 1.2.0 vs laterite 0.12.0. Both agree on the findings.

Validation

Read → typed — the honest comparison for real work. python-ags4 needs AGS4_to_dataframe + convert_to_numeric on every group to get numbers, and still leaves dates as text; laterite.read is born-typed, dates included.

Read → strings — like for like, both returning pandas frames of text.

Validation — peak RSS

Read → typed — peak RSS

Read → strings — peak RSS

Peak RSS of one fresh process per cell; the ratio is python-ags4's peak over laterite's, so above 1 laterite holds less. The largest rung is time-only. Read → strings measured on the pyarrow accelerator hop.

The compat door holds more than python-ags4 — those ratios sit below 1 on purpose: that is the cost of the pandas bridge, and publishing it is the point. laterite.validate and laterite.read are the recommended paths, on memory as on time.

The time ratio holds as files grow — the gap is a constant factor, not a head start that erodes. Reproduce any of this with uv run python tools/bench-vs-python-ags4.py in the repo: it generates the rungs, verifies each against a pinned SHA-256 so a change to the generator can't move the numbers unnoticed, and prints these exact tables.

Parity + clean-room

121 / 131 of python-ags4 1.2.0's own test suite passes through laterite.compat (92 %); the 10 remaining are deliberate non-closures, documented rule by rule. A weekly job compares the two public surfaces, so a function added upstream can't quietly go missing here.

compat is a package mirroring upstream's own layout, so upstream's import shapes work with one token changed:

from python_ags4 import AGS4          # becomes:
from laterite.compat import AGS4

from python_ags4.AGS4 import AGS4_to_dataframe   # becomes:
from laterite.compat.AGS4 import AGS4_to_dataframe

AGS4, check, utils and data are real, distinct submodules — so code that depends on module identity (monkeypatching check.X and expecting AGS4 not to see it) behaves as it does upstream. from laterite import compat as AGS4 also still works: the flat namespace is unchanged.

Two caveats before you swap. No top-level python_ags4 package ships, and that is deliberate and permanent — inside this wheel it would collide with the real distribution in site-packages. So the import token changes; the rest of the line does not. And compat mirrors the library API only: python-ags4's ags4_cli command is not mirrored, because laterite ships lat instead with its own JSON shapes.

The validator is clean-room from the published AGS4 specification, not adapted from another library's source — python-ags4 is LGPL-3.0, and that separation is what lets laterite ship under MIT. Details: COMPAT.md · OBSERVATIONS.md.

One engine, every stack

laterite on PyPI is the Python surface of one Rust AGS4 engine, shared across:

Scriptable output is byte-identical across all of them, so a CI gate and a notebook can't disagree.

Docs

Full documentation — Learn, Cookbook, Concepts, and the Python API reference — at https://docs.laterite.dev/.