Skip to main content

laterite — a Rust-backed AGS4 reader, writer and validator

laterite

A Rust-backed AGS4 toolchain for the AGS4 geotechnical data format — validate, read as typed data, query, build, fix, diff, certify, and convert ↔ Excel — with a modern, born-typed polars API.

Coming from python-ags4? laterite.compat is a faithful, faster stand-in for its AGS4 and AGS4.utils modules — swap from python_ags4 import AGS4 for from laterite import compat as AGS4 and keep your code.

ci python cov PyPI Python versions License: MIT

Install

pip install laterite                     # base AGS4 (polars + duckdb, pyarrow-free)
pip install "laterite[compat]"           # + pandas (python-ags4 drop-in) — still pyarrow-free
pip install "laterite[compat,pyarrow]"   # + the optional pyarrow accelerator (or [all])

Requires Python ≥ 3.12. The wheel is abi3, so one binary covers 3.12 / 3.13 / 3.14. Installing it also puts the lat CLI on your PATH.

The [compat] drop-in is pyarrow-free and fast on its own; adding pyarrow swaps the pandas step for pyarrow's to_pandas and unlocks the Arrow-backed string dtype — an accelerator, never a requirement.

Use

import laterite

# Validate — errors + warnings by default (FYI is opt-in)
report = laterite.validate("delivery.ags")
report.is_valid
for rule, findings in report.by_rule().items():
    print(rule, len(findings))

# Read born-typed columns: a 2DP heading is a float, a DT a datetime
ags = laterite.read("delivery.ags")
ags.groups                       # ['PROJ', 'LOCA', 'SAMP', …]
ags["LOCA"]["LOCA_GL"][0]        # → 12.3  (a polars DataFrame per group)

# SQL across groups, no conversion step
ags.sql("SELECT loca_id, count(*) FROM SAMP GROUP BY 1")

# Repair a dirty file into a fresh handle, then keep working with it
fixed = ags.fix(risky=True)      # pads short rows, transliterates non-ASCII, …

# Typed graph: PROJ → LOCA → SAMP → …
from laterite.ags4 import read_typed
for loca in read_typed("delivery.ags").locas:
    print(loca.loca_id, loca.loca_gl)

# python-ags4 drop-in — swap the import, keep your code
from laterite import compat as AGS4
tables, headings = AGS4.AGS4_to_dataframe("delivery.ags")
AGS4.dataframe_to_AGS4(tables, headings, "round-trip.ags")

read returns born-typed polars frames by default (or pandas with read(..., backend="pandas")) — both pyarrow-free, read back from a Python-owned in-memory DuckDB engine.

More than a faster python-ags4

python-ags4 is the reference Python library for AGS4 — validation plus pandas read/write — and it inspired this project. laterite matches that surface and adds a toolchain on top:

laterite python-ags4
Validate — numbered AGS4 rules
Read → data frames ✅ born-typed polars or pandas pandas, all strings
Build / write AGS4 · Excel ↔ AGS4
Repair engine (fix)
SQL across groups · revision diff
Validity certificates (.ags.idx)
Transport — zstd compress + age encrypt
Typed PROJ → LOCA → SAMP graph
pyarrow required no (optional accelerator) via pandas' own deps

Performance

Synthetic, spec-valid AGS4 from ags4-forge — the wide scaffold: 123 groups, realistic type mix, zero findings. macOS arm64, hot files, mean of 5 warm runs, python-ags4 1.2.0 vs laterite 0.8.0. Both agree on the findings.

Validation

File python-ags4 check_file laterite.validate speedup
4.9 MB · 459 BH 1.5 s 50 ms 30.0×
24.9 MB · 2,219 BH 3.7 s 266 ms 13.9×
102.7 MB · 8,872 BH 12.3 s 1.1 s 11.7×
275.5 MB · 22,813 BH 34.1 s 2.6 s 13.0×
549.7 MB · 45,107 BH 70.0 s 5.4 s 12.9×

Read → typed — the honest comparison for real work. python-ags4 needs AGS4_to_dataframe + convert_to_numeric on every group to get numbers, and still leaves dates as text; laterite.read is born-typed, dates included.

File python-ags4 + convert_to_numeric laterite.read speedup
4.9 MB 187 ms 26 ms 7.2×
24.9 MB 811 ms 136 ms 6.0×
102.7 MB 3.4 s 541 ms 6.3×
275.5 MB 8.9 s 1.4 s 6.4×
549.7 MB 17.5 s 2.9 s 6.0×

Read → strings — like for like, both returning pandas frames of text.

File python-ags4 AGS4_to_dataframe laterite.compat speedup
4.9 MB 144 ms 49 ms 2.9×
24.9 MB 718 ms 206 ms 3.5×
102.7 MB 2.8 s 870 ms 3.2×
275.5 MB 7.3 s 2.2 s 3.3×
549.7 MB 15.2 s 4.6 s 3.3×

The ratio holds as files grow — the gap is a constant factor, not a head start that erodes. Reproduce any of this with uv run python tools/bench-vs-python-ags4.py in the repo: it generates the rungs, verifies each against a pinned SHA-256 so a change to the generator can't move the numbers unnoticed, and prints these exact tables.

Parity + clean-room

121 / 131 of python-ags4 1.2.0's own test suite passes through laterite.compat (92 %); the 10 remaining are deliberate non-closures, documented rule by rule. A weekly job compares the two public surfaces, so a function added upstream can't quietly go missing here.

Two caveats worth knowing before you swap the import. compat mirrors the library API — python-ags4's ags4_cli command is not mirrored, because laterite ships lat instead with its own JSON shapes; and compat is one flat module rather than a package, so from laterite.compat import AGS4 (a submodule import) is not the shape — use from laterite import compat as AGS4.

The validator is clean-room from the published AGS4 specification, not adapted from another library's source — python-ags4 is LGPL-3.0, and that separation is what lets laterite ship under MIT. Details: COMPAT.md · OBSERVATIONS.md.

One engine, every stack

laterite on PyPI is the Python surface of one Rust AGS4 engine, shared across:

Surface Package Get it
Python laterite — PyPI pip install laterite
Node.js laterite — npm npm install laterite
CLI lat bundled with this wheel
DuckDB laterite_ags4 — community extension INSTALL laterite_ags4 FROM community;
Browser validator + data explorer — WASM open in a browser

Scriptable output is byte-identical across all of them, so a CI gate and a notebook can't disagree.

Docs

Full documentation — Learn, Cookbook, Concepts, and the Python API reference — at https://niko86.github.io/laterite/docs/.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

laterite-0.9.0.tar.gz (1.1 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

laterite-0.9.0-cp312-abi3-win_amd64.whl (7.2 MB view details)

Uploaded CPython 3.12+Windows x86-64

laterite-0.9.0-cp312-abi3-manylinux2014_x86_64.whl (7.1 MB view details)

Uploaded CPython 3.12+

laterite-0.9.0-cp312-abi3-macosx_11_0_arm64.whl (6.3 MB view details)

Uploaded CPython 3.12+macOS 11.0+ ARM64

File details

Details for the file laterite-0.9.0.tar.gz.

File metadata

  • Download URL: laterite-0.9.0.tar.gz
  • Upload date:
  • Size: 1.1 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for laterite-0.9.0.tar.gz
Algorithm Hash digest
SHA256 436f5e43198f74cd1fc1cd8f7cc318cac0ed7d225fb1a927df41251cb37bbfe6
MD5 56a04934c3b7228904d74626e4634f9f
BLAKE2b-256 bf767f2013799a8cd5ca0f5d2b6b356755ff4f8e0eb88b180b41bcd63b79920f

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.9.0.tar.gz:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file laterite-0.9.0-cp312-abi3-win_amd64.whl.

File metadata

  • Download URL: laterite-0.9.0-cp312-abi3-win_amd64.whl
  • Upload date:
  • Size: 7.2 MB
  • Tags: CPython 3.12+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for laterite-0.9.0-cp312-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 d7e6c8bc3d4053aa783231c6a1f6fe69453c6d56119022003db6ae2b5c7623b6
MD5 ad8e677bf55515dc92e273e6391e0b34
BLAKE2b-256 35d308f3159655f6eb8fec7e79fd59cb440f4dd146fafc57d3747470b34cd0a1

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.9.0-cp312-abi3-win_amd64.whl:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file laterite-0.9.0-cp312-abi3-manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for laterite-0.9.0-cp312-abi3-manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7b25e7ee47b5386c95d37314387c91374645d471c6cea458d556c6026cb1e131
MD5 3c40d1447dffc808cec9c0ba6202cfe7
BLAKE2b-256 623ae99a11e2bc6dd803e5b0a8db706792b311e613c69e3d44f7b7554b92ac3a

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.9.0-cp312-abi3-manylinux2014_x86_64.whl:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file laterite-0.9.0-cp312-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for laterite-0.9.0-cp312-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 617e5033bdc85610e6fcd3ed94cea4e41198a63740a4825f0b3d4667a3dae00f
MD5 e1900d0ba5dd93a1c5e0d21c7be35b88
BLAKE2b-256 5e59fb7146881f1ac181d42cc20fd6d5484639fb9cee422382f2e770821f4154

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.9.0-cp312-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.12.0

5 files

0.11.0

5 files

0.10.1

4 files

0.10.0

4 files

This release

0.9.0 This release

4 files

0.8.0

4 files

0.7.0

4 files

0.6.2

4 files

0.6.1

4 files

0.6.0

4 files

0.5.1

4 files

0.5.0

4 files

0.4.0

3 files

0.2.0

3 files

0.1.0

4 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page