Skip to main content

laterite — a Rust-backed AGS4 reader, writer and validator

laterite

A Rust-backed AGS4 toolchain for the AGS4 geotechnical data format — validate, read as typed data, query, build, fix, diff, certify, and convert ↔ Excel — with a modern, born-typed polars API.

Coming from python-ags4? laterite.compat is a faithful, faster stand-in for its AGS4 and AGS4.utils modules — swap from python_ags4 import AGS4 for from laterite import compat as AGS4 and keep your code.

ci python cov PyPI Python versions License: MIT

Install

pip install laterite                     # base AGS4 (polars + duckdb, pyarrow-free)
pip install "laterite[compat]"           # + pandas (python-ags4 drop-in) — still pyarrow-free
pip install "laterite[compat,pyarrow]"   # + the optional pyarrow accelerator (or [all])

Requires Python ≥ 3.12. The wheel is abi3, so one binary covers 3.12 / 3.13 / 3.14. Installing it also puts the lat CLI on your PATH.

The [compat] drop-in is pyarrow-free and fast on its own; adding pyarrow swaps the pandas step for pyarrow's to_pandas and unlocks the Arrow-backed string dtype — an accelerator, never a requirement.

Use

import laterite

# Validate — errors + warnings by default (FYI is opt-in)
report = laterite.validate("delivery.ags")
report.is_valid
for rule, findings in report.by_rule().items():
    print(rule, len(findings))

# Read born-typed columns: a 2DP heading is a float, a DT a datetime
ags = laterite.read("delivery.ags")
ags.groups                       # ['PROJ', 'LOCA', 'SAMP', …]
ags["LOCA"]["LOCA_GL"][0]        # → 12.3  (a polars DataFrame per group)

# SQL across groups, no conversion step
ags.sql("SELECT loca_id, count(*) FROM SAMP GROUP BY 1")

# Repair a dirty file into a fresh handle, then keep working with it
fixed = ags.fix(risky=True)      # pads short rows, transliterates non-ASCII, …

# Typed graph: PROJ → LOCA → SAMP → …
from laterite.ags4 import read_typed
for loca in read_typed("delivery.ags").locas:
    print(loca.loca_id, loca.loca_gl)

# python-ags4 drop-in — swap the import, keep your code
from laterite import compat as AGS4
tables, headings = AGS4.AGS4_to_dataframe("delivery.ags")
AGS4.dataframe_to_AGS4(tables, headings, "round-trip.ags")

read returns born-typed polars frames by default (or pandas with read(..., backend="pandas")) — both pyarrow-free, read back from a Python-owned in-memory DuckDB engine.

More than a faster python-ags4

python-ags4 is the reference Python library for AGS4 — validation plus pandas read/write — and it inspired this project. laterite matches that surface and adds a toolchain on top:

laterite python-ags4
Validate — numbered AGS4 rules
Read → data frames ✅ born-typed polars or pandas pandas, all strings
Build / write AGS4 · Excel ↔ AGS4
Repair engine (fix)
SQL across groups · revision diff
Validity certificates (.ags.idx)
Transport — zstd compress + age encrypt
Typed PROJ → LOCA → SAMP graph
pyarrow required no (optional accelerator) via pandas' own deps

Performance

Synthetic, spec-valid AGS4 from ags4-forge — the wide scaffold: 123 groups, realistic type mix, zero findings. macOS arm64, hot files, mean of 5 warm runs, python-ags4 1.2.0 vs laterite 0.8.0. Both agree on the findings.

Validation

File python-ags4 check_file laterite.validate speedup
4.9 MB · 459 BH 1.5 s 50 ms 30.0×
24.9 MB · 2,219 BH 3.7 s 266 ms 13.9×
102.7 MB · 8,872 BH 12.3 s 1.1 s 11.7×
275.5 MB · 22,813 BH 34.1 s 2.6 s 13.0×
549.7 MB · 45,107 BH 70.0 s 5.4 s 12.9×

Read → typed — the honest comparison for real work. python-ags4 needs AGS4_to_dataframe + convert_to_numeric on every group to get numbers, and still leaves dates as text; laterite.read is born-typed, dates included.

File python-ags4 + convert_to_numeric laterite.read speedup
4.9 MB 187 ms 26 ms 7.2×
24.9 MB 811 ms 136 ms 6.0×
102.7 MB 3.4 s 541 ms 6.3×
275.5 MB 8.9 s 1.4 s 6.4×
549.7 MB 17.5 s 2.9 s 6.0×

Read → strings — like for like, both returning pandas frames of text.

File python-ags4 AGS4_to_dataframe laterite.compat speedup
4.9 MB 144 ms 49 ms 2.9×
24.9 MB 718 ms 206 ms 3.5×
102.7 MB 2.8 s 870 ms 3.2×
275.5 MB 7.3 s 2.2 s 3.3×
549.7 MB 15.2 s 4.6 s 3.3×

The ratio holds as files grow — the gap is a constant factor, not a head start that erodes. Reproduce any of this with uv run python tools/bench-vs-python-ags4.py in the repo: it generates the rungs, verifies each against a pinned SHA-256 so a change to the generator can't move the numbers unnoticed, and prints these exact tables.

Parity + clean-room

121 / 131 of python-ags4 1.2.0's own test suite passes through laterite.compat (92 %); the 10 remaining are deliberate non-closures, documented rule by rule. A weekly job compares the two public surfaces, so a function added upstream can't quietly go missing here.

Two caveats worth knowing before you swap the import. compat mirrors the library API — python-ags4's ags4_cli command is not mirrored, because laterite ships lat instead with its own JSON shapes; and compat is one flat module rather than a package, so from laterite.compat import AGS4 (a submodule import) is not the shape — use from laterite import compat as AGS4.

The validator is clean-room from the published AGS4 specification, not adapted from another library's source — python-ags4 is LGPL-3.0, and that separation is what lets laterite ship under MIT. Details: COMPAT.md · OBSERVATIONS.md.

One engine, every stack

laterite on PyPI is the Python surface of one Rust AGS4 engine, shared across:

Surface Package Get it
Python laterite — PyPI pip install laterite
Node.js laterite — npm npm install laterite
CLI lat bundled with this wheel
DuckDB laterite_ags4 — community extension INSTALL laterite_ags4 FROM community;
Browser validator + data explorer — WASM open in a browser

Scriptable output is byte-identical across all of them, so a CI gate and a notebook can't disagree.

Docs

Full documentation — Learn, Cookbook, Concepts, and the Python API reference — at https://niko86.github.io/laterite/docs/.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

laterite-0.10.1.tar.gz (656.6 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

laterite-0.10.1-cp312-abi3-win_amd64.whl (7.2 MB view details)

Uploaded CPython 3.12+Windows x86-64

laterite-0.10.1-cp312-abi3-manylinux2014_x86_64.whl (7.1 MB view details)

Uploaded CPython 3.12+

laterite-0.10.1-cp312-abi3-macosx_11_0_arm64.whl (6.3 MB view details)

Uploaded CPython 3.12+macOS 11.0+ ARM64

File details

Details for the file laterite-0.10.1.tar.gz.

File metadata

  • Download URL: laterite-0.10.1.tar.gz
  • Upload date:
  • Size: 656.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for laterite-0.10.1.tar.gz
Algorithm Hash digest
SHA256 4c3a4e2cb73a4c7b8ddd46ec1c903a7af77b90c86439c905b6110897e25c6164
MD5 75145f7ef5a32b58e978a8319a82a15b
BLAKE2b-256 ebb14dc7292d9d6b0b9c19e8d8302ce36113054ece2853c8f2452b226d5ebcf8

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.10.1.tar.gz:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file laterite-0.10.1-cp312-abi3-win_amd64.whl.

File metadata

  • Download URL: laterite-0.10.1-cp312-abi3-win_amd64.whl
  • Upload date:
  • Size: 7.2 MB
  • Tags: CPython 3.12+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for laterite-0.10.1-cp312-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 1ca9c92596d25a6b9e4ba0c4e270fff6de3360357e9dee5a87c216652d74a556
MD5 c0b52d3511c569ef2c7173937d0bec44
BLAKE2b-256 089fdcba12da23b4163afdd376f5fce773d99cc7479eaae5abfdfc2a55b4348b

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.10.1-cp312-abi3-win_amd64.whl:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file laterite-0.10.1-cp312-abi3-manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for laterite-0.10.1-cp312-abi3-manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d7ccec78b207c8fdd79c1713bcec7076d7e429cd6fa53568262e63ec73dc41ef
MD5 00eb2efccbbdb9cb5f2960aeaf35f376
BLAKE2b-256 35ad6c51e84903829861732eb1863f15524dd764206e10305db10042bb4df107

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.10.1-cp312-abi3-manylinux2014_x86_64.whl:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file laterite-0.10.1-cp312-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for laterite-0.10.1-cp312-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 19067c2224d56f551fc1743160c7bb77d4c532a76de9f294e3f0546d879fbea4
MD5 99ddc6d8e085a0af0bb62b1030ed0344
BLAKE2b-256 72271ff73e76b8c146e21339f9fd2efe234e1af0c5de465bd2abb79579e3f454

See more details on using hashes here.

Provenance

The following attestation bundles were made for laterite-0.10.1-cp312-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on niko86/laterite

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.12.0

5 files

0.11.0

5 files

This release

0.10.1 This release

4 files

0.10.0

4 files

0.9.0

4 files

0.8.0

4 files

0.7.0

4 files

0.6.2

4 files

0.6.1

4 files

0.6.0

4 files

0.5.1

4 files

0.5.0

4 files

0.4.0

3 files

0.2.0

3 files

0.1.0

4 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page