Skip to main content

Xtalate

CI Version License: Apache-2.0 Python 3.11+

The trusted translation layer between computational-chemistry file formats — a converter that tells you exactly what it kept, what it lost, and why.

Every conversion produces a structured Conversion Report (what was preserved, dropped, or fabricated, and the reason for each) and an automatic Validation Report (the output re-parsed and diffed against the source to prove the report told the truth). The guiding rule is simple: never silently lose scientific information. If you diffed the input and output by hand, nothing should surprise you that Xtalate didn't already tell you about.

What it is

Xtalate does one thing: loss-aware, fully transparent conversion between structure and trajectory file formats. It is a single Python package with four ways to use it, all over the same engine:

  • A library — parse a file into one Canonical Object, convert it, read the reports.
  • A CLI (xtalate) — inspect / convert / validate / capabilities, JSON-emitting and CI-native.
  • An HTTP service (/v1) — a FastAPI app that exposes the engine as async jobs; ships as the optional service extra.
  • A Web UI — a Next.js front end that walks upload → inspect → convert → recover → record → download, rendering the engine's reports verbatim.

The scientific logic lives in the engine and nowhere else; the service and the UI are thin presenters that embed the report models unchanged. Visualization, structure editing, MD, and analysis are explicitly out of scope — Xtalate stores and translates data computed elsewhere.

Formats

Seven formats, every one read and written, so every pair among them converts (the nightly suite runs the full 7 × 7 matrix):

XYZ · extended XYZ (ASE-backed) · POSCAR · CONTCAR (incl. the velocity block, Cartesian + Direct) · XDATCAR · ASE .traj · CIF.

CIF is treated as real crystallography. Cell parameters become lattice vectors, fractional coordinates become Cartesian at the parser boundary, and symmetry is expanded from the operations the file declares — parsed as exact affine maps over rationals, with sites on a symmetry element merged on a physical 0.05 Å threshold. A file that names a space group but declares no operations is refused, never read as a partial structure. Site occupancy is a first-class canonical field (atoms.occupancies); a target that cannot represent a partial occupancy says so in the report rather than dropping it silently. The exporter writes every atom explicitly under an identity symmetry loop with no space-group symbol — the coordinates it emits are the already-expanded cell, and any symbol above them would assert a setting they no longer encode.

What every conversion gives you

  • Inspect first. The Information Discovery Engine reports a ✓/✗ inventory of which canonical fields a file actually contains, each annotated with the format's capability — no conversion required.
  • Predict loss before writing. A per-format Capability Matrix tells the exporter what the target can hold before it writes, so loss is predicted (P5), not discovered after the fact.
  • Recover explicitly, never guess. When a target needs a field the source lacks — a lattice, velocities, masses — or can hold only one frame, you supply a preset choice and it is recorded as an Assumption. With no choice, the conversion refuses rather than inventing data (P4). Fabrication is exactly what you asked for and nothing more.
  • Validate, always. Every completed conversion is re-parsed through the ordinary reader and diffed against the expected object under a numeric tolerance profile (default / strict / loose, or a custom table). There is no switch to skip it.
  • Absence is information (P3). The Canonical Model distinguishes "the source never had this" (None) from "the source had it, and the value is zero." Parsers never default an absent field.
  • Scale to large trajectories. A frame-chunked streaming core keeps pipeline memory sub-linear in the number of frames — a 10⁴-configuration XDATCAR streams at roughly constant memory and yields a report byte-identical to the materialized path.
  • Third-party formats via plugins. A parser/exporter shipped in a separate package is discovered automatically through Python entry points (xtalate.parsers / xtalate.exporters), with no fork or edit; it joins sniffing, discovery, conversion, and validation on equal footing. First-party formats hold no privileged API.

Install

pip install xtalate                 # the library + the `xtalate` CLI
pip install "xtalate[service]"      # add the FastAPI /v1 service layer
# or, from a checkout:
pip install -e ".[dev]"

Requires Python ≥ 3.11. The only scientific dependency is ASE (for extended XYZ and the ASE .traj format); NumPy and pydantic power the Canonical Model, and PyYAML parses custom tolerance tables and corpus manifests.

Quickstart (CLI)

Inspect a file — see what's actually inside it, before converting anything:

$ xtalate inspect water_traj.xyz
File:   water_traj.xyz  (164 bytes)
Format: Plain XYZ [xyz]  confidence 0.9
Structure: 2 frame(s) × 3 atoms; species O, H

Canonical fields (✓ present / ✗ absent / ◐ mixed · read capability):
  ✓ atoms.symbols                    [full]  — O, H, H
  ✓ atoms.positions                  [full]  — 2 frame(s) × 3 atoms, Cartesian (Å)
  ✗ atoms.masses                     [none]
  ✗ cell.lattice_vectors             [none]
  … (16 canonical leaf paths, each shown present or absent)

Convert a 2-frame, lattice-less XYZ trajectory to POSCAR. POSCAR needs a single structure and a lattice, so we supply two explicit recovery choices; each becomes a recorded Assumption:

$ xtalate convert water_traj.xyz --to poscar -o POSCAR \
    --recover frame_selection=last \
    --recover missing_lattice=bounding_box,padding_ang=5.0
Conversion Report  [final · completed · permissive]
  xyz → poscar
  preserved (2): atoms.symbols, atoms.positions
  removed (2):   custom_per_frame['xyz:comment']; 1 dropped frame
  supplied (2):  cell.lattice_vectors, cell.pbc  (from A2)
  assumptions (2):
    ~ A1 frame_selection=last:   frame 1 of 2 retained …
    ~ A2 missing_lattice=bounding_box:  axis-aligned box + 5.0 Å padding …

Validation Report  [passed]  (tolerance profile: default)
  ✓ atom_count · ✓ species_preservation · ✓ positions_rmsd · ✓ lattice_consistency
  ✓ frame_count · – numeric_field_fidelity · ✓ metadata_preservation
  ✓ absence_conformance · ✓ report_consistency

Without the --recover flags the same command refuses (exit code 2) and prints exactly which decisions are needed — a refusal is a first-class, reported outcome, never a silent default. Exit codes make the CLI CI-native: 0 ok · 2 refused · 3 validation failed · 4 parse error · 5 warnings under --mode strict · 1 usage error. Any command accepts --json to emit the report schema verbatim. See the CLI reference.

Quickstart (library)

from xtalate.registry import default_registry
from xtalate.conversion import ConversionEngine

registry = default_registry()
with open("in.extxyz", "rb") as fh:
    source = registry.get_parser("extxyz").parse(fh, filename="in.extxyz").canonical

result = ConversionEngine(registry).convert(
    source, source_format_id="extxyz", target_format_id="poscar",
)
print(result.report.model_dump_json(indent=2))   # the Conversion Report
print(result.validation.status)                   # "passed"
with open("POSCAR", "wb") as fh:
    fh.write(result.output)

A complete, runnable example is in examples/convert_extxyz_to_poscar.py.

Quickstart (HTTP service)

Run a dependency-free instance (SQLite + local filesystem, jobs executed in-process):

pip install "xtalate[service]"
python -m backend                       # http://localhost:8000

…or bring up the full stack — API, worker, PostgreSQL, MinIO, Redis — with one command:

docker compose up --build --wait
curl -s "http://localhost:8000/v1/health?ready=true"

Conversion is an async job; a refusal comes back as a completed HTTP-200 job, not an error:

BASE=http://localhost:8000/v1
FILE_ID=$(curl -s -F "file=@in.extxyz" "$BASE/upload" | jq -r .file_id)
JOB=$(curl -s "$BASE/convert" -H 'content-type: application/json' \
  -d "{\"file_id\":\"$FILE_ID\",\"target_format_id\":\"poscar\"}" | jq -r .job_id)
curl -s "$BASE/jobs/$JOB" | jq '.state, .result.conversion_report.status'
CID=$(curl -s "$BASE/jobs/$JOB" | jq -r .result.conversion_id)
curl -s "$BASE/download/$CID" -o POSCAR

Reports outlive the bytes they describe: input and output expire on independent lifecycle windows while GET /v1/conversions/{id} still serves both reports. The full flow — including interactive recovery (allow_recovery → pause → resume) — is walked with curl in docs/API.md, and the machine-readable contract is the committed docs/openapi.json.

Web UI

A faithful web front end over /v1 — the whole workflow in a browser, with the loss report as the thing you cannot miss. It is a presentation layer only (carries no scientific logic): every number, code, and reason on screen is the engine's own, rendered verbatim. The wizard is four steps — upload & inspect, preview the predicted loss, convert & recover (one decision card per unresolved scenario, no option preselected, the exact Assumption shown before you confirm it), then a consolidated record whose outcome header is quantitative ("Converted — 7 fields removed", never "Done!") and whose download control sits below the loss summary by layout law. Two read-only pages — /formats (the Capability Matrix, generated from GET /v1/capabilities) and /history — make the engine's own knowledge browsable, and the whole docs/ corpus renders as an in-app docs site.

Self-hosting is the primary supported deployment: a hardened docker-compose.prod.yml points at external Postgres and S3-compatible storage. See the self-hosting guide.

How it works

Native File → Format Sniffer → Parser → Canonical Object → Exporter → Target Format
                                             ↑        ↓
                         Information Discovery   Capability Matrix
                         Recovery Engine (explicit only) → Validation Engine

The Canonical Object is the only thing that crosses the parser/exporter boundary — no parser ever calls another parser, and no conversion takes a format-to-format shortcut (P2). That single spine is what makes adding a format O(1) in the number already present: each of XDATCAR, ASE .traj, and CIF arrived as one parser and one exporter plus a Capability Matrix row, joining sniffing, discovery, conversion, validation, and the full n×n round-trip matrix without a single edit to any other format.

The design and its principles are in docs/ARCHITECTURE.md; the library and CLI surface in docs/API.md; building and extending Xtalate in docs/DEVELOPER_GUIDE.md. Architectural decisions (the D-log) and the master specification are maintained privately; public commits may reference decision IDs. If you need the rationale for a particular decision, open an issue.

Versioning and stability

Xtalate follows Semantic Versioning (the current release is shown in the badge above), and the version number protects a named public surface: within a major series each of these evolves additively only — new formats, scenarios, optional fields, and hooks arrive with safe defaults; nothing already documented is removed, renamed, or given a new meaning. Anything that would break one of them waits for the next major version, with migration notes.

  • The canonical schema — field names, shapes, unit conventions, and absence semantics (P3) of the eight-category Canonical Model.
  • The report schemas — DiscoveryReport, ConversionReport, ValidationReport, embedded verbatim in every result.
  • The plugin SDK ABCs — ParserPlugin, ExporterPlugin, the streaming surface, ParseResult/ParseIssue/ParseError, and FormatCapabilities (see CONTRIBUTING.md).
  • The /v1 REST surface — endpoints, response envelopes, and error codes; docs/openapi.json is its versioned, machine-readable form.
  • The documented CLI — the four subcommands, their flags, the --json convention, and the exit-code ladder 0–5 (see the CLI reference).

The internal, _-prefixed surface is explicitly not part of the contract.

Two version axes, moving under distinct rules. The product version (xtalate.__version__, pyproject.toml, CITATION.cff — guarded to agree) is the SemVer of the distribution and bumps on every release. The canonical schema_version is the on-the-wire version of the Canonical Model, stamped into every object; it bumps only when the schema itself changes, and always behind a real migration — an older stored object is carried forward on load and gains a ConversionRecord(operation="migrate"), never silently. A release can ship without a schema change, so the two numbers are decoupled by design.

Development

pip install -e ".[dev]"
ruff check . && ruff format --check .    # lint + format
mypy                                     # types (strict)
lint-imports                             # acyclic package layering (P2)
pytest                                   # tests

CI runs this matrix on Python 3.11 and 3.13, plus the corpus governance suite over both corpus roots (manifest schema + license, source hashes, ATTRIBUTIONS.md regeneration) and a coverage ratchet.

Contributing

Contributions are welcome — see CONTRIBUTING.md. The invited path is corpus contributions: real, licensed sample files that harden the converter. A golden case (tests/golden/) asserts what a file should produce and needs an expectation you verified by hand; a wild case (tests/wild/) is a real third-party file asserting what it does produce — the exact set of issue codes plus the composition it declares for itself — so it needs a triage rather than a derivation. Both need a manifest and a license; no manifest, no license, no merge. Parser plugins are welcome too — the plugin SDK is a stable contract (see CONTRIBUTING.md for the stability promise and its scope).

License

Apache-2.0 — see LICENSE and NOTICE.

Metadata

Release files for xtalate 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for xtalate 1.0.0
File Size Uploaded Provenance Actions
xtalate-1.0.0.tar.gz 1.7 MB Without provenance Details for xtalate-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for xtalate 1.0.0
File Interpreter ABI Platform Actions
xtalate-1.0.0-py3-none-any.whl Python 3 none any Details for xtalate-1.0.0-py3-none-any.whl

Total release size: 1.9 MB

Release files / xtalate-1.0.0.tar.gz

Download URL xtalate-1.0.0.tar.gz
Size 1.7 MB
Tags Source
SHA-256 checksum
How to use checksums
f06359a8c5bb20b087e426327752bc4fdfb00ff5470c98fa686f8996388691ba
BLAKE2b-256 checksum
How to use checksums
b19cc12099c9365c6070954be7d5e8f0795cbf6194733cb87b0645932afc3f4c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.6

Release files / xtalate-1.0.0-py3-none-any.whl

Download URL xtalate-1.0.0-py3-none-any.whl
Size 255.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
89f07c9cb5eccb8187c64c4a14c39eb7eab2eee7972710446699e47fda208be3
BLAKE2b-256 checksum
How to use checksums
3ad4c605cd1eaea7c625c6c86293c6fec1131c93df873b0c4bc12641b9f10f3d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.14.6

Security for xtalate 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page