Skip to main content

Abraxos

PyPI version Documentation status CI License: MIT

Abraxos helps you build pandas ETL pipelines that can keep processing after a row fails. Read CSVs, transform and validate DataFrames, and load them into SQL while keeping rejected records available for inspection.

Documentation · API reference · Upgrade guide

Installation

Abraxos requires Python 3.11+ and pandas 2.3.3+, including pandas 3.x.

uv venv --python 3.11
uv pip install abraxos

The base package includes CSV reading, transformations, and DataFrame utilities. Install the optional integrations for SQLAlchemy and Pydantic as needed. Both extras are used by the example below.

uv pip install 'abraxos[sql,validate]'

The same package names and extras work with python -m pip install in an activated virtual environment. The sql extra installs SQLAlchemy 2.0.44 or later, and validate installs Pydantic 2.12 or later.

Quick start

This pipeline reads a CSV, trims names, validates ages, and creates a SQLite table with a primary key. The sample includes one rejected record at each of the CSV, validation, and SQL stages.

from io import StringIO

from pydantic import BaseModel, Field
from sqlalchemy import Column, Integer, MetaData, String, Table, create_engine

from abraxos import read_csv, to_sql, transform, validate

source = StringIO(
    'person_id,name,age\n1, Ada ,28\noops,too,many,fields\n2, Grace ,invalid\n1, Duplicate ,34\n'
)
extracted = read_csv(source)
cleaned = transform(extracted.dataframe, lambda batch: batch.assign(name=batch['name'].str.strip()))


class Person(BaseModel):
    id: int = Field(alias='person_id')
    name: str
    age: int = Field(ge=0)


checked = validate(cleaned.success_df, Person, output='model')
people = Table(
    'people',
    MetaData(),
    Column('id', Integer, primary_key=True),
    Column('name', String(100), nullable=False),
    Column('age', Integer, nullable=False),
)

engine = create_engine('sqlite://')
try:
    with engine.begin() as connection:
        loaded = to_sql(checked.success_df, people.name, connection, table=people)
        stored = connection.execute(people.select().order_by(people.c.id)).all()
finally:
    engine.dispose()

print('CSV rejections:', extracted.bad_lines)
print('Validation rejections:', checked.errored_df['name'].tolist())
print('SQL rejections:', loaded.errored_df['name'].tolist())
print('Stored people:', stored)
CSV rejections: [['oops', 'too', 'many', 'fields']]
Validation rejections: ['Grace']
SQL rejections: ['Duplicate']
Stored people: [(1, 'Ada', 28)]

Working with results

Operation Result and recovery behavior
read_csv(source) dataframe contains accepted records. bad_lines contains records with the wrong number of fields. A CSV syntax error that prevents recovery raises CsvParseError with its source and starting line.
read_csv(source, chunksize=1000) Produces the same result type for each chunk. Rejected records count toward chunksize. Pass dtype for consistent conversion across chunks.
transform(df, callback) Retries failed batches on smaller partitions. Pass split_on_error=False for operations such as ranking or normalization that need the entire frame.
validate(df, Model) Maps validated values to input columns by default. Pass output='model' for serialized fields and defaults, or column_map for an explicit input-to-output mapping.
to_sql(df, name, con) Rolls back and retries recoverable append failures on SQLAlchemy SQLite and PostgreSQL connections. Pass table= to define the schema for a new table.
use_sql(df, con, insert) Executes a supplied insert statement and leaves the SQLAlchemy transaction open for the caller to commit or roll back.

transform, validate, to_sql, and use_sql return success_df, errored_df, and errors. Terminal SQL failures raise LoadError. A raw sqlite3.Connection supports one standalone write with autocommit disabled. Use SQLAlchemy for transactional retries and table replacement.

Development

uv sync --locked
uv run --locked pytest
uv run --locked sphinx-build -W -b html docs docs/_build/html
uv run --locked sphinx-build -W -b doctest docs docs/_build/doctest

See the contribution guide for linting, type checks, and PostgreSQL integration tests. See the changelog for release history. Abraxos is maintained by Odos Matthews and licensed under MIT.

Metadata

Release files for abraxos 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for abraxos 0.2.0
File Size Uploaded Provenance Actions
abraxos-0.2.0.tar.gz 50.8 kB With provenance Details for abraxos-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for abraxos 0.2.0
File Interpreter ABI Platform Actions
abraxos-0.2.0-py3-none-any.whl Python 3 none any Details for abraxos-0.2.0-py3-none-any.whl

Total release size: 79.8 kB

Release files / abraxos-0.2.0.tar.gz

Download URL abraxos-0.2.0.tar.gz
Size 50.8 kB
Tags Source
SHA-256 checksum
How to use checksums
3aab666ab46e7075e5743ca92d4c2d61178fdbc48e2177e89ff8202d48502c3d
BLAKE2b-256 checksum
How to use checksums
1fdcc93d81859a6a2c9f15afa68d5e00d2e151290f9a14b8c590792eb0cbf779
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / abraxos-0.2.0-py3-none-any.whl

Download URL abraxos-0.2.0-py3-none-any.whl
Size 29.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
880576235369bbb77efda9a1e3c667cc1dd59ee0233c47fca3cdfd8498657a86
BLAKE2b-256 checksum
How to use checksums
85948265bf6190c7b8252e5ea4833f5f939fffe1a363d3379ce901efac042313
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Security for abraxos 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for abraxos-0.2.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page