omniload

Copy data between any source and any destination.

About

omniload is a polyglot data loader framework based on dlt, the open-source Python library for building data pipelines. dlt does the heavy lifting (schema inference, normalization, and incremental load state); omniload wraps it behind a single CLI and a URI scheme, so you can copy data between any source and any destination without writing pipeline code, either from your shell or the Python API.

omniload provides the same efficient incremental data loading modes inherited by dlt: append, merge, and delete+insert.

Install

Container

For running omniload in a container environment, use the provided OCI images.

docker run --rm ghcr.io/panodata/omniload:latest version

Package

For running or installing the omniload Python package, we are recommending to use uv. ¹²

uvx omniload

Alternatively, if you like to install it account-wide on your system:

uv tool install omniload

Synopsis

CLI

Instruct omniload to read from the PostgreSQL table public.some_data, and write to the BigQuery warehouse table omniload.some_data.

omniload ingest \
    --source-uri 'postgresql://admin:admin@localhost:8837/web?sslmode=disable' \
    --source-table 'public.some_data' \
    --dest-uri 'bigquery://<your-project-name>?credentials_path=/path/to/service/account.json' \
    --dest-table 'omniload.some_data'

Python API

Using omniload.run_ingest, you can run the same ingestion from your own application. The keyword arguments map one-to-one to the CLI options, and it returns the dlt LoadInfo for the run (or None for a dry run).

from omniload import run_ingest

info = run_ingest(
    source_uri="sqlite:///./source.db",
    dest_uri="duckdb:///./warehouse.duckdb",
    source_table="main.some_table",
    dest_table="public.some_table",
)
print(info)

See the Python API documentation for dry runs, error handling, and passing enum options as strings.

Handbook

Please visit the quickstart documentation, or inspect the list of supported sources and destinations.

Project

Contribute

Contributions are very much welcome. Please visit the sandbox documentation to learn how to spin up a development environment on your workstation and submit patches, or create a ticket to report a bug or propose a feature.

Status

Breaking changes should be expected until a 1.0 release, so version pinning is strongly recommended, especially when using this software as a library. For example:

pip install 'omniload[full]==0.0.42'

License

The project is licensed under the MIT License, see the LICENSE file for details. Some components are licensed under the Apache 2.0 license, see the NOTICE file for details.

Acknowledgements

This project would not have been possible without the amazing work by the authors and contributors to SQLAlchemy, dlt, and ingestr, turtles all the way down. Kudos.

You can install uv using pip install uv, or another method that matches your operating system. ↩
While installing omniload with vanilla pip is possible, it is an order of magnitude slower. ↩

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

omniload-0.9.0.tar.gz (425.5 kB view details)

Uploaded Aug 4, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

omniload-0.9.0-py3-none-any.whl (573.3 kB view details)

Uploaded Aug 4, 2026 Python 3

File details

Details for the file omniload-0.9.0.tar.gz.

File metadata

Download URL: omniload-0.9.0.tar.gz
Upload date: Aug 4, 2026
Size: 425.5 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for omniload-0.9.0.tar.gz
Algorithm	Hash digest
SHA256	`71410057004ece8e60959eca438f129739c6e729cc85955b6d5103fac07555b7`
MD5	`6b07e50020df280e7e13a59c33ae79f3`
BLAKE2b-256	`2cd4bbc6cdbb82193dfc555aa71267cf7ede8397f31dd4e1e17d3ec98551ba00`

See more details on using hashes here.

Provenance

The following attestation bundles were made for omniload-0.9.0.tar.gz:

Publisher: release-pypi.yml on panodata/omniload

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: omniload-0.9.0.tar.gz
- Subject digest: 71410057004ece8e60959eca438f129739c6e729cc85955b6d5103fac07555b7
- Sigstore transparency entry: 2337829853
- Sigstore integration time: Aug 4, 2026
Source repository:
- Permalink: panodata/omniload@a9b8429e6e36764c20b33dc1b36ad3e6cb1f82b0
- Branch / Tag: refs/tags/v0.9.0
- Owner: https://github.com/panodata
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: release-pypi.yml@a9b8429e6e36764c20b33dc1b36ad3e6cb1f82b0
- Trigger Event: push