omnidbm 0.1.4

omnidbm moves data between any supported database and any other. MongoDB, PostgreSQL, MySQL, SQLite, Redis, CSV, JSONL — pick a source, pick a destination, transfer. One package, one CLI, one lossless data bus.

Features

• Any-to-any transfers — every connector speaks both directions; no source/destination pairing limits
• Lossless type bus — datetimes, ObjectIds, Binary, and Decimals survive round trips via JSON-safe type tags
• Zero hardcoding — everything is configured per-run with CLI flags; nothing is baked in
• Automatic schema — PostgreSQL, MySQL, and SQLite destinations are created and typed for you
• Interactive wizard — guided setup for every supported database
• Preflight checks — doctor verifies connectivity before you start
• Battle-tested basics — batch size, per-table limits, filters, drop-first, conflict policies, dry runs, index copy

Install

pip install omnidbm

pip install "omnidbm[mysql]"   # MySQL support
pip install "omnidbm[redis]"   # Redis support
pip install "omnidbm[all]"     # everything

Requires Python 3.10+. MongoDB (pymongo) and PostgreSQL (psycopg) drivers are included; MySQL and Redis drivers are optional extras.

Quick start

# Transfer every table, schema included
omnidbm transfer -s mongodb://user:pass@host:27017/db -d postgresql://user:pass@host:5432/db

# Selective: specific tables, bigger batches, wipe the destination first
omnidbm transfer -s mongodb://... -d postgresql://... -t users,orders --batch-size 2000 --drop-first

# Dry run: count only, no writes
omnidbm transfer -s csv://./data -d mongodb://... --dry-run

# Guided interactive setup
omnidbm wizard

# Inspect a source before moving anything
omnidbm inspect mongodb://user:pass@host:27017/db --sample 3

# Connectivity preflight
omnidbm doctor mongodb://... postgresql://...

Options

URI schemes

Redis treats key prefixes (split on :) as tables; (all) targets every key. Documents use the shape {"_key", "_value", "_ttl"} and support strings, hashes, lists, and sets.

Python API

from omnidbm import transfer, TransferConfig, ConnectorConfig, TableSpec

config = TransferConfig(
    source=ConnectorConfig(uri="mongodb://user:pass@host:27017/db"),
    dest=ConnectorConfig(uri="postgresql://user:pass@host:5432/db"),
    tables=[TableSpec(source="users", dest="users")],
    drop_first=True,
)
results = transfer(config)

Custom connectors

Subclass BaseConnector, implement connect, list_tables, read_stream, write_stream, and close, then register:

from omnidbm.core.connector import BaseConnector, register


@register("myproto")
class MyConnector(BaseConnector): ...

How the type bus works

Every document is normalized to JSON-safe values on the wire, tagged so nothing is lost in transit:

Development

git clone https://github.com/Notookk/Omnidbm
pip install -e ".[dev]"

pytest          # run the test suite
ruff check .    # lint
ruff format .   # format

Roadmap

• Resumable transfers with checkpoints
• Saved config profiles
• Parallel table transfers
• Incremental / delta sync

License

MIT — see LICENSE.