bani-tools 1.0.1

An open-source database migration engine powered by Apache Arrow

Bani

An open-source database migration engine powered by Apache Arrow.

Bani migrates schema, data, and indexes across relational databases using Apache Arrow as a universal columnar interchange format. Define migrations declaratively with BDL, programmatically with the Python SDK, or let an AI agent drive them via the MCP server.

Features

• 5 Database Connectors -- PostgreSQL, MySQL, MSSQL, Oracle, SQLite
• Apache Arrow Engine -- Columnar interchange for high-throughput batch transfers
• Declarative BDL -- Define migrations in XML or JSON, version control everything
• Python SDK -- Fluent ProjectBuilder API for programmatic migrations
• CLI -- 11 commands: run, validate, preview, init, schema inspect, and more
• MCP Server -- 10 tools for AI agents (Claude, Cursor, etc.)
• Web Dashboard -- Real-time migration monitoring with React UI
• Cross-Platform -- macOS app, Linux packages, Windows installer, Docker

Quick Start

Install

Download the installer for your platform from the releases page (macOS .dmg, Windows .exe, Linux .deb/.rpm/AppImage), or:

# pip (Python 3.11+)
pip install bani-tools
bani ui   # opens the Web UI in your browser

# Docker
docker pull banilabs/bani:latest

Your First Migration

1. Set up database credentials as environment variables:

export SOURCE_USER=myuser
export SOURCE_PASS=mypassword
export TARGET_USER=pguser
export TARGET_PASS=pgpassword

2. Create a migration project:

bani init --source mysql --target postgresql --out my-migration.bdl

3. Run the migration:

bani run my-migration.bdl

Python SDK

from bani.sdk import BaniProject, ProjectBuilder

project = (
    ProjectBuilder("my-migration")
    .source("mysql", host="localhost", port=3306, database="source_db",
            username_env="SOURCE_USER", password_env="SOURCE_PASS")
    .target("postgresql", host="localhost", port=5432, database="target_db",
            username_env="TARGET_USER", password_env="TARGET_PASS")
    .batch_size(100_000)
    .build()
)

result = BaniProject(project).run()
print(f"Migrated {result.total_rows_written} rows in {result.duration_seconds:.1f}s")

Or load from a BDL file:

from bani.sdk import Bani

result = Bani.load("my-migration.bdl").run()

MCP Server (AI Agent Integration)

Add to your Claude Desktop configuration:

{
  "mcpServers": {
    "bani": {
      "command": "bani",
      "args": ["mcp", "start"]
    }
  }
}

Then ask Claude: "Inspect the schema of my MySQL database and generate a migration to PostgreSQL."

Supported Databases

Database	Versions	Source	Sink	Driver
PostgreSQL	9.6 -- 17	Yes	Yes	psycopg 3.x
MySQL	5.5 -- 8.4	Yes	Yes	PyMySQL
SQL Server	2019 -- 2022	Yes	Yes	pyodbc / pymssql
Oracle	11g -- 23c	Yes	Yes	oracledb
SQLite	3.x	Yes	Yes	sqlite3 (stdlib)

Architecture

Bani uses Apache Arrow RecordBatch as its universal data interchange format. Source connectors read database rows into Arrow batches; sink connectors write Arrow batches to the target database. This gives N type mappers (one per connector) instead of N x N, and enables high-throughput columnar transfers with minimal Python overhead.

Source DB --> Source Connector --> Arrow RecordBatch --> Sink Connector --> Target DB

Key components:

• Connectors -- Pluggable source/sink pairs discovered via Python entry points
• Orchestrator -- Manages table ordering (dependency-aware), batching, parallelism, and checkpointing
• BDL Parser -- Reads XML or JSON migration definitions into a ProjectModel
• SDK -- ProjectBuilder for programmatic construction, SchemaInspector for introspection
• MCP Server -- Exposes migration tools to AI agents via the Model Context Protocol

Web UI

Launch the web dashboard:

bani ui

Monitor migrations in real-time with progress tracking, table-level status, and error reporting.

Documentation

Full documentation is available at docs.bani.tools:

• Getting Started -- Install and run your first migration in under 10 minutes
• BDL Reference -- Complete specification for the Bani Definition Language
• CLI Reference -- All commands, flags, and output formats
• Python SDK -- Programmatic migration API
• MCP Server -- AI agent integration guide
• Connector Reference -- Per-database configuration and type mappings

Development

# Clone and install
git clone https://github.com/mugumedavid/bani.git
cd bani
uv sync --all-extras --dev

# Run quality gates
ruff check && ruff format --check
mypy --strict
pytest

# Or use make
make all

See CONTRIBUTING.md for the full development guide.

License

Apache-2.0. See LICENSE for details.

Links

• Website -- Project homepage
• Documentation -- Technical docs and guides
• GitHub -- Source code and issues
• Docker Hub -- Container images
• Discord -- Community chat