Skip to main content

TUI wizard and CLI for migrating vector data into CyborgDB

Project description

cyborgdb-migrate

PyPI - Version PyPI - License PyPI - Python Version

A TUI wizard and CLI for migrating vector data from popular vector databases into CyborgDB — the encrypted vector database.

Supports Pinecone, Qdrant, Weaviate, ChromaDB, and Milvus as sources, with encrypted-at-rest indexes, checkpoint & resume, and post-migration verification built in.

Key Features

  • Interactive TUI — step-by-step wizard powered for migration
  • Headless CLI — non-interactive mode for scripts and CI/CD pipelines
  • Checkpoint & resume — automatically saves progress; resume interrupted migrations
  • Spot-check verification — post-migration vector and metadata integrity checks
  • Double-buffered I/O — overlaps extraction and upsert for maximum throughput

Getting Started

To get started in minutes, check out the CyborgDB Quickstart Guide.

Installation

pip install cyborgdb-migrate

Install with support for your source database:

# Individual sources
pip install "cyborgdb-migrate[pinecone]"
pip install "cyborgdb-migrate[qdrant]"
pip install "cyborgdb-migrate[weaviate]"
pip install "cyborgdb-migrate[chromadb]"
pip install "cyborgdb-migrate[milvus]"

# All sources at once
pip install "cyborgdb-migrate[all]"

Usage

Interactive (TUI)

cyborgdb-migrate

The wizard walks you through selecting a source, entering credentials, connecting to CyborgDB, and running the migration with live progress.

Headless (CLI)

Create a TOML config file (see example-config.toml):

[source]
type = "pinecone"
api_key = "${PINECONE_API_KEY}"
index = "my-index"

[destination]
host = "http://localhost:8000"
api_key = "${CYBORGDB_API_KEY}"
create_index = true
index_name = "my-cyborgdb-index"
index_type = "ivfflat"

[options]
batch_size = 200
checkpoint_every = 10
cyborgdb-migrate --config migration.toml

Resume an interrupted migration:

cyborgdb-migrate --config migration.toml --resume

Supported Sources

Source Extras Notes
Pinecone pinecone Supports namespaces
Qdrant qdrant Scroll-based pagination
Weaviate weaviate Supports named vectors
ChromaDB chromadb Local and remote modes
Milvus milvus Supports partitions, content field heuristic

Configuration Reference

For full configuration details, CLI options, and exit codes, see the Configuration Guide.

Documentation

For more information on CyborgDB, see the Cyborg Docs.

License

The CyborgDB Migration Tool is licensed under the MIT License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cyborgdb_migrate-0.1.1.tar.gz (40.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

cyborgdb_migrate-0.1.1-py3-none-any.whl (45.3 kB view details)

Uploaded Python 3

File details

Details for the file cyborgdb_migrate-0.1.1.tar.gz.

File metadata

  • Download URL: cyborgdb_migrate-0.1.1.tar.gz
  • Upload date:
  • Size: 40.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for cyborgdb_migrate-0.1.1.tar.gz
Algorithm Hash digest
SHA256 2d29a4910493264e398b5d404e8194f3f90e6ed7e7bb84f9a7e31144b81b31c0
MD5 0ccae8d1e2260880d927c9f1c9f00498
BLAKE2b-256 336bcebc25b13e5b7fdb2796e989f45256a74db00c895fecdc642045661dad91

See more details on using hashes here.

Provenance

The following attestation bundles were made for cyborgdb_migrate-0.1.1.tar.gz:

Publisher: ci.yml on cyborginc/cyborgdb-migrate

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file cyborgdb_migrate-0.1.1-py3-none-any.whl.

File metadata

File hashes

Hashes for cyborgdb_migrate-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 b9c9bacf8ecc9436d2edda6160d42b1c12f93dd7096a4f59dd9e1b3a95798100
MD5 68822835167fedc6b1180d584b9007ec
BLAKE2b-256 a6e8e7af31727e41a298fb3c253816b3d18e627fcc1d9ff1ca894b2e6d9518c5

See more details on using hashes here.

Provenance

The following attestation bundles were made for cyborgdb_migrate-0.1.1-py3-none-any.whl:

Publisher: ci.yml on cyborginc/cyborgdb-migrate

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page