Skip to main content

Universal SQLite Synchronization Core - A dependency-grade, local-first, offline-first SQLite synchronization primitive

Project description

sqlite-sync-core

Python 3.11+ License: AGPL-3.0 PyPI Status: Production Ready

A production-ready SQLite synchronization library for distributed systems.

sqlite-sync-core provides a powerful, local-first synchronization engine that works seamlessly across multi-peer networks. It handles the "hard parts" of sync (HLC clocks, causality, delta bundles, and conflict resolution) while providing a clean API for developers.


๐Ÿš€ Quick Start

Installation

pip install sqlite-sync-core

CLI Usage

# Initialize a database for sync
sqlite-sync init myapp.db --table tasks

# Start a sync server
sqlite-sync start myapp.db --port 8000

# Sync with a remote peer
sqlite-sync sync myapp.db http://peer:8000

# Run as daemon (background sync)
sqlite-sync sync myapp.db http://peer:8000 --daemon

# Discover peers on LAN
sqlite-sync peers --discover --auto-add

# Add a column migration (syncs to peers)
sqlite-sync migrate myapp.db --table tasks --add-column priority --type INTEGER

๐Ÿ—๏ธ Features

  • Hybrid Logical Clocks: Causal ordering with wall-clock correlation
  • Conflict Resolution: LWW, Field-Level Merge, or Custom strategies
  • Multi-Transport: HTTP and WebSocket sync
  • P2P Discovery: Zero-config peer discovery on LAN
  • Schema Evolution: Migrations that sync across the network
  • Background Sync: Scheduler with daemon mode

Library Usage

Initialize and Sync

from sqlite_sync import SyncEngine
from sqlite_sync.transport.http_transport import HTTPTransport
from sqlite_sync.scheduler import SyncScheduler

# Initialize the engine
engine = SyncEngine("app.db")
engine.initialize()

# Enable sync for a table
with engine:
    engine.enable_sync_for_table("tasks")

# Set up transport and scheduler
transport = HTTPTransport("http://peer:8000", engine.device_id)
scheduler = SyncScheduler(engine, transport, interval_seconds=10)

# Start background sync
scheduler.start(in_background=True)

Schema Migrations

from sqlite_sync.schema_evolution import SchemaManager

with engine:
    sm = SchemaManager(engine.connection)
    
    # Add a column - will sync to all peers
    migration = sm.add_column("tasks", "priority", "INTEGER", default_value="0")
    print(f"Migration created: {migration.migration_id.hex()}")

Generate and Import Bundles

# Generate a delta bundle for a peer
bundle_path = engine.generate_bundle(
    peer_device_id=peer_b_id,
    output_path="delta.db"
)

# Import bundle on receiving peer
result = engine.import_bundle("delta.db")
print(f"Applied: {result.applied_count}, Conflicts: {result.conflict_count}")

Core Invariants

# Invariant Description
1 Causal Consistency HLC ensures correct partial ordering of operations
2 Deterministic Replay Same operations = same state across all replicas
3 Conflict Tolerance Detects and resolves conflicts explicitly
4 Offline-First Works entirely without cloud or internet

Architecture

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚     Your Application            โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                โ”‚ Uses
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚       sqlite-sync-core          โ”‚
โ”‚  (Engine, Transport, Scheduler) โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                โ”‚ Persists to
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚         SQLite Database         โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

CLI Commands

Command Description
init Initialize database for sync
start / serve Start HTTP sync server
sync Run sync (one-off or daemon)
status Show sync status and conflicts
resolve Resolve conflicts interactively
migrate Manage schema migrations
peers Discover and manage peers
snapshot Create database snapshot

License

AGPL-3.0 for Open Source. Contact shivaysinghrajput@proton.me for commercial licensing.


Built for developers who need reliable sync infrastructure.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sqlite_sync_core-0.6.1.tar.gz (102.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

sqlite_sync_core-0.6.1-py3-none-any.whl (114.2 kB view details)

Uploaded Python 3

File details

Details for the file sqlite_sync_core-0.6.1.tar.gz.

File metadata

  • Download URL: sqlite_sync_core-0.6.1.tar.gz
  • Upload date:
  • Size: 102.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.6

File hashes

Hashes for sqlite_sync_core-0.6.1.tar.gz
Algorithm Hash digest
SHA256 c327d2a90eedb2295318f3800e0b32e7a769e8f6a4d6a10754fd4ca738513f5a
MD5 2187eb840cdb3396dd0653c8445bb405
BLAKE2b-256 4f51bb1a6a13198c756c97f0a3fdafafb21ee91c3f613880e3240deb749854aa

See more details on using hashes here.

File details

Details for the file sqlite_sync_core-0.6.1-py3-none-any.whl.

File metadata

File hashes

Hashes for sqlite_sync_core-0.6.1-py3-none-any.whl
Algorithm Hash digest
SHA256 3a250d8885c5784f13e08822d33ce2708479c84c1e5e159a61373e1953d14f0b
MD5 bff864192ce15ffc06c39b0f4c5ec5e3
BLAKE2b-256 63565086aec495b33630e221c62b9d432a3e06eaac6b584bd6c969f9e05e3605

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page