Skip to main content

A Python package for using Dolt database via Python.

Project description

Current State - Deprecated

DoltPy was created in an era where Dolt was a command-line tool. We've worked very hard to make Dolt a MySQL compatible database. To use Dolt with Python, start a Dolt SQL server using dolt sql-server and connect with any Python MySQL client. Use the exposed Dolt stored procedures or system tables to access version control functionality. Popular libraries like Pandas all support MySQL connectivity.

Dolt MySQL client support works in any language, not just Python. Thus, it is more time efficient for us to focus on that interface.

DoltPy

This is DoltPy, the Python API for Dolt. Python is the language of choice for data science and data engineering, and thus we thought it would be wise to publish an API for building automated workflows on top of Dolt and DoltHub, a collaboration platform for Dolt databases.

Installation

You need to install Dolt, which is documented here. It's easy for *nix users:

$ sudo bash -c 'curl -L https://github.com/liquidata-inc/dolt/releases/latest/download/install.sh | sudo bash'

We also distribute Dolt as a Homebrew formula:

$ brew install dolt

Finally, for Windows users our release page has .zip and .msi files.

Once Dolt is installed you can install Doltpy using pip:

$ pip install doltpy

Overview

Doltpy is broken up into modules.

doltpy.cli

This is the most important module. It effectively wraps the Dolt command-line-interface (CLI) that is exposed by the Go binary. The CLI is exposed more or less exactly as it is implemented, returning wrapper objects where appropriate.

It's implementation has moved to a separate repository here

doltpy.cli.read and doltpy.cli.write

These modules provide basic read and write interfaces for reading and writing a variety of tabular data formats, including:

  • CSV files
  • pandas.DataFrame
  • columns, that is dictionaries of lists, i.e. {'col': [...vals...], ...}
  • rows, that is lists of dictionaries, i.e. [{'col': val, ...}, ...]

doltpy.sql

This module provides tools for interacting with Dolt via a Python based SQL connector. The most important class is DoltSQLContext, which has concrete subclasses DoltSQLServerContext and DoltSQLEngineContext. DoltSQLServerContext is for users that want to write Python scripts that use and manage the Dolt SQL Server instance as a child process. DoltSQLEngineContext is for users who want to interact with a remote Dolt SQL Server.

These classes have equivalents of the read and write functions in doltpy.cli.read and doltpy.cli.write for writing CSV files, pandas.DataFrame objects, rows, and columns.

doltpy.sql.sql_sync

This package provides tools for syncing data to and from Dolt, and other relational databases. Currently there is support for MySQL, Postgres, and Oracle. You can find a more detailed description of how to use SQL Sync tools here.

doltpy.etl

This module provides a set of tools for scripting ETL/ELT workflows. At Liquidata we use it internally to push datasets onto DoltHub.

More Information

As alluded to above, you can find a more detailed description of Doltpy here.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

doltpy-2.0.14.tar.gz (549.1 kB view details)

Uploaded Source

Built Distribution

doltpy-2.0.14-py3-none-any.whl (553.6 kB view details)

Uploaded Python 3

File details

Details for the file doltpy-2.0.14.tar.gz.

File metadata

  • Download URL: doltpy-2.0.14.tar.gz
  • Upload date:
  • Size: 549.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.8.16 Linux/5.15.0-1031-azure

File hashes

Hashes for doltpy-2.0.14.tar.gz
Algorithm Hash digest
SHA256 8edf22c680db9866e56b5fb4fd09b59c39e2e75001ae36e3e02b60d786a956e8
MD5 3b0b5d310600844a915a3c4e127a2c65
BLAKE2b-256 759646f3a16beb0265ed75e8bcc3cde6f83e2be7efebcac6953bb3205bc6e7f8

See more details on using hashes here.

File details

Details for the file doltpy-2.0.14-py3-none-any.whl.

File metadata

  • Download URL: doltpy-2.0.14-py3-none-any.whl
  • Upload date:
  • Size: 553.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.3.2 CPython/3.8.16 Linux/5.15.0-1031-azure

File hashes

Hashes for doltpy-2.0.14-py3-none-any.whl
Algorithm Hash digest
SHA256 0edf3f0fb03edf9b639d538b623318f28857566e10e44ccdbab85d028827bcfe
MD5 ed5624bbc18d3a5729dec42bbe024035
BLAKE2b-256 87ec9a7188967b521c251fbfd6efcb17e39215a41c4a99ba6df56a7e6dca4e0f

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page