tidbcloudy 1.1.1

(Unofficial) Python SDK for TiDB Cloud

Python SDK for TiDB Cloud

tidbcloudy is an unofficial Python SDK for TiDB Cloud. If you encounter any problems or have suggestions, feel free to open an issue on the GitHub repository Oreoxmt/tidbcloudy.

Table of contents

• Introduction

  • Compatibility with TiDB Cloud API
  • Enhancements comparing to original TiDB Cloud API

• Installation

• Usage

  • Prerequisites
  • List all resources in your organization
  • Create a project and manage your AWS CMEK New in 1.0.9
  • Create a cluster
  • Connect to TiDB
  • Modify a cluster
  • Backup and restore
  • Pause or resume your cluster New in 1.0.0
  • Get monthly bills of your organization New in 1.0.8
  • Delete all resources

• Related projects

Introduction

For more information about TiDB Cloud API, see TiDB Cloud API Documentation (v1beta and v1beta1).

TiDB Cloud is a fully-managed Database-as-a-Service (DBaaS) that brings everything great about TiDB to your cloud.

If you do not have a TiDB Cloud account yet, you can sign up here. For more details about TiDB Cloud, refer to TiDB Cloud Documentation.

You can use this SDK to access TiDB Cloud and manage your billings, projects, clusters, backups and restores:

• manage your billings of your organization (get)
• manage your TiDB Cloud projects (list, create) and manage the AWS CMEK of your projects (get, create)
• list all available cloud providers (AWS and GCP), regions, and specifications before creating or modifying a cluster
• manage your TiDB Serverless or TiDB Dedicated clusters (create, modify, pause, resume, get, list, delete)
• manage your backups of a cluster (create, get, list, delete)
• manage your restores of a project (create, get, list)

Compatibility with TiDB Cloud API

tidbcloudy is compatible with TiDB Cloud API. Endpoints added in v1beta Release 20230228 and v1beta Release 20230905 are not supported for now. The following table lists the compatibility between tidbcloudy and TiDB Cloud API.

	TiDB Cloud API v1beta1	TiDB Cloud API v1beta
2023-09-28	20230905	20230801	20230602	20230328	20230321	20230228	20230214	20230104	20221028	20220920	20220906	20220823	20220809
1.1.1	✅	❌	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.1.0	✅	❌	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.10	✅	❌	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.9	✅	❌	✅	✅	✅	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.8	✅	❌	✅	✅	❌	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.7	❌	❌	✅	✅	❌	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.6	❌	❌	✅	✅	❌	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.5	❌	❌	✅	✅	❌	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.4	/	/	/	/	❌	✅	❌	✅	✅	✅	✅	✅	✅	✅
1.0.3	/	/	/	/	/	/	/	/	/	✅	✅	✅	✅	✅
1.0.2	/	/	/	/	/	/	/	/	/	✅	✅	✅	✅	✅
1.0.1	/	/	/	/	/	/	/	/	/	/	/	✅	✅	✅
1.0.0	/	/	/	/	/	/	/	/	/	/	/	/	✅	✅
0.2.2	/	/	/	/	/	/	/	/	/	/	/	/	/	✅
0.2.1	/	/	/	/	/	/	/	/	/	/	/	/	/	✅
0.2.0	/	/	/	/	/	/	/	/	/	/	/	/	/	✅

Enhancements comparing to original TiDB Cloud API

• Iterate over resources instead of manual pagination
• Connect to a TiDB cluster using the MySQL client
• Get a Project using a Project ID
• Configure your cluster with method chaining
• Add your current IP address automatically
• Wait for the cluster to be ready when creating/modifying a cluster
• Case-insensitive when setting cluster type, cloud provider, and component name

Installation

pip3 install tidbcloudy

⚠️ Make sure that you have installed mysql client in your environment. For more information, see PyMySQL/mysqlclient.

Usage

Prerequisites

• Create a TiDB Cloud account.
• Create a TiDB Cloud API key. To manage your API keys, see TiDB Cloud API Documentation.
• Install the latest version of tidbcloudy.

To get full code examples, see the examples folder.

Note:

It is recommended to set environment variables for your API public and private key. For example, in bash, you can:

export PUBLIC_KEY=your_api_public_key
export PRIVATE_KEY=your_api_private_key

List all resources in your organization

This feature is available in tidbcloudy 1.0.9 or later.

To get the full code example of listing all projects, clusters, backup tasks, and restore tasks in your organization, see 0_list_resources.py.

import os

import tidbcloudy
from tidbcloudy.specification import ClusterType

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)

for project in api.iter_projects():
    print(project)
    if project.aws_cmek_enabled:
        for aws_cmek in project.iter_aws_cmek():
            print(aws_cmek)
    for cluster in project.iter_clusters():
        print(cluster)
        if cluster.cluster_type == ClusterType.DEDICATED:
            for backup in cluster.iter_backups():
                print(backup)
    for restore in project.iter_restores():
        print(restore)

Create a project and manage your AWS CMEK

To create a project, run the 8_manage_project.py.

import os

import tidbcloudy

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
debug_mode = os.environ.get("TIDBCLOUDY_LOG")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
# Create a project with AWS CMEK enabled
project = api.create_project(name="0", aws_cmek_enabled=True, update_from_server=True)
print(project)

# Configure AWS CMEK for the project
project.create_aws_cmek([("your_aws_region_1", "your_aws_kms_arn_1"), ("your_aws_region_2", "your_aws_kms_arn_2")])

# List all AWS CMEKs of the project
for cmek in project.iter_aws_cmek():
    print(cmek)

Create a cluster

Before creating a cluster, you should list all available provider regions and cluster configuration specifications. For more details, run the 1_list_provider_regions.py.

import os

import tidbcloudy

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)

for spec in api.list_provider_regions():
    print(f"- type: {spec.cluster_type.value}")
    print(f"  provider: {spec.cloud_provider.value}")
    print(f"  region: {spec.region}")
    print(f"  components:")
    for tidb in spec.tidb:
        print(f"  - tidb: {tidb.node_size}; "
              f"min={tidb.node_quantity_range.min} step={tidb.node_quantity_range.step}")
    for tikv in spec.tikv:
        print(f"  - tikv: {tikv.node_size}; "
              f"min={tikv.node_quantity_range.min} "
              f"step={tikv.node_quantity_range.step}; "
              f"{tikv.storage_size_gib_range.min}..{tikv.storage_size_gib_range.max} GiB")
    for tiflash in spec.tiflash:
        print(
            f"  - tiflash: {tiflash.node_size}; "
            f"min={tiflash.node_quantity_range.min} step={tiflash.node_quantity_range.step}; "
            f"{tiflash.storage_size_gib_range.min}..{tiflash.storage_size_gib_range.max} GiB")

Note:

Creating a cluster might cost money. For more details, see TiDB Cloud pricing details.

To create a TiDB Serverless cluster, run the 2_1_create_serverless_cluster.py.

To create a TiDB Dedicated cluster, run the 2_2_create_dedicated_cluster.py.

The following takes creating a TiDB Serverless cluster as an example:

import os
import tidbcloudy
from tidbcloudy.specification import CreateClusterConfig

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
debug_mode = os.environ.get("TIDBCLOUDY_LOG")
project_id = os.environ.get("PROJECT_ID", "1234567890123456789")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
project = api.get_project(project_id, update_from_server=True)

config = CreateClusterConfig()
config\
    .set_name("serverless-0") \
    .set_cluster_type("DEVELOPER") \
    .set_cloud_provider("AWS") \
    .set_region("us-west-2") \
    .set_root_password("your_root_password")
cluster = project.create_cluster(config)
print(cluster)

cluster.wait_for_available(interval_sec=1)
print(cluster)

Connect to TiDB

To connect to your TiDB cluster, run the 3_connect_mysql.py.

import os

import tidbcloudy
from tidbcloudy.specification import ClusterStatus

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
project_id = os.environ.get("PROJECT_ID", "1234567890123456789")
cluster_id = os.environ.get("CLUSTER_ID", "1234567890123456789")

print("Connecting to TiDB Cloud...")
api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
project = api.get_project(project_id, update_from_server=True)
cluster = project.get_cluster(cluster_id)
print(cluster)

if cluster.status.cluster_status == ClusterStatus.AVAILABLE:
    connection_strings = cluster.status.connection_strings
    connection = cluster.connect(type="standard", database="test", password=os.environ.get("CLUSTER_PWD", "your_root_password"))
    print(connection)
    with connection:
        with connection.cursor() as cursor:
            cursor.execute("SELECT DATABASE();")
            m = cursor.fetchone()
            print(m[0])

Modify a cluster

Note:

Modify a cluster might cost money. For more details, see TiDB Cloud pricing details.

To modify a cluster, run the 4_scale_a_cluster.py.

import os

import tidbcloudy
from tidbcloudy.specification import UpdateClusterConfig

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
project_id = os.environ.get("PROJECT_ID", "1234567890123456789")
cluster_id = os.environ.get("CLUSTER_ID", "1234567890123456789")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
project = api.get_project(project_id, update_from_server=True)
cluster = project.get_cluster(cluster_id)

print("The original config is: {}".format(cluster.config.components.to_object()))

new_config = UpdateClusterConfig()
new_config.update_component("tiflash", node_quantity=1, node_size="8C64G", storage_size_gib=500)
cluster.update(new_config)
cluster.wait_for_available()

print("The new config is: {}".format(cluster.config.components.to_object()))

Backup and restore

Note:

Backup or restore a cluster might cost money. For more details, see TiDB Cloud pricing details.

To create a backup and restore, run the 5_backup_restore.py

import os

import tidbcloudy
from tidbcloudy.specification import CreateClusterConfig

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
project_id = os.environ.get("PROJECT_ID", "1234567890123456789")
cluster_id = os.environ.get("CLUSTER_ID", "1234567890123456789")
backup_id = "1234567"

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
project = api.get_project(project_id, update_from_server=True)
cluster = project.get_cluster(cluster_id)
print("Create a manual backup task")
backup = cluster.create_backup(name="backup-1", description="automatically generated by tidbcloudy")
print(backup)

config = CreateClusterConfig()
config \
    .set_cluster_type("DEDICATED") \
    .set_cloud_provider("AWS") \
    .set_region("us-west-2") \
    .set_port(4399) \
    .set_root_password("your_root_password") \
    .set_component("tidb", "8C16G", 1) \
    .set_component("tikv", "8C32G", 3, 500) \
    .set_component("tiflash", "8C64G", 1, 500) \
    .add_current_ip_access()
print("Create a restore task from backup_id={}".format(backup_id))
restore = project.create_restore(backup_id=backup_id, name="restore-1", cluster_config=config)
restore_task = project.get_restore(restore.id)
print(restore_task.to_object())
for restore in project.iter_restores():
    print(restore)

Pause or resume your cluster

This feature is available in tidbcloudy 1.0.0 or later.

To pause or resume your cluster, run the 6_pause_cluster.py.

import os

import tidbcloudy
from tidbcloudy.specification import ClusterStatus

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
project_id = os.environ.get("PROJECT_ID", "1234567890123456789")
cluster_id = os.environ.get("CLUSTER_ID", "1234567890123456789")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
project = api.get_project(project_id, update_from_server=True)
cluster = project.get_cluster(cluster_id)

if cluster.status.cluster_status == ClusterStatus.AVAILABLE:
    print("Pause the cluster id={}".format(cluster_id))
    cluster.pause()
if cluster.status.cluster_status == ClusterStatus.PAUSED:
    print("Resume the cluster id={}".format(cluster_id))
    cluster.resume()
if cluster.status.cluster_status == ClusterStatus.RESUMING:
    print("Wait for the RESUMING cluster id={} to be available".format(cluster_id))
    cluster.wait_for_available()

Get monthly bills of your organization

This feature is available in tidbcloudy 1.0.8 or later.

To get the billing information of your organization, run the v1beta1_get_monthly_bill.py.

import os

import tidbcloudy

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
debug_mode = os.environ.get("TIDBCLOUDY_LOG")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
billing = api.get_monthly_bill(month="2023-10")
# billing = api.get_monthly_bill(month="202310")
# billing = api.get_current_month_bill()
print(billing)
print(billing.overview)
print(billing.summaryByProject)
print(billing.summaryByService)

Delete all resources

Warning:

This is a destructive operation. It will delete all resources in the project. DO NOT run this script in a production environment.

To delete all clusters and backup tasks in your project, run the 7_delete_resources.py.

import os

import tidbcloudy
from tidbcloudy.specification import ClusterType

public_key = os.environ.get("PUBLIC_KEY")
private_key = os.environ.get("PRIVATE_KEY")
project_id = os.environ.get("PROJECT_ID", "1234567890123456789")

api = tidbcloudy.TiDBCloud(public_key=public_key, private_key=private_key)
project = api.get_project(project_id, update_from_server=True)
for cluster in project.iter_clusters():
    print(cluster)
    if cluster.cluster_type == ClusterType.DEDICATED:
        for backup in cluster.iter_backups():
            print(backup)
            backup.delete()
    cluster.delete()

Related projects

• Go SDK: go-tidbcloud-sdk-v1 by @c4pt0r
• Official TiDB Cloud CLI: tidbcloud-cli | User documentation
• Official code samples in Go and Python: tidbcloud-api-samples