datus-starrocks 0.1.2

StarRocks database adapter for Datus

datus-starrocks

StarRocks database adapter for Datus.

Overview

StarRocks is a high-performance analytical database that uses the MySQL protocol. This adapter extends the MySQL connector with StarRocks-specific features:

• Multi-catalog support
• Materialized views
• StarRocks-specific metadata queries

Installation

pip install datus-starrocks

This will automatically install the required dependencies:

• datus-agent
• datus-mysql (which includes datus-sqlalchemy)

Usage

The adapter is automatically registered with Datus when installed. Configure your database connection:

database:
  type: starrocks
  host: localhost
  port: 9030
  username: root
  password: your_password
  catalog: default_catalog
  database: your_database

Or use programmatically:

from datus_starrocks import StarRocksConnector

# Create connector
connector = StarRocksConnector(
    host="localhost",
    port=9030,
    user="root",
    password="your_password",
    catalog="default_catalog",
    database="mydb"
)

# Use context manager for automatic cleanup
with connector:
    # Test connection
    connector.test_connection()

    # Get catalogs
    catalogs = connector.get_catalogs()
    print(f"Catalogs: {catalogs}")

    # Get databases in catalog
    databases = connector.get_databases(catalog_name="default_catalog")
    print(f"Databases: {databases}")

    # Get tables
    tables = connector.get_tables(catalog_name="default_catalog", database_name="mydb")
    print(f"Tables: {tables}")

    # Get materialized views
    mvs = connector.get_materialized_views(database_name="mydb")
    print(f"Materialized Views: {mvs}")

    # Get materialized views with DDL
    mvs_with_ddl = connector.get_materialized_views_with_ddl(database_name="mydb")
    for mv in mvs_with_ddl:
        print(f"\n{mv['table_name']}:")
        print(mv['definition'])

    # Execute query
    result = connector.execute_query("SELECT * FROM users LIMIT 10")
    print(result.sql_return)

Features

StarRocks-Specific Features

• Multi-catalog support: Query across multiple catalogs
• Materialized views: Full support for StarRocks materialized views
• Catalog management: Switch between catalogs seamlessly

Inherited from MySQL

• Full CRUD operations (SELECT, INSERT, UPDATE, DELETE)
• DDL execution (CREATE, ALTER, DROP)
• Metadata retrieval (tables, views, schemas)
• Sample data extraction
• Multiple result formats (pandas, arrow, csv, list)
• Connection pooling and management

StarRocks-Specific Examples

Working with Catalogs

# List all catalogs
catalogs = connector.get_catalogs()

# Switch catalog
connector.switch_context(catalog_name="hive_catalog")

# Query with explicit catalog
tables = connector.get_tables(
    catalog_name="hive_catalog",
    database_name="my_hive_db"
)

Materialized Views

# Get materialized views
mvs = connector.get_materialized_views(database_name="mydb")

# Get materialized views with full DDL
mvs_with_ddl = connector.get_materialized_views_with_ddl(database_name="mydb")

for mv in mvs_with_ddl:
    print(f"Name: {mv['table_name']}")
    print(f"Database: {mv['database_name']}")
    print(f"Catalog: {mv['catalog_name']}")
    print(f"Definition: {mv['definition']}")

Fully-Qualified Names

StarRocks supports three-part names: catalog.database.table

# Build full name
full_name = connector.full_name(
    catalog_name="default_catalog",
    database_name="mydb",
    table_name="users"
)
# Result: `default_catalog`.`mydb`.`users`

# Query with full name
result = connector.execute_query(f"SELECT * FROM {full_name} LIMIT 10")

Requirements

• Python >= 3.12
• StarRocks >= 2.0
• datus-agent >= 0.2.1
• datus-mysql >= 0.1.0

Testing

Quick Start

# 1. Start StarRocks test container
docker-compose up -d && sleep 60
docker exec datus-starrocks-test mysql -h127.0.0.1 -P9030 -uroot \
  -e "CREATE DATABASE IF NOT EXISTS test;"

# 2. Run tests
./scripts/test.sh unit         # Unit tests (60 tests, ~0.03s)
./scripts/test.sh integration  # Integration tests (35 tests, ~1.5s)
./scripts/test.sh acceptance   # Acceptance tests (28 tests, CI subset)
./scripts/test.sh all          # All tests

Test Types

• Unit tests (60): Configuration and connector logic with Mocks (no database needed)
• Integration tests (35): Real database operations (catalog, materialized views, SQL)
• Acceptance tests (28): Critical functionality subset for CI/CD

Environment Variables

Tests use these default values (automatically set by ./scripts/test.sh):

• STARROCKS_HOST=localhost
• STARROCKS_PORT=9030
• STARROCKS_USER=root
• STARROCKS_PASSWORD=""
• STARROCKS_DATABASE=test

Connection Cleanup

The connector includes special handling for PyMySQL cleanup errors that can occur with StarRocks connections. Use the context manager pattern for automatic cleanup:

with StarRocksConnector(...) as connector:
    # Your code here
    pass
# Connection automatically cleaned up

License

Apache License 2.0

Related Packages

• datus-mysql - MySQL adapter (base for StarRocks)
• datus-sqlalchemy - SQLAlchemy base connector
• datus-snowflake - Snowflake adapter