seahorse-coral

gRPC-native Python client library for SeahorseDB via Coral.

The low-level Python surface is intentionally:

• Arrow-first for tabular reads (scan, search, hybrid)
• typed-model-first for table metadata, ingest/export, and index status results
• explicit about post-processing (to_pyarrow(), to_pandas(), to_polars(), to_json())

Cluster administration is intentionally outside this package. Node inspection, active-set flush, segment retry/status, rebalance, placement, and runtime tuning remain available through Coral's operational interfaces and the internal Rust coral-client.

Additional docs:

• Build from source: docs/build.md
• Release to PyPI: docs/release.md
• Advanced usage: docs/advanced.md
• Compatibility policy: docs/compatibility.md

Quickstart (Recommended)

This project intentionally supports multiple ways to define a schema (preset / components / builder). To keep onboarding simple, we recommend starting with the preset schema and only moving to Advanced when you need customization.

1) Create a Coral client

import seahorse_coral as sc

coral = sc.Coral("http://localhost:8080")

Coral and AsyncCoral use the same gRPC-native contract as the Rust client.

2) Create a table (preset: id + vector + metadata)

import seahorse_coral as sc

schema = sc.default_vector_table_schema(
    dim=384,
    # Optional:
    # id_type=sc.ScalarType.STRING,
)

table = coral.create_table("documents", schema=schema)

Table mutation and index readiness helpers return typed Python models:

counts = table.indexed_row_count()  # queryable reader-side counts
print(counts.total_row_count)
print(counts.indexed_counts)

table.update_rows("metadata = '{\"source\":\"updated\"}'", where="id = 1")
table.delete_rows(where="id = 2")

The preset creates:

• id: INT64 data column by default. SeahorseDB primary-key columns use the legacy composite value format; the SaaS primary path is Arrow LargeUtf8/LARGE_STRING even though Coral's public scalar API names it STRING.
• vector: dense vector column
• metadata: STRING (nullable; store JSON-encoded strings if you want structured metadata)

(Optional) Schema building (SchemaBuilder)

If you need customization (more columns, segmentation, multiple indexes, etc.), build a schema explicitly.

A) Create table with components (no SchemaBuilder object)

import seahorse_coral as sc

table = coral.create_table(
    "documents",
    columns=[
        sc.int64_column("id", nullable=False),
        sc.vector_column("vector", dim=384),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[sc.hnsw_index("vector")],  # List[IndexDefinition]
)

Only single-column hash segmentation is supported by the distributed Python SDK. Value, hierarchical, and composite segmentation are intentionally not exposed.

B) SchemaBuilder (constructor style)

import seahorse_coral as sc

schema = sc.SchemaBuilder(
    columns=[
        sc.int64_column("id", nullable=False),
        sc.vector_column("vector", dim=384),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[sc.hnsw_index("vector", space=sc.IndexSpace.COSINE)],
)

table = coral.create_table("documents", schema=schema)

C) SchemaBuilder (fluent / chain style)

import seahorse_coral as sc

schema = (
    sc.SchemaBuilder()
    .int64("id", nullable=False)
    .vector("vector", dim=384)
    .metadata()
    .with_primary_key("id")
    .hnsw("vector", space=sc.IndexSpace.COSINE)
)

table = coral.create_table("documents", schema=schema)

3) Insert rows

import json

table.insert_rows(
    [
        {"id": 1, "vector": [0.1, 0.2, 0.3], "metadata": json.dumps({"source": "a"})},
        {"id": 2, "vector": [0.2, 0.1, 0.0], "metadata": json.dumps({"source": "b"})},
    ]
)

(Optional) More insert options

Write APIs are explicit by mode.

import seahorse_coral as sc

# 1) JSONL string (each line is a JSON object)
jsonl = (
    '{"id": 4, "vector": [0.4, 0.4, 0.4], "metadata": "{}"}\n'
    '{"id": 5, "vector": [0.5, 0.5, 0.5], "metadata": "{}"}\n'
)
table.insert_jsonl(jsonl)

# 2) Local Parquet file
# - client converts Parquet -> Arrow IPC stream -> gRPC upload stream
table.insert_parquet("./data/documents.parquet", batch_size=8192)

# 3) Single remote Parquet file
# - server reads the object directly
table.insert_parquet(
    sc.s3_file(
        "path/to/documents.parquet",
        bucket="my-bucket",
        access_key="YOUR_ACCESS_KEY",
        secret_key="YOUR_SECRET_KEY",
        region="ap-northeast-2",
    ),
    options=sc.ImportOptions(reader_batch_size=8192),
)

# 4) Multi-file import from S3
request = sc.s3_file(
    ["path/to/a.parquet", "path/to/b.parquet"],
    bucket="my-bucket",
    access_key="YOUR_ACCESS_KEY",
    secret_key="YOUR_SECRET_KEY",
    region="ap-northeast-2",
)
options = sc.ImportOptions(
    format=sc.FileFormat.PARQUET,
    reader_batch_size=8192,  # reader record batch size
    max_concurrent_files=4,  # optional
)
result = table.import_files(request, options=options)
print(result.total_inserted_row_count)

# `import_files()` is strict by default.
# If any file fails, sc.PartialImportError or sc.ImportFilesError is raised
# and the exception carries the same ImportFilesResult via `.result`.

# 5) Arrow IPC stream bytes (advanced)
# - bytes, pyarrow.Table, pyarrow.RecordBatch, and list[RecordBatch] are supported
table.insert_arrow(arrow_ipc_bytes)

UPSERT full rows

Table.upsert() and AsyncTable.upsert() accept the same Arrow-compatible values as insert_arrow(): Arrow IPC bytes, pyarrow.Table, pyarrow.RecordBatch, or a same-schema sequence of record batches.

import pyarrow as pa
import seahorse_coral as sc

upsert_table = coral.create_table(
    "upsert_documents",
    columns=[
        sc.string_column("id", nullable=False),
        sc.vector_column("vector", dim=3),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[sc.hnsw_index("vector")],
)

separator = "\x1e"
rows = pa.table(
    {
        "id": pa.array(
            [f"document{separator}1", f"document{separator}2"],
            type=pa.large_string(),
        ),
        "vector": pa.FixedSizeListArray.from_arrays(
            pa.array([0.1, 0.2, 0.3, 0.3, 0.2, 0.1], type=pa.float32()),
            3,
        ),
        "metadata": pa.array(
            ['{"source":"refresh"}', '{"source":"new"}'],
            type=pa.large_string(),
        ),
    }
)

result = upsert_table.upsert(rows)
print(result.upserted_row_count)
print(result.inserted_row_count)
print(result.replaced_row_count)

UPSERT is available only for primary-key tables and every input row must contain the full table schema. It accepts Arrow input only; JSONL, dictionaries, Parquet paths, and partial field updates are not supported. LastWins applies only to duplicate primary keys within one Writer apply. One request can be split into concurrently scheduled applies, so duplicate keys in separate record batches have no request-wide input-order guarantee. Send each primary key only once per request when deterministic ordering is required.

Coral uses the same streaming validation, incoming-row segment routing, bounded dispatch, and partial-failure boundary as BatchInsert. Primary-key lookup and replacement are local to the destination segment. Moving a primary key to a different segmentation value does not remove the old segment's row. If a later slice fails, earlier slices can remain applied and the call returns an error without partial success counts or rollback.

The SDK does not retry UPSERT automatically and does not provide exactly-once request semantics. After a timeout or connection loss, the mutation may already have succeeded. A caller may resend the same full-row payload when logical value convergence is acceptable, but the inserted/replaced counts, tombstones, physical rows, and WAL records can differ. During rollout, enable UPSERT traffic only after every current and failover Writer supports UPSERT replay, and never roll back to an older Writer after the first UPSERT WAL record. See the UPSERT WAL rollout contract.

4) Search (dense)

# Dense vector search
#
# Note:
# - `index` is the index name, typically the same as the vector column name.
vec = table.index("vector")

result = vec.search([0.1, 0.2, 0.3], top_k=10)

result = vec.search(
    [0.1, 0.2, 0.3],
    top_k=10,
    ef_search=128,
    select="id, metadata, distance",
    where="id > 0",
    read_visibility="checkpoint",
)

read_visibility defaults to "latest". A partition, implicit whole-segment, or open-ended set scope includes the complete durable WAL prefix captured for each selected Reader segment; an explicit bounded set-only scope remains checkpoint-backed because it does not own the open tail. Use "checkpoint" to read only the checkpoint currently served by each Reader and avoid WAL replay. This is a segment-local visibility boundary, not a table-global atomic snapshot. See the compatibility policy before enabling explicit checkpoint during a rolling upgrade.

5) Consume Arrow-first tabular results

scan() and search() return ResultSet.

result = table.scan(select="id, metadata", limit=100)

# Low-level Arrow-native access
batches = result.to_record_batches()
arrow_table = result.to_pyarrow()

# Explicit convenience conversions
rows = result.to_json()
df = result.to_pandas()

For batch vector search, use ResultSets.

results = vec.search_batch([[0.1, 0.2, 0.3], [0.3, 0.2, 0.1]], top_k=10)

for result in results:
    print(result.to_pyarrow())

Large-result paths are exposed separately.

for batch in table.scan_stream(select="id, metadata"):
    process(batch)

for result in vec.search_batch_stream([[0.1, 0.2, 0.3], [0.3, 0.2, 0.1]], top_k=10):
    process(result.to_pyarrow())

6) Bootstrap schema from a parquet file

Use schema_from_parquet() when you want to start from an existing parquet layout and then adjust the schema before table creation. A plain string path is read from the client machine, and remote/object-store sources should be passed as FileSource.

import seahorse_coral as sc

schema = coral.schema_from_parquet("./data/documents.parquet")
schema.with_primary_key("id")

table = coral.create_table("documents_from_parquet", schema=schema)

7) Export or download parquet

export_parquet() writes files on the Coral server side. download_parquet() and download_parquet_stream() bring the result back to the client process, and local disk writes stay explicit via write_to() or download_parquet_to().

import seahorse_coral as sc

result = table.export_parquet(
    sc.local_directory("/var/lib/coral/exports/documents"),
    where="id > 100",
    mode="single_file",
)
print(result.files)

downloaded = table.download_parquet(limit=1000)
print(downloaded.filename)
downloaded.write_to("./documents-sample.parquet")

table.download_parquet_to("./documents-full.parquet", where="id > 100")

(Optional) Sparse & hybrid search

Sparse/hybrid search requires a table that has a sparse vector column + inverted index.

import seahorse_coral as sc

schema = sc.SchemaBuilder(
    columns=[
        sc.int64_column("id", nullable=False),
        sc.vector_column("vector", dim=384),
        sc.sparse_vector_column("sparse_emb"),
        sc.metadata_column("metadata"),
    ],
    primary_key=["id"],
    indexes=[
        sc.hnsw_index("vector"),
        sc.inverted_index("sparse_emb"),
    ],
)

table = coral.create_table("documents_hybrid", schema=schema)

# Sparse vector search (BM25 / inverted index)
sparse_query = "1:0.8 5:0.6 12:0.4"
result = table.index("sparse_emb").search_sparse(
    sparse_query,
    top_k=10,
    bm25_k=1.2,
    bm25_b=0.75,
)

# Hybrid search (dense + sparse + fusion)
# - requires dense_column + sparse_column
result = table.hybrid_search(
    dense_column="vector",
    dense_query=[0.1, 0.2, 0.3],
    sparse_column="sparse_emb",
    sparse_query=sparse_query,
    top_k=10,
    options=sc.HybridSearchOptions(
        fusion="rrf",
        rrf_k=60,
        alpha=0.7,
    ),
)

Next steps

• Advanced schema options (segmentation / indexes): docs/advanced.md
• Build from source / development: docs/build.md