Greyhorse ClickHouse library

Greyhorse framework library for ClickHouse support, built on greyhorse.strand/greyhorse.river/greyhorse.rock (the same rock/river/strand architecture as the sibling greyhorse-sqla package) and the asynch driver.

The primary API is the PIECES, not a ready-made module to subclass:

• ClickHouseFragment -- material: how to build a ClickHouseAsyncEngine (construction only, no lifecycle)
• ClickHouseBorder -- lifecycle: start/stop the engine's connection pool, and a real liveness probe (is_alive()) for health checks
• ClickHouseAccess -- access: hand out a connection or a cursor, both Shared (ClickHouse has no transactions, so there is no apply/cancel outcome to model -- see greyhorse_clickhouse/contexts.py's own module docstring)

An application lists the pieces it needs, alongside pieces from any other storage library, directly on its own Module. ClickHouseModule is a convenience wrapper bundling all three for the common single-storage case. See examples/ (start with examples/03_pieces.py) for the runnable versions of both shapes.

Usage

The first two snippets below are complete programs -- the first was run against a live ClickHouse while this section was written, and the second builds its declarations as shown. The multi-storage one is a SKETCH (note the {...}), and says so. The longer, commented versions of all three live in examples/, where the test suite executes them on every run, so those cannot rot silently.

One engine, one cursor

ClickHouseModule is the ready-made bundle for the single-storage case. The config reaches the engine's constructor through the same args={Type: value} door every greyhorse.strand resource uses -- there is no ClickHouse-specific wiring.

from greyhorse.app.private.runtime.invoke import invoke_sync
from greyhorse.run import wrap_sync
from greyhorse.strand import running

from greyhorse_clickhouse import ClickHouseCursorCtx, ClickHouseModule, EngineConf


def main() -> None:
    conf = EngineConf(dsn='clickhouse://default:@localhost:9000/default')

    with running(ClickHouseModule, args={EngineConf: conf}) as module:
        cursor_ctx = module.get(ClickHouseCursorCtx).unwrap()

        async def query() -> int:
            async with cursor_ctx as cursor:
                await cursor.execute('SELECT 1 AS one')
                row = await cursor.fetchone()
                return row['one']

        print(invoke_sync(query))


wrap_sync(main)

cursor is an ordinary asynch DictCursor, borrowed for the length of the async with block and handed back at its end. A connection is available the same way through ClickHouseConnCtx.

Everything here is async, and start() never connects

There is no sync twin: asynch is asyncio-native and this package follows it. Both products are AsyncShared -- ClickHouse has no transactions (commit()/rollback() raise NotSupportedError unconditionally), so there is no apply/cancel outcome to model and nothing to hand out as Mut.

ClickHouseAsyncEngine.start() builds the pool but opens no socket. A module over an unreachable DSN therefore comes up cleanly, and the failure surfaces at the first real borrow -- which is what lets the border's check() report the outage and drive repair instead of the whole floor refusing to start.

A consumer that knows nothing about greyhorse

The point of the split: the class that talks to ClickHouse takes a context by TYPE and imports nothing from this package. Only the component says how it is wired.

from typing import ClassVar

from greyhorse.app.private.runtime.invoke import invoke_sync
from greyhorse.strand import Component, Handle, HttpBinding, Shared, Use

from greyhorse_clickhouse import (
    ClickHouseAccess,
    ClickHouseAsyncEngine,
    ClickHouseCursorCtx,
    ClickHouseModule,
)


class PingApi:
    def __init__(self, cursor: ClickHouseCursorCtx) -> None:
        self._cursor = cursor

    def ping(self) -> int:
        return invoke_sync(self._ping)

    async def _ping(self) -> int:
        async with self._cursor as cursor:
            await cursor.execute('SELECT 1 AS one')
            row = await cursor.fetchone()
            return row['one']


class PingComponent(Component):
    imports: ClassVar = (Shared[ClickHouseAsyncEngine],)
    providers: ClassVar = (ClickHouseAccess,)
    exports: ClassVar = PingApi
    handlers: ClassVar = Handle(PingApi.ping, HttpBinding.Route(verb='GET', path=''))


class App(ClickHouseModule):
    name = 'ping-app'
    components: ClassVar = {'ping': Use(PingComponent)}

Runnable, with the gateway wiring around it, in examples/02_component.py.

Two storages on one floor

ClickHouseModule is sugar for the single-storage case. An application that needs ClickHouse next to something else does not subclass it -- it lists the pieces from each library directly on its own Module:

class App(Module):
    fragments: ClassVar = (ClickHouseFragment, CacheFragment)
    resources: ClassVar = (
        Resource(ClickHouseAsyncEngine, operators=ClickHouseBorder),
        Resource(CacheEngine, operators=CacheBorder),
    )
    produces: ClassVar = (
        Produce(ClickHouseAsyncEngine, provider=ClickHouseAccess, name='clickhouse'),
        Produce(CacheEngine, provider=CacheAccess, name='cache'),
    )
    components: ClassVar = {...}

examples/03_pieces.py is that shape, runnable, with a second storage built from the same three declarations.

Health

ClickHouseBorder.check() asks ClickHouseAsyncEngine.is_alive(), which opens a DEDICATED one-off connection and pings it -- deliberately not a borrow from the shared pool. A pool that is merely full is a local capacity condition, not evidence that the server is unreachable, and reporting the two as the same thing drives repair churn against a healthy database. The probe is bounded and its answer cached for a short cooldown, so a tick loop never stalls for the length of a real outage.

Configuration

The engine takes an EngineConf -- a DSN plus pool bounds -- handed in through args={EngineConf: ...} when the module is built:

``EngineConf(dsn='clickhouse://user:pass@host:9000/db')``

ClickHouseSettings builds that DSN from the environment instead, for deployments that configure by env var rather than in code. It reads the CH_ prefix (case-insensitive) and a .env file: CH_HOST, CH_PORT, CH_USER, CH_PASSWORD, CH_DATABASE, CH_POOL_MIN_SIZE, CH_POOL_MAX_SIZE. A whole CH_DSN may be given instead, in which case the parts are ignored.

```python
settings = ClickHouseSettings()
conf = EngineConf(
    dsn=settings.dsn, pool_min_size=settings.pool_min_size, pool_max_size=settings.pool_max_size
)
```

The pool sizes have to be carried over explicitly, as above: EngineConf is built from the DSN, and dsn=settings.dsn alone would leave the pool at EngineConf's own defaults regardless of what CH_POOL_* said.

CH_PASSWORD_FILE points at a Docker/Kubernetes-secret-style file and, when set, is AUTHORITATIVE: it overrides an inline CH_PASSWORD, and a missing, unreadable, non-UTF-8 or empty file raises at config-validation time rather than falling back to whatever CH_PASSWORD happened to hold. That is deliberate -- start() does not connect, so a silently empty credential would otherwise surface as a production outage rather than a startup failure.

Both models redact the password from repr()/str() and from rendered validation errors, so an accidental logger.info(conf) cannot print it.

ClickHouse itself has no embeddable, driver-free fallback the way SQLite backs greyhorse-sqla's own test suite -- every genuinely interesting path here needs a live server. The test suite and the examples are both split accordingly; see below.

Install

For consuming the library, from an index -- this is what most readers want:

``pip install greyhorse-clickhouse``

or, with uv:

``uv add greyhorse-clickhouse``

The compression extra adds clickhouse-cityhash, needed only for the CityHash-based codecs (asynch already ships lz4/zstd support without it):

``pip install 'greyhorse-clickhouse[compression]'``

Development (inside the greyhorse monorepo)

Everything below runs from a checkout of the greyhorse monorepo, at data/clickhouse/, and is for working on this package itself, not for consuming it. pyproject.toml's [tool.uv.sources] points greyhorse at ../../core, a path that only resolves inside that checkout -- uv sync against a standalone download (e.g. an unpacked sdist) does not work; use "Install" above instead. tests/ and examples/ referenced below are monorepo/sdist paths, not part of the installed wheel.

• Set up the project

  uv python pin 3.14

  uv venv

  uv sync

  source .venv/bin/activate

• Run the tests

  No server needed -- runs everywhere, every path that does not need a live ClickHouse to mean anything:

  uv run pytest tests -q

  With a live server -- everything, including the tests/examples that are otherwise skipped for want of one:

  docker-compose -f tests/docker-compose.yml up -d --wait

  export CLICKHOUSE_TEST_DSN='clickhouse://greyhorse:greyhorse@localhost:9000/greyhorse'

  export CLICKHOUSE_DSN='clickhouse://greyhorse:greyhorse@localhost:9000/greyhorse'

  uv run pytest tests -q

  docker-compose -f tests/docker-compose.yml down -v

  If 8123 or 9000 are already taken on this machine, set CLICKHOUSE_HTTP_PORT / CLICKHOUSE_NATIVE_PORT before up, and point both DSNs above at the same native host port.

  CLICKHOUSE_TEST_DSN gates the suite's own live tests (tests/conf.py's requires_clickhouse mark); CLICKHOUSE_DSN is what examples/*.py read when run directly or through tests/test_examples.py's live half. Every example also runs -- and is asserted -- with NO server reachable at all: that half needs no Docker and is what catches startup-level wiring drift (a renamed export, a changed Module field) before it ever reaches a live-server run. It does not cover every dispatch-time regression -- examples/README.md states exactly which example proves what.

• Format code commands

  ruff check --unsafe-fixes --fix

  ruff format