Metadata-Version: 2.5
Name: greyhorse-clickhouse
Version: 0.5.5
Summary: ClickHouse storage for greyhorse applications
Project-URL: Homepage, https://gitlab.com/max-plutonium/greyhorse
Project-URL: Repository, https://gitlab.com/max-plutonium/greyhorse
Author-email: Max Plutonium <plutonium.max@gmail.com>
Maintainer-email: Max Plutonium <plutonium.max@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: asynch,clickhouse,database,greyhorse,olap
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Database
Classifier: Topic :: Database :: Front-Ends
Classifier: Topic :: Software Development :: Libraries
Classifier: Typing :: Typed
Requires-Python: >=3.14
Requires-Dist: asynch~=0.3.1
Requires-Dist: greyhorse~=0.5.5
Requires-Dist: pydantic-settings~=2.14.2
Requires-Dist: pydantic>=2.13
Provides-Extra: compression
Requires-Dist: asynch[compression]~=0.3.1; extra == 'compression'
Description-Content-Type: text/markdown

Greyhorse ClickHouse library
=============================

Greyhorse framework library for ClickHouse support, built on
`greyhorse.strand`/`greyhorse.river`/`greyhorse.rock` (the same
rock/river/strand architecture as the sibling `greyhorse-sqla` package) and
the [`asynch`](https://github.com/long2ice/asynch) driver.

The primary API is the PIECES, not a ready-made module to subclass:

- `ClickHouseFragment` -- material: how to build a `ClickHouseAsyncEngine`
  (construction only, no lifecycle)
- `ClickHouseBorder` -- lifecycle: start/stop the engine's connection pool,
  and a real liveness probe (`is_alive()`) for health checks
- `ClickHouseAccess` -- access: hand out a connection or a cursor, both
  Shared (ClickHouse has no transactions, so there is no apply/cancel
  outcome to model -- see `greyhorse_clickhouse/contexts.py`'s own module
  docstring)

An application lists the pieces it needs, alongside pieces from any other
storage library, directly on its own `Module`. `ClickHouseModule` is a
convenience wrapper bundling all three for the common single-storage case.
See `examples/` (start with `examples/03_pieces.py`) for the runnable
versions of both shapes.

Usage
-----

The first two snippets below are complete programs -- the first was run
against a live ClickHouse while this section was written, and the second
builds its declarations as shown. The multi-storage one is a SKETCH (note the
`{...}`), and says so. The longer, commented versions of all three live in
`examples/`, where the test suite executes them on every run, so those cannot
rot silently.

### One engine, one cursor

`ClickHouseModule` is the ready-made bundle for the single-storage case. The
config reaches the engine's constructor through the same `args={Type: value}`
door every `greyhorse.strand` resource uses -- there is no ClickHouse-specific
wiring.

```python
from greyhorse.app.private.runtime.invoke import invoke_sync
from greyhorse.run import wrap_sync
from greyhorse.strand import running

from greyhorse_clickhouse import ClickHouseCursorCtx, ClickHouseModule, EngineConf


def main() -> None:
    conf = EngineConf(dsn='clickhouse://default:@localhost:9000/default')

    with running(ClickHouseModule, args={EngineConf: conf}) as module:
        cursor_ctx = module.get(ClickHouseCursorCtx).unwrap()

        async def query() -> int:
            async with cursor_ctx as cursor:
                await cursor.execute('SELECT 1 AS one')
                row = await cursor.fetchone()
                return row['one']

        print(invoke_sync(query))


wrap_sync(main)
```

`cursor` is an ordinary `asynch` `DictCursor`, borrowed for the length of the
`async with` block and handed back at its end. A connection is available the
same way through `ClickHouseConnCtx`.

### Everything here is async, and `start()` never connects

There is no sync twin: `asynch` is asyncio-native and this package follows it.
Both products are `AsyncShared` -- ClickHouse has no transactions
(`commit()`/`rollback()` raise `NotSupportedError` unconditionally), so there
is no apply/cancel outcome to model and nothing to hand out as `Mut`.

`ClickHouseAsyncEngine.start()` builds the pool but opens no socket. A module
over an unreachable DSN therefore comes up cleanly, and the failure surfaces
at the first real borrow -- which is what lets the border's `check()` report
the outage and drive repair instead of the whole floor refusing to start.

### A consumer that knows nothing about greyhorse

The point of the split: the class that talks to ClickHouse takes a context by
TYPE and imports nothing from this package. Only the component says how it is
wired.

```python
from typing import ClassVar

from greyhorse.app.private.runtime.invoke import invoke_sync
from greyhorse.strand import Component, Handle, HttpBinding, Shared, Use

from greyhorse_clickhouse import (
    ClickHouseAccess,
    ClickHouseAsyncEngine,
    ClickHouseCursorCtx,
    ClickHouseModule,
)


class PingApi:
    def __init__(self, cursor: ClickHouseCursorCtx) -> None:
        self._cursor = cursor

    def ping(self) -> int:
        return invoke_sync(self._ping)

    async def _ping(self) -> int:
        async with self._cursor as cursor:
            await cursor.execute('SELECT 1 AS one')
            row = await cursor.fetchone()
            return row['one']


class PingComponent(Component):
    imports: ClassVar = (Shared[ClickHouseAsyncEngine],)
    providers: ClassVar = (ClickHouseAccess,)
    exports: ClassVar = PingApi
    handlers: ClassVar = Handle(PingApi.ping, HttpBinding.Route(verb='GET', path=''))


class App(ClickHouseModule):
    name = 'ping-app'
    components: ClassVar = {'ping': Use(PingComponent)}
```

Runnable, with the gateway wiring around it, in `examples/02_component.py`.

### Two storages on one floor

`ClickHouseModule` is sugar for the single-storage case. An application that
needs ClickHouse next to something else does not subclass it -- it lists the
pieces from each library directly on its own `Module`:

```python
class App(Module):
    fragments: ClassVar = (ClickHouseFragment, CacheFragment)
    resources: ClassVar = (
        Resource(ClickHouseAsyncEngine, operators=ClickHouseBorder),
        Resource(CacheEngine, operators=CacheBorder),
    )
    produces: ClassVar = (
        Produce(ClickHouseAsyncEngine, provider=ClickHouseAccess, name='clickhouse'),
        Produce(CacheEngine, provider=CacheAccess, name='cache'),
    )
    components: ClassVar = {...}
```

`examples/03_pieces.py` is that shape, runnable, with a second storage built
from the same three declarations.

### Health

`ClickHouseBorder.check()` asks `ClickHouseAsyncEngine.is_alive()`, which
opens a DEDICATED one-off connection and pings it -- deliberately not a
borrow from the shared pool. A pool that is merely full is a local capacity
condition, not evidence that the server is unreachable, and reporting the two
as the same thing drives repair churn against a healthy database. The probe is
bounded and its answer cached for a short cooldown, so a tick loop never
stalls for the length of a real outage.


Configuration
-------------

The engine takes an `EngineConf` -- a DSN plus pool bounds -- handed in
through `args={EngineConf: ...}` when the module is built:

    ``EngineConf(dsn='clickhouse://user:pass@host:9000/db')``

`ClickHouseSettings` builds that DSN from the environment instead, for
deployments that configure by env var rather than in code. It reads the
`CH_` prefix (case-insensitive) and a `.env` file: `CH_HOST`, `CH_PORT`,
`CH_USER`, `CH_PASSWORD`, `CH_DATABASE`, `CH_POOL_MIN_SIZE`,
`CH_POOL_MAX_SIZE`. A whole `CH_DSN` may be given instead, in which case
the parts are ignored.

    ```python
    settings = ClickHouseSettings()
    conf = EngineConf(
        dsn=settings.dsn, pool_min_size=settings.pool_min_size, pool_max_size=settings.pool_max_size
    )
    ```

The pool sizes have to be carried over explicitly, as above: `EngineConf` is
built from the DSN, and `dsn=settings.dsn` alone would leave the pool at
`EngineConf`'s own defaults regardless of what `CH_POOL_*` said.

`CH_PASSWORD_FILE` points at a Docker/Kubernetes-secret-style file and,
when set, is AUTHORITATIVE: it overrides an inline `CH_PASSWORD`, and a
missing, unreadable, non-UTF-8 or empty file raises at config-validation
time rather than falling back to whatever `CH_PASSWORD` happened to hold.
That is deliberate -- `start()` does not connect, so a silently empty
credential would otherwise surface as a production outage rather than a
startup failure.

Both models redact the password from `repr()`/`str()` and from rendered
validation errors, so an accidental `logger.info(conf)` cannot print it.

ClickHouse itself has no embeddable, driver-free fallback the way SQLite
backs `greyhorse-sqla`'s own test suite -- every genuinely interesting path
here needs a live server. The test suite and the examples are both split
accordingly; see below.


Install
-------

For consuming the library, from an index -- this is what most readers want:

    ``pip install greyhorse-clickhouse``

or, with `uv`:

    ``uv add greyhorse-clickhouse``

The `compression` extra adds `clickhouse-cityhash`, needed only for the
CityHash-based codecs (`asynch` already ships lz4/zstd support without it):

    ``pip install 'greyhorse-clickhouse[compression]'``


Development (inside the `greyhorse` monorepo)
-----------------------------------------------

Everything below runs from a checkout of the `greyhorse` monorepo, at
`data/clickhouse/`, and is for working on this package itself, not for
consuming it. `pyproject.toml`'s `[tool.uv.sources]` points `greyhorse` at
`../../core`, a path that only resolves inside that checkout -- `uv sync`
against a standalone download (e.g. an unpacked sdist) does not work; use
"Install" above instead. `tests/` and `examples/` referenced below are
monorepo/sdist paths, not part of the installed wheel.

- Set up the project

    ``uv python pin 3.14``

    ``uv venv``

    ``uv sync``

    ``source .venv/bin/activate``

- Run the tests

  **No server needed** -- runs everywhere, every path that does not need a
  live ClickHouse to mean anything:

    ``uv run pytest tests -q``

  **With a live server** -- everything, including the tests/examples that are
  otherwise skipped for want of one:

    ``docker-compose -f tests/docker-compose.yml up -d --wait``

    ``export CLICKHOUSE_TEST_DSN='clickhouse://greyhorse:greyhorse@localhost:9000/greyhorse'``

    ``export CLICKHOUSE_DSN='clickhouse://greyhorse:greyhorse@localhost:9000/greyhorse'``

    ``uv run pytest tests -q``

    ``docker-compose -f tests/docker-compose.yml down -v``

  If 8123 or 9000 are already taken on this machine, set
  `CLICKHOUSE_HTTP_PORT` / `CLICKHOUSE_NATIVE_PORT` before `up`, and point
  both DSNs above at the same native host port.

  `CLICKHOUSE_TEST_DSN` gates the suite's own live tests (`tests/conf.py`'s
  `requires_clickhouse` mark); `CLICKHOUSE_DSN` is what `examples/*.py` read
  when run directly or through `tests/test_examples.py`'s live half. Every
  example also runs -- and is asserted -- with NO server reachable at all:
  that half needs no Docker and is what catches startup-level wiring drift
  (a renamed export, a changed `Module` field) before it ever reaches a
  live-server run. It does not cover every dispatch-time regression --
  `examples/README.md` states exactly which example proves what.

- Format code commands

    ``ruff check --unsafe-fixes --fix``

    ``ruff format``
