Metadata-Version: 2.4
Name: ytsaurus-clickhouse-proxy
Version: 0.1.0
Summary: FastAPI proxy that exposes YTsaurus CHYT as a ClickHouse HTTP/JDBC endpoint for DBeaver, DataGrip and other SQL clients
Author-email: Alexey Voronko <klipefrem@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/klipbn/ytsaurus_clickhouse_proxy
Project-URL: Repository, https://github.com/klipbn/ytsaurus_clickhouse_proxy
Project-URL: Bug Tracker, https://github.com/klipbn/ytsaurus_clickhouse_proxy/issues
Keywords: ytsaurus,clickhouse,chyt,jdbc,dbeaver,datagrip,proxy,fastapi
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: Internet :: WWW/HTTP :: HTTP Servers
Classifier: Framework :: FastAPI
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.115.0
Requires-Dist: uvicorn[standard]>=0.30.0
Requires-Dist: httpx>=0.27.0
Dynamic: license-file

<div align="center">

# ytsaurus-clickhouse-proxy

**A lightweight FastAPI proxy that makes [YTsaurus](https://ytsaurus.tech) CHYT look like a native ClickHouse endpoint — so DBeaver, DataGrip and any JDBC client just works.**

[![PyPI version](https://badge.fury.io/py/ytsaurus-clickhouse-proxy.svg)](https://badge.fury.io/py/ytsaurus-clickhouse-proxy)
[![Python Versions](https://img.shields.io/pypi/pyversions/ytsaurus-clickhouse-proxy)](https://pypi.org/project/ytsaurus-clickhouse-proxy/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Downloads](https://pepy.tech/badge/ytsaurus-clickhouse-proxy)](https://pepy.tech/project/ytsaurus-clickhouse-proxy)
[![FastAPI](https://img.shields.io/badge/FastAPI-0.115+-009688.svg?logo=fastapi)](https://fastapi.tiangolo.com)
[![Docker](https://img.shields.io/badge/Docker-ready-2496ED.svg?logo=docker)](https://hub.docker.com)

</div>

---

## What it does

YTsaurus ships with **CHYT** — a ClickHouse-over-YT engine — but its HTTP interface isn't perfectly compatible with JDBC drivers.  
`ytsaurus-clickhouse-proxy` sits in front of CHYT and:

- Intercepts `system.tables`, `system.columns`, `system.databases` queries and returns a **virtual catalog** you define in a single Python file
- Rewrites friendly table aliases (`analytics.orders`) into real YT paths (`//home/my_project/prod/orders`) transparently
- Handles both single tables and **date-partitioned directories** (`concatYtTablesRange`)
- Strips incompatible query parameters (e.g. `compress=1`) before forwarding
- Works out of the box with **DBeaver**, **DataGrip**, and any other JDBC ClickHouse client

```
DBeaver / DataGrip
       │  JDBC (ClickHouse driver)
       ▼
ytsaurus-clickhouse-proxy  :8123
       │  Rewrites queries · Injects virtual catalog
       ▼
YTsaurus CHYT clique
       │
       ▼
   YT Tables
```

---

## Installation

```bash
pip install ytsaurus-clickhouse-proxy
```

Or from source:

```bash
git clone https://github.com/klipbn/ytsaurus_clickhouse_proxy.git
cd ytsaurus-clickhouse-proxy
pip install -e .
```

---

## Quick Start

### 1. Configure your table catalog

Edit `yt_clickhouse_proxy/logical_catalog.py` and describe which YT tables you want to see in your IDE:

```python
LOGICAL_CATALOG = {
    # Schema name visible in DBeaver
    "analytics": {
        # Alias → single YT table
        "orders": {
            "type": "table",
            "path": "//home/my_project/prod/orders",
        },
        # Alias → date-partitioned directory
        "events_1d": {
            "type": "range",
            "base": "//home/my_project/logs/events/1d",
            "sample": "2024-01-01",   # used for schema introspection
        },
    },
}
```

### 2. Set environment variables

```bash
export YT_PROXY=http://your-yt-cluster.example.com
export YT_TOKEN_PATH=~/.yt/token       # file containing your OAuth token
export CLIQUE_ALIAS=ch_public          # CHYT clique name
```

### 3. Run

```bash
ytsaurus-clickhouse-proxy
# or
python -m yt_clickhouse_proxy
# or with uvicorn directly
uvicorn yt_clickhouse_proxy:app --host 0.0.0.0 --port 8123
```

---

## Connect from DBeaver

| Field | Value |
|---|---|
| **Driver** | ClickHouse |
| **JDBC URL** | `jdbc:clickhouse://127.0.0.1:8123/analytics` |
| **Username** | your CHYT clique alias (e.g. `ch_public`) |
| **Password** | your YT OAuth token |
| **Driver property** `compress` | `false` |
| **Driver property** `check_table_existence` | `false` |

<img src="img/dbeaver.jpg" alt="DBeaver connection settings" width="600">

Once connected, you'll see your logical schemas and tables in the navigator:

<img src="img/query.jpg" alt="Running a query in DBeaver" width="600">

---

## Environment Variables

| Variable | Default | Description |
|---|---|---|
| `YT_PROXY` | *(required)* | YTsaurus cluster HTTP address |
| `YT_TOKEN_PATH` | `~/.yt/token` | Path to file containing your OAuth token |
| `CLIQUE_ALIAS` | `ch_public` | CHYT clique alias |
| `DEFAULT_FORMAT` | `RowBinaryWithNamesAndTypes` | ClickHouse wire format |
| `MAX_TABLES_IN_COLUMNS_UNION` | `100` | Max tables fetched in a single `system.columns` request |

---

## Docker

### Build and run

```bash
docker build -t ytsaurus-clickhouse-proxy .

docker run -d \
  --name ytsaurus-clickhouse-proxy \
  -p 8123:8123 \
  -v ~/.yt/token:/root/.yt/token:ro \
  -e YT_PROXY=http://your-yt-cluster.example.com \
  -e CLIQUE_ALIAS=ch_public \
  ytsaurus-clickhouse-proxy
```

### With Docker Compose

```bash
# edit YT_PROXY in docker-compose.yml first
docker compose up -d
docker compose logs -f
```

---

## Example SQL Queries

```sql
-- List all tables in your catalog
SELECT name, engine, database
FROM system.tables
WHERE database = 'analytics';

-- Inspect column types (JDBC metadata)
SELECT TABLE_NAME, COLUMN_NAME, TYPE_NAME, DATA_TYPE, ORDINAL_POSITION
FROM system.columns
WHERE TABLE_NAME = 'orders'
ORDER BY ORDINAL_POSITION;

-- Query data via alias
SELECT date, count() AS cnt
FROM analytics.events_1d
WHERE date >= '2024-01-01'
GROUP BY date
ORDER BY date;

-- Or pass a raw YT path directly (CHYT native syntax)
SELECT *
FROM concatYtTablesRange('//home/my_project/logs/events/1d', '2024-01-01', '2024-01-31')
LIMIT 100;
```

---

## Logical Catalog Reference

```python
LOGICAL_CATALOG = {
    "<schema_name>": {
        "<table_alias>": {
            # ── Option A: single table ──────────────────────────────────
            "type": "table",
            "path": "//home/my_project/some_table",

            # ── Option B: date-partitioned directory ────────────────────
            "type": "range",
            "base": "//home/my_project/logs/events/1d",
            "sample": "2024-01-01",   # one valid partition for DESCRIBE
        },
    },
}
```

- `type: "table"` — proxied as `` SELECT … FROM `//path` ``
- `type: "range"` — proxied as `SELECT … FROM concatYtTablesRange('base', …)` with the full date range passed by the caller

---

## Known Limitations

- `system.columns` **without a table filter** returns a limited result set to avoid hitting CHYT's `max_query_size` limit
- No authentication layer — the proxy inherits your YT token; do not expose port 8123 publicly without a firewall
- `compress=1` from JDBC is silently dropped (CHYT handles its own transport compression)

---

## Contributing

Pull requests are welcome! Please open an issue first to discuss what you'd like to change.

```bash
git clone https://github.com/klipbn/ytsaurus_clickhouse_proxy.git
cd ytsaurus-clickhouse-proxy
pip install -e ".[dev]"
```

---

## License

MIT © 2026 Alexey Voronko
