Metadata-Version: 2.5
Name: onehouse-python-sdk
Version: 0.3.0
Summary: Onehouse Python SDK — modular data-plane clients for LakeBase and beyond
Project-URL: Homepage, https://github.com/onehouseinc/onehouse-python-sdk
Project-URL: Repository, https://github.com/onehouseinc/onehouse-python-sdk
Project-URL: Issues, https://github.com/onehouseinc/onehouse-python-sdk/issues
Author-email: Onehouse Team <eng@onehouse.ai>
License: Proprietary
Keywords: auth,azure,lakebase,oauth2,oidc,onehouse,postgresql,saml
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest-mock>=3.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: lakebase
Requires-Dist: certifi>=2024.1; extra == 'lakebase'
Requires-Dist: psycopg2-binary>=2.9; extra == 'lakebase'
Provides-Extra: resources
Requires-Dist: requests>=2.28; extra == 'resources'
Description-Content-Type: text/markdown

# onehouse-python-sdk

Python SDK for connecting to [Onehouse](https://www.onehouse.ai) data-plane and control-plane services.

The base package has **zero required dependencies**. Drivers ship as optional extras so you only install what you need.

```bash
pip install onehouse-python-sdk[lakebase]    # PostgreSQL/LakeBase SQL client
pip install onehouse-python-sdk[resources]   # Control-plane REST client
```

## LakeBase

LakeBase is Onehouse's PostgreSQL-compatible managed lakehouse engine. This SDK provides a `psycopg2`-backed client that handles the browser-based and federated authentication flows LakeBase clusters require, on top of standard username/password auth.

### Installation

```bash
pip install onehouse-python-sdk[lakebase]
```

Requires Python 3.9+.

### Quickstart

```python
from onehouse_python_sdk import LakebaseClient

# Username / password
with LakebaseClient().connect(
    host="<cluster-host>",
    port=5432,
    dbname="mydb",
    user="admin",
    password="secret",
) as client:
    rows = client.fetchall("SELECT * FROM mytable WHERE id = %s", (42,))
```

### Authentication flows

| Flow | Parameters |
|------|------------|
| Username / password | `user`, `password` |
| OIDC Device Flow (Okta / Auth0) | `browser_auth=true`, `oidc_client_id`, `oidc_issuer_url`, `oidc_iam_role` |
| Azure AD OAuth2 | `browser_auth=true`, `azure_oauth_tenant_id`, `azure_oauth_client_id`, `azure_oauth_client_secret` |
| Azure Entra ID SAML | `browser_auth=true`, `azure_tenant_id`, `azure_entity_id` |
| Built-in login form | `browser_auth=true` _(default when no IdP params are set)_ |
| External redirect | `browser_auth=true`, `auth_redirect_url` |

```python
# OIDC Device Flow
client = LakebaseClient().connect(
    host="<cluster-host>",
    port=5432,
    dbname="mydb",
    browser_auth="true",
    oidc_client_id="0oaXXXXX",
    oidc_issuer_url="https://myorg.okta.com",
    oidc_iam_role="arn:aws:iam::123456789012:role/LakeBaseRole",
)

# Azure AD OAuth2
client = LakebaseClient().connect(
    host="<cluster-host>",
    port=5432,
    dbname="mydb",
    browser_auth="true",
    azure_oauth_tenant_id="your-tenant-id",
    azure_oauth_client_id="your-client-id",
    azure_oauth_client_secret="your-client-secret",
)
```

DSN string form is also supported:

```python
client = LakebaseClient().connect(
    "postgresql://<cluster-host>:5432/mydb"
    "?browser_auth=true"
    "&oidc_client_id=0oaXXXX"
    "&oidc_issuer_url=https://myorg.okta.com"
    "&oidc_iam_role=arn:aws:iam::123456789012:role/LakeBaseRole"
)
```

### Client API

All clients implement the `SqlClient` interface:

| Method | Description |
|--------|-------------|
| `connect(dsn=None, **kwargs) → self` | Establish connection, returns self for chaining |
| `execute(sql, params)` | Run a statement, return rowcount |
| `fetchall(sql, params)` | Run a query, return `list[tuple]` |
| `fetchone(sql, params)` | Run a query, return first row or `None` |
| `cursor()` | Raw cursor for advanced use |
| `raw_connection` | Underlying `psycopg2` connection |
| `close()` | Close the connection |
| `__enter__` / `__exit__` | Context manager — closes on exit |

### Notes

- **Credential caching** — auth tokens are cached for 4 minutes per connection parameters to avoid repeated browser prompts within the same process.
- **Callback port** — the local auth callback server defaults to port `8888` at path `/lakebase`. Override with `auth_callback_port` and `auth_callback_path`.

## Onehouse Resources (Control-Plane API)

`OnehouseResources` wraps the Onehouse REST API for managing platform resources — clusters, lakes, flows, jobs, table services, and more. It posts SQL statements to `https://api.onehouse.ai/v1/resource/` and polls `/v1/status/{requestId}` for the result.

### Installation

```bash
pip install onehouse-python-sdk[resources]
```

### Quickstart

```python
from onehouse_python_sdk import OnehouseResources

client = OnehouseResources(
    account_uid="...", project_uid="...",
    api_key="...", api_secret="...",
    link_uid="...", region="us-west-2", user_uid="...",
)

# Typed helper — blocks until the operation reaches a terminal status.
result = client.create_cluster(
    "prod",
    type="Managed",
    max_ocu=10,
    min_ocu=1,
    options={"worker.type": "oh-general-4"},
)
print(result.api_status)        # ApiStatus.SUCCESS
print(result.api_response)      # raw API payload
```

### Credentials

Credentials resolve from three sources, in order (highest wins):

1. **Explicit constructor arguments** (shown above).
2. **Environment variables** — `ONEHOUSE_ACCOUNT_UID`, `ONEHOUSE_PROJECT_UID`, `ONEHOUSE_API_KEY`, `ONEHOUSE_API_SECRET`, `ONEHOUSE_LINK_UID`, `ONEHOUSE_REGION`, `ONEHOUSE_USER_UID`. Optional: `ONEHOUSE_BASE_URL`, `ONEHOUSE_PROFILE`, `ONEHOUSE_CREDENTIALS_FILE`. `ONEHOUSE_BASE_URL` is the API host (default `https://api.onehouse.ai`); a trailing `/v1` is dropped, since every request adds it.
3. **INI credentials file** at `~/.onehouse/credentials` (override with `ONEHOUSE_CREDENTIALS_FILE`).

```ini
# ~/.onehouse/credentials
[default]
account_uid = 92e5f1ab-...
project_uid = 3afe72cd-...
api_key     = j+m8wRhgpKYFTLxCHNDzQA==
api_secret  = tXpzrqfUBNK9yhS5+FmLM37xwfhVeZygJntCzHG4Dpq=
link_uid    = da56fe8b-...
region      = us-west-2
user_uid    = ...

[staging]
account_uid = ...
```

```python
# Read from environment / default profile.
client = OnehouseResources()

# Pick a named profile.
client = OnehouseResources(profile="staging")
```

Missing fields produce an `AuthError` listing which fields are unset and how to supply them. A world-readable credentials file triggers a warning — `chmod 600 ~/.onehouse/credentials`.

### OAuth sign-in instead of an API key

A person can sign in through the browser instead of holding an API key. external-api runs an OAuth 2.1 authorization server (authorization code + PKCE). Its access token is a Firebase ID token for the signed-in user and lasts an hour. Its refresh token rotates on every use and lasts 30 days. The org must have `enable_external_api_bearer_auth` on.

```python
from pathlib import Path
from onehouse_python_sdk._session import FileSessionStore
from onehouse_python_sdk.resources import oauth

store = FileSessionStore(Path.home() / ".onehouse" / "session.json")
oauth.login("https://api.onehouse.ai", store)  # opens the browser, catches the code on 127.0.0.1

client = OnehouseResources(
    account_uid="...", project_uid="...", link_uid="...", region="us-west-2",
    token_source=oauth.FirebaseRefreshTokenSource("https://api.onehouse.ai", store),
)
```

- With a `token_source`, `api_key`, `api_secret` and `user_uid` are not required. Calls send `Authorization: Bearer` and never the key pair or user id: external-api takes the user from the token. `link_uid` and `region` are still required, because statements that reach a data plane need them.
- The token source refreshes 60 s before expiry and once after a 401. Refresh runs under the store's cross-process lock and re-reads the store first. Concurrent processes therefore share one rotation; presenting a rotated refresh token twice revokes the whole session.
- `oauth.logout(base_url, store)` revokes the refresh token and deletes the session. An access token already issued stays valid until it expires, at most an hour.
- `FileSessionStore` writes one `0600` file. The `oh` CLI keeps the refresh token in the OS keychain instead and falls back to this file.
- The client is the static public client `oh-cli`, registered for `http://127.0.0.1/oauth/callback` on any port.

## AI Gateway tokens

`onehouse_python_sdk.ai_gateway` gets tokens for a data plane's AI Gateway. It uses the standard library only, and `resources` never imports it. Every call needs the VPN or the customer network.

```python
from onehouse_python_sdk import ai_gateway

gateway = "https://ai-gateway-<id>.dp.onehouse.ai"

# A service principal: client_credentials with its API key pair. 300 s tokens, re-minted.
source = ai_gateway.ServicePrincipalTokenSource(gateway, api_key, api_secret)

# A person: sign in once on Keycloak's `onehouse-cli` client, then refresh.
ai_gateway.login(gateway, store)
source = ai_gateway.KeycloakSessionTokenSource(gateway, store)

token = source.get()            # send as X-Onehouse-Authorization (or Authorization on /mcp)
token = source.get(force=True)  # after the gateway answers 401
```

- `ai_gateway.discover(gateway)` follows the gateway's own discovery chain: RFC 9728 protected-resource metadata, then RFC 8414 or OIDC metadata for the issuer it names. It checks that the issuer matches and that the audience is on the gateway's host. Nothing is derived from the hostname or the link id. A gateway with auth off raises `AuthDisabledError`.
- The human session stores only the refresh token. The access token (7 days on that client) stays in memory and is never written to disk.
- `ai_gateway.logout(gateway, store)` ends the Keycloak session best effort and always deletes the stored token.

### Three ways to run a command

```python
# (1) Blocking — submit, poll, return the terminal status. Most common.
result = client.execute("SHOW CLUSTERS")

# (2) Non-blocking — submit now, poll later. Good for long-running ops or
# parallel orchestration where you don't want to hold a thread.
submitted = client.submit("CREATE CLUSTER `prod` TYPE = 'Managed' MAX_OCU = 10 MIN_OCU = 1")
# ... do other work, persist submitted.request_id ...
status = client.get_status(submitted.request_id)
while status.api_status == ApiStatus.PENDING:
    time.sleep(5)
    status = client.get_status(submitted.request_id)

# (3) Typed helpers — same blocking semantics as execute(), but build the SQL for you.
client.create_cluster("prod", type="Managed", max_ocu=10, min_ocu=1)
client.show_clusters()
client.delete_cluster("prod")
```

### Typed helpers

`OnehouseResources` exposes one method per Phase 1 SQL command (~50 methods across 11 resource families): Clusters, Lakes, Databases, Tables, Catalogs, Sources, Flows, Transformations, Validations, Table Services, Jobs, Service Principals, API Tokens. Every typed helper accepts the same trailing kwargs: `unsafe_raw`, `timeout`, `poll_interval`.

```python
client.create_lake(
    "analytics",
    lake_type="MANAGED",
    bucket_path="s3://my-bucket/lake",
    default_services_cluster="services",
)

from onehouse_python_sdk.resources.sql.commands import PartitionKeyField

client.create_flow(
    "events_pipeline",
    source="my_kafka_source",
    lake="analytics",
    database="events",
    table_name="page_views",
    write_mode="MUTABLE",
    cluster="ingest_cluster",
    catalogs=["my_glue_catalog"],
    record_key_fields=["id"],
    partition_key_fields=[
        PartitionKeyField("date", partition_type="DATE_STRING",
                          input_format="yyyy-mm-dd", output_format="yyyy-mm-dd"),
    ],
    min_sync_frequency_mins=5,
    options={"kafka.topic.name": "page_views"},
)
```

#### `CREATE FLOW` — `WITH` options reference

Source-specific options and advanced configs are passed via the `options` dict. All keys and values are strings.

**Kafka source**

| Key | Description |
|-----|-------------|
| `kafka.topic.name` | Name of the source Kafka topic. |
| `kafka.startingOffsets` | `'earliest'` (default) or `'latest'`. |

**S3 source**

| Key | Description |
|-----|-------------|
| `s3.folder.uri` | `s3://bucket/prefix/` folder to ingest from. |
| `s3.file.format` | `AVRO`, `JSON`, `CSV`, `ORC`, `PARQUET`, or `XML`. |
| `s3.file.extension` | File extension, e.g. `'.parquet'`, `'.jsonl'`. |
| `s3.source.bootstrap` | `'TRUE'` or `'FALSE'`. |
| `s3.source.if_infer_fields_from_source_path` | `'TRUE'` to extract field values from path segments like `field=value`. |
| `s3.source.fields_to_infer` | Comma-separated field names to extract from the path, e.g. `'schema,table,date'`. |
| `s3.source.file_path_pattern` | Optional glob applied under `s3.folder.uri`. |
| `s3.file.csv.header` | `'TRUE'` if the first row is a header. |
| `s3.file.json.multiline` | `'TRUE'` for single-record-per-file (pretty-printed) JSON. |
| `s3.file.xml.row.tag` | XML element that delimits one record, e.g. `'Product'`. |
| `s3.file.xml.value.tag` | Synthetic field name for element text content with attributes (default `'_VALUE'`). |
| `s3.file.xml.null.value` | String token treated as `null`. |
| `s3.file.xml.ignore.namespace` | `'TRUE'` to strip namespace prefixes from element names. |

**GCS source** — same as S3 with `gcs.` prefix (e.g. `gcs.folder.uri`, `gcs.file.format`, etc.).

**Onehouse table source**

| Key | Description |
|-----|-------------|
| `source.table.name` | Name of the source Onehouse table. |
| `source.table.database` | Database containing the source table. |
| `source.table.lake` | Lake containing the source table. |

**Postgres source**

| Key | Description |
|-----|-------------|
| `postgres.table.name` | Postgres table to ingest. |
| `postgres.schema.name` | Postgres schema name. |

**MySQL source**

| Key | Description |
|-----|-------------|
| `mysql.table.name` | MySQL table to ingest. |

**Oracle source**

| Key | Description |
|-----|-------------|
| `oracle.table.name` | Oracle table to ingest. |
| `oracle.schema.name` | Oracle schema (owner). |

**Schema registry** (Kafka sources)

| Key | Description |
|-----|-------------|
| `schema.registry.type` | `'glue'`, `'confluent'`, `'jar'`, or `'file'`. |
| `schema.registry.glue.name` | Glue registry name. |
| `schema.registry.confluent.servers` | Confluent Schema Registry URL. |
| `schema.registry.confluent.key` | Confluent API key. |
| `schema.registry.confluent.secret` | Confluent API secret. |
| `schema.registry.proto.jar.location` | S3 path to the proto schema JAR. |
| `schema.registry.file.base.path` | Base path for file-based schema registry. |
| `schema.registry.file.full.path` | Full path to a specific schema file. |

**Cross-source advanced configs**

| Key | Description |
|-----|-------------|
| `table.configured.base.path` | Custom storage location for the table (disables Clean & Restart). |
| `table.partition.style` | `'default'` or `'hive'`. |
| `flow.delayThreshold.numSyncIntervals` | Multiplied by sync frequency to set the delay threshold. `'0'` disables the Delayed state. |
| `flow.deduplicationPolicy` | `'none'` (default) or `'drop'` — deduplicates append-only flows by record key. |

```python
```

ACL / privilege / role / group commands aren't exposed as typed helpers yet — use `client.execute("GRANT ...")` until they land in a future release.

#### `unsafe_raw` escape hatch

The builder validates resource names against `^[A-Za-z][A-Za-z0-9_-]*$` and rejects unknown enum values to catch obvious typos. If a name doesn't match (e.g. legacy resources with dots in the name) or you're using a SQL feature the SDK hasn't been updated for, pass `unsafe_raw=True` on the call to skip validation:

```python
client.create_cluster("legacy.name", type="CustomType", unsafe_raw=True)
```

The argument is deliberately named so its uses are easy to find in `grep`.

### Error handling

```
OnehouseSdkError
└── ResourcesError                     # raised by the resources/ subpackage
    ├── AuthError                      # missing/invalid credentials
    ├── SqlParseError                  # HTTP 400 + grpc-message — server rejected the SQL
    ├── OperationFailedError           # terminal status FAILED or INVALID
    └── OperationTimeoutError          # polling exceeded the configured timeout
```

`OperationTimeoutError` carries the `request_id` of the in-flight operation — you can resume polling with `get_status(request_id)` rather than re-submitting (which would create a duplicate resource).

### Client API

| Method | Description |
|--------|-------------|
| `submit(statement) → SubmitResponse` | POST `/v1/resource/`, return the `requestId`. |
| `get_status(request_id) → StatusResponse` | GET `/v1/status/{id}`, return the parsed status. |
| `execute(statement, timeout=, poll_interval=) → StatusResponse` | Submit + poll until terminal. Blocks. |
| `<verb>_<resource>(...)` (~50 methods) | Typed wrappers around `execute()` that build SQL for you. |

### Notes

- **Lazy import** — `from onehouse_python_sdk import OnehouseResources` works even when the `[resources]` extra isn't installed; the first network call raises a clear "install `[resources]` extra" error.
- **Rate limit** — projects are capped at 10 QPS. The transport retries 429 responses with bounded exponential backoff.
- **Not a SqlClient** — `OnehouseResources` is a control-plane HTTP client and intentionally does not implement the `SqlClient` interface (no `cursor`, `fetchall`, etc.).
