Metadata-Version: 2.4
Name: kawa-dsl
Version: 2.0.0
Summary: Terraform-like CLI for managing KAWA workspaces as code
Author-email: Kawa Analytics <emmanuel@kawa.ai>
License: MIT License
        
        Copyright (c) 2026 Kawa Analytics
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://www.kawa.ai
Keywords: kawa,dsl,data,dashboard,etl,workflow,cli
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: kywy==0.35.0b1
Requires-Dist: tomli-w>=1.0
Requires-Dist: python-dotenv>=1.0
Requires-Dist: pyyaml>=6.0
Dynamic: license-file

# kawa-dsl

Terraform-like CLI for managing KAWA workspaces as code — dashboards, scripts, datasources, workflows, and agents are defined in TOML files under `dsl/` and pushed to KAWA via `kawa commit`.

## Install

```bash
pip install --pre kawa-dsl
```

The `--pre` flag is required because the CLI depends on `kywy==0.35.0b1` (a pre-release).

After installation, `kawa` is available on your PATH:

```bash
kawa --help
```

## Configure

The CLI talks to your KAWA server using two environment variables. Set them in your shell, or drop a `.env` file in the directory you run `kawa` from — `python-dotenv` picks it up automatically.

| Variable | Required | Description |
|---|---|---|
| `KAWA_API_URL` | yes | Base URL of your KAWA server, e.g. `https://kawa.mycompany.com`. No trailing slash. |
| `KAWA_API_KEY` | yes | API key from the KAWA UI → *Settings → API keys*. Sent as `Authorization: Bearer <key>`. |
| `KAWA_DSL_HOME` | no | Where local config lives. Defaults to `./.kawa` (cwd-relative). |

Example `.env`:

```env
KAWA_API_URL=https://kawa.mycompany.com
KAWA_API_KEY=ka_live_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```

## Quick start

```bash
mkdir my-workspace && cd my-workspace
kawa init                          # creates .kawa/ in cwd
kawa checkout <workspace_id>       # clones the workspace into ./dsl/
# edit files under dsl/
kawa commit                        # pushes changes back to KAWA
```

`kawa --help` lists every subcommand. Common ones: `pull`, `commit`, `refresh`, `status`, `run`, `app list/create/attach`, `providers list-databases/list-tables/test-query`.

---

## Local development

### Prerequisites

- Python 3.12+
- Node.js 22+
- [OpenCode CLI](https://opencode.ai) installed (`curl -fsSL https://opencode.ai/install | bash`)
- A KAWA API key

### Configuration

Create a `.env` file in your workspace directory:

```env
KAWA_API_URL=https://try.kawa.ai
KAWA_API_KEY=your-api-key
KAWA_WORKSPACE=1
```

For the chat frontend, create `chat/.env` with the same values plus your Anthropic key (used for the AI dataset/field/block suggestions):

```env
KAWA_API_URL=https://try.kawa.ai
KAWA_API_KEY=your-api-key
KAWA_WORKSPACE=1
ANTHROPIC_API_KEY=sk-ant-...
```

The Vite dev server injects `KAWA_API_KEY` into proxied `/authentication` requests automatically, so the KAWA iframe authenticates without a manual login.

### Starting the services

You need two terminals:

**Terminal 1 — OpenCode server** (API on port 4096):

```bash
cd <your-workspace-directory>
opencode serve --port 4096 --hostname 127.0.0.1
```

**Terminal 2 — Chat frontend** (Vite dev server on port 5173):

```bash
cd chat
npm install   # first time only
npm run dev
```

Then open http://localhost:5173. No nginx required — Vite handles everything locally.

### How it works without nginx

In Docker, nginx reverse-proxies between the chat SPA, OpenCode, kawa-bridge, and the KAWA backend. For local dev, the Vite dev server replaces all of this:

| Route | Docker (nginx) | Local (Vite) |
|---|---|---|
| `/api/*` | Proxy → OpenCode :4199 | Proxy → OpenCode :4096 |
| `/authentication` | Proxy → KAWA backend | Proxy → KAWA with API key injection |
| `/kawa/*` | Proxy → KAWA backend (HTML rewriting, cookie mirroring) | Not needed (iframe uses direct KAWA URL) |
| `/internal/suggest-*` | Proxy → kawa-bridge :4097 | Vite plugin (calls Anthropic API directly) |
| `/internal/workspace/status` | Proxy → kawa-bridge | Vite plugin (in-memory state) |
| `/internal/workspace` | Proxy → kawa-bridge | Vite plugin (in-memory state) |
| `/internal/get-model`, `/set-model` | Proxy → kawa-bridge (reads/writes opencode.json) | Vite plugin (in-memory state) |
| `/internal/get-plan`, `/save-plan` | Proxy → kawa-bridge (reads/writes plan.md) | Vite plugin (in-memory state) |
| `/internal/data-environment`, `/datasources`, `/upload-file`, `/save-file`, `/extraction-metadata`, `/create-*-datasource` | Proxy → kawa-bridge | Vite plugin (proxies to KAWA with API key) |

### How authentication works

Locally, auth uses the `KAWA_API_KEY` from `.env`. The `kawa` CLI reads `.env` via `python-dotenv` and tries two credentials in order:

1. `KAWA_API_KEY` — API key auth, used by pip installs and local dev
2. `$KAWA_DSL_HOME/.token` — OIDC id_token written by the bridge after browser login (Docker/prod only)

The chat frontend's Vite proxy (`chat/vite.config.js`) forwards `/authentication` requests to KAWA with `x-kawa-api-key` and `x-kawa-workspace-id` headers injected, so the embedded KAWA iframe authenticates transparently.

## Autoimprove — Autonomous ML Experiment Loop

`kawa autoimprove` runs an autonomous ML research loop: AI agents write model code, KAWA executes it on your data, metrics are collected, and only improvements are kept. The best model is always the starting point for the next round.

```bash
kawa autoimprove init                          # interactive setup (sheet, target, metric, slots)
kawa autoimprove run --max=30 --model=sonnet   # run for 30 minutes with parallel slots
```

How it works:
- **N parallel slots** — each slot independently cycles: AI writes a `build_and_predict()` function → assembles into a KAWA script → pushes → runs workflow → collects F1/precision/recall/confusion matrix
- **History-driven** — every run's results (including failures) feed into the next agent's prompt, so it learns what works and what doesn't
- **Ratchet pattern** — only improvements are kept, failed approaches are reverted, the best model file is preserved at `/tmp/autoimprove_best_<name>.py`
- **Continuous** — slots don't wait for each other; as soon as one finishes, a new agent is spawned for that slot

Key files:
- `dsl/autoimprove.py` — the CLI implementation (`init` + `run`)
- `context/.opencode/skills/ml-research/SKILL.md` — skill for agent-driven interactive mode
- `context/.opencode/commands/research.md` — command for interactive loop in opencode

## E2E tests

The test suite (`tests/test_e2e.py`) runs 48 tests against a live KAWA instance. Each run creates a fresh workspace, runs all tests, then archives it.

### Prerequisites

- KAWA running at `http://localhost:8090`
- `KAWA_LOCAL_API_KEY` environment variable set (typically via `~/.bash_profile`)

### Running the tests

```bash
source ~/.bash_profile   # loads KAWA_LOCAL_API_KEY
python3.12 tests/test_e2e.py
```

### What it covers

- **Scripts & datasources**: push, sync, loading modes
- **Dashboard widgets**: table (fields, group_by, filters, duplicate columns), indicator, bar, pie, line, scatter, boxplot, text — with all chart parameters (color, time_sampling, area, doughnut, show_values, show_labels, multi-series)
- **Conditional formatting**: single color (text values, number range, target_columns, apply_to_all), color scale (2-point, 3-point)
- **Computed columns**: lookup columns with query verification, formula columns (creation only — KAWA bug prevents querying)
- **Workflows**: all task types (COMPUTE, PYTHON, ETL, EMAIL, CHART, IF_ELSE, GENERATE_OUTPUT), all binding types (static, task_property, task_column_aggregation), multi-step chains, IF/ELSE IF/ELSE branching, edit roundtrip, workflow execution with polling
- **Agents**: create, update, delete, linked workflows, capabilities
- **Query DSL**: select, group_by, filter, order_by, limit, no_limit, multi-aggregation with aliases, datasource queries
- **Integration**: script → datasource → sheet → lookup → query, Google Sheet CSV loading, 2-script merge → ETL → query with data verification

### Test isolation

Each test run creates a dedicated workspace (`e2e-<timestamp>`) and archives it on exit — even on error or interrupt (via `atexit` + signal handlers). Tests are idempotent within a run.

## Docker

The Docker image runs the full stack: nginx reverse proxy, OpenCode AI agent, kawa-bridge, and the SolidJS chat frontend.

### Build

```bash
./build.sh            # tags as kawa-dsl:latest
./build.sh v2.0.0     # custom tag
```

### Run locally

```bash
docker run -p 8090:80 \
  -e KAWA_URL=https://try.kawa.ai \
  -e KAWA_WORKSPACE=1 \
  -e ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} \
  -e UNSPLASH_ACCESS_KEY=${UNSPLASH_ACCESS_KEY} \
  kawa-dsl
```

Then open http://localhost:8090. The KAWA iframe handles authentication — no API key needed.

### Docker Compose example

```yaml
services:
  kawa-cobuilder:
    image: kawa-dsl:latest
    ports:
      - 8090:80
    environment:
      KAWA_URL: https://try.kawa.ai
      KAWA_WORKSPACE: 1
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
      UNSPLASH_ACCESS_KEY: ${UNSPLASH_ACCESS_KEY}
```

```bash
ANTHROPIC_API_KEY=sk-ant-... docker compose up
```

### Environment variables

| Variable | Required | Description |
|---|---|---|
| `KAWA_URL` | Yes | KAWA instance URL (e.g. `https://try.kawa.ai`) |
| `KAWA_WORKSPACE` | No | Workspace ID (default: `1`) |
| `ANTHROPIC_API_KEY` | Yes | Powers the OpenCode AI agent and LLM suggestions |
| `UNSPLASH_ACCESS_KEY` | No | Unsplash API key for photo downloads (free at [unsplash.com/developers](https://unsplash.com/developers)) |

Authentication to KAWA is handled automatically: the user logs in via the `/kawa/` iframe, and nginx captures the session cookie for backend use.

### Architecture

The container runs three services via supervisord:

| Port | Service | Role |
|---|---|---|
| 80 (exposed) | nginx | Reverse proxy, SPA host, KAWA proxy |
| 4096 (internal) | OpenCode | AI agent server (DSL commands via SSE) |
| 4097 (internal) | kawa-bridge | Workspace management, LLM suggestions, file uploads |
