Metadata-Version: 2.4
Name: sangam
Version: 0.1.22
Summary: Intelligent type-checking based code analysis and generation CLI tool
Author: sangam team
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Code Generators
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: mypy>=0.900
Requires-Dist: flask>=2.0
Requires-Dist: rich>=13.0
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: license
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# sangam

> An **LLM harness** — a runtime and orchestration framework for coding agents
> backed by large language models.

`sangam` is being built as an **LLM harness**: a runtime that hosts
LLM-driven coding agents, gives them isolated places to run code, and a typed set
of tools to call. The design rests on four pieces —

- **Runtime-agnostic execution** — a single `Executor` interface
  (`RunSpec` in, `RunResult` out) with runtimes as plugins (bundled Python venv,
  Node/nvm, Docker/Podman, subprocess — and any you install), so the agent never
  depends on a specific runtime.
- **Typed tool registry** — each tool (read/write/edit files, run code,
  type-check, run tests, search) exposes a JSON schema, letting the LLM act
  through function-calling.
- **Model client** — an LLM abstraction (`complete` / `chat` / `tools`) that can
  talk to any provider: any OpenAI-compatible endpoint, or local models (Ollama /
  LM Studio). When no API key is set it falls back to a local model automatically.
- **Agent loop** — plan → act (call a tool) → observe (tests/lint/output) →
  reflect, with conversation memory and resumable state.

It started as a mypy-focused natural-language CLI, and those capabilities — mypy
type analysis, file operations, and resumable task history — stay on as a
built-in toolset and a working surface. Full architecture in
[`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).

## Status

The harness is under active construction: the core agent loop, multi-runtime
executors, tool registry, model client, session store, headless HTTP API server,
VS Code extension, and the local webapp are in place (Phases 1–4 and 6
complete), along with the Phase 5 hardening set — `sangam doctor`,
`sangam config`, `sangam history`, end-to-end sandbox limits, and a
CI-friendly non-interactive test mode. The desktop app surface is the remaining
roadmap. See [`todo.md`](todo.md) for the roadmap and
[`docs/features/`](docs/features/README.md) for one document per feature.

**Available now**

- Core engine split from the CLI surface (`core/engine.py`) — usable as a library.
- **Runtimes are plugins** — one `Executor` contract (`RunSpec` in,
  `RunResult` out) and one plugin contract (`sangam/core/plugins/`), so adding a
  runtime no longer means editing the core. Five plugins ship with sangam:
  `python` (own per-project venv + pip), `node` (nvm + npm), `shell` (your own
  shell), `container` (Docker/Podman), `subprocess` (plain fallback). They are
  *bundled plugins*, not a privileged tier: the same registry, the same lazy
  import, the same enable/disable state as a plugin you install yourself.
  See [`docs/features/runtime-plugins.md`](docs/features/runtime-plugins.md)
  and [`docs/features/multi-runtime-executors.md`](docs/features/multi-runtime-executors.md).

  ```console
  # 1. what is installed — manifests only, no plugin code is imported
  $ sangam plugin list --probe

  # 2. use a runtime by name or alias, or let the command be inferred
  $ sangam exec --runtime python "print('hello')"
  $ sangam exec --runtime js "console.log('hello')"

  # 3. add one — a distribution declaring `entry_points={"sangam.runtimes": …}`,
  #    or a folder dropped in ~/.sangam/plugins/<name>/runtime.json
  $ pip install sangam-runtime-deno
  $ sangam plugin list          # third-party: listed, disabled, source shown
  $ sangam plugin enable deno   # opt in, persisted in the config
  ```

  **Migration note (breaking, Phase 8).** The transitional runtime package that
  used to live inside the core is **deleted**, and the core no longer
  re-exports the executors. Two import lines change:

  ```python
  # before — executors imported from the old core runtime package,
  #          or re-exported by the core itself
  from sangam.core import VenvExecutor

  # after
  from sangam.runtimes import VenvExecutor                # a bundled plugin
  from sangam.core.plugins.api import RunSpec              # the contract
  from sangam.core import get_runtime_registry            # or: don't import at all
  ```

  `get_runtime_registry().executor("python")` is the supported way to get one,
  and it works for third-party plugins too. `sangam.core.__all__` is now
  `SangamCLI`, the sandbox policy names and `get_runtime_registry`. The
  `~/.sangam/environments.json` format, the `sangam plugin` CLI and
  `GET /api/runtimes` are unchanged. The full before/after table is in
  [`docs/ARCHITECTURE.md` §4.5.5](ARCHITECTURE.md#455-what-86-removed).

- **Host-enforced sandbox** (`core/sandbox.py`) — `SandboxPolicy` +
  `SandboxedExecutor` wrapping **every** executor the host hands out, builtin or
  plugin: working-directory scope (path-level validation, symlink resolution),
  wall-clock timeout clamping, and POSIX resource limits (`cpu_time`, `memory`,
  `nproc`, `fsize`, `nofile`). Off by default (`SANDBOX_ENABLED` /
  `SANDBOX_TIMEOUT`, or `sandbox.enabled` / `sandbox.timeout`). It constrains
  the `RunSpec` a runtime is handed; it is not a container, and
  `allow_network=False` only clears proxy variables. The container runtime is
  the isolation boundary. Since **2.58** the filesystem scope also covers the
  **file tools** — `read_file` / `write_file` / `edit_file` / `search` and
  `FileOperations` now refuse a path outside the configured roots instead of
  opening it, with the check resolving symlinks and `..` first. Since **2.61**
  that one is switchable: `SANDBOX_SCOPE_ROOTS` / `sandbox.scope_roots` (an
  `os.pathsep` list; unset or empty means **unrestricted**, so nothing changes
  for anyone who has not opted in) is applied at `ToolRegistry.execute` — the
  one chokepoint the REPL, `sangam agent` and the headless server all dispatch
  through, so a tool added later is scoped by construction. A root that does
  not exist fails at startup rather than quietly refusing everything. Still a
  **policy** boundary, not a security sandbox.
  See [`docs/features/sandbox-manager.md`](docs/features/sandbox-manager.md).
- **Agent loop** — `Agent` (`core/agent.py`) orchestrating `Planner`
  (`core/planner.py`), `Memory` (`core/memory.py`), the `ToolRegistry`, and an
  optional `ModelClient` in a plan → act → observe → reflect loop. Runs in a
  deterministic mode when no model client is configured.
- **Model client** (`core/model/`) — `ModelClient` protocol with
  `OpenAIModelClient` (any OpenAI-compatible endpoint) and `LocalModelClient`
  (local providers such as Ollama / LM Studio); credentials read from env/config.
  `build_model_client(config)` picks the backend — the entry selected from the
  model catalog when one is configured, otherwise OpenAI when an API key is set,
  otherwise a local model (auto-detecting an available model from the running
  server when `MODEL_NAME` is unset).
- **Model catalog** (`core/model/catalog.py`) — `~/.sangam/models.yaml` is the
  source of truth for which models sangam can use, following Continue's
  `config.yaml` `models` shape (`name`, `provider`, `model`, optional `apiBase` /
  `apiKey` / `roles` / `capabilities` / `defaultCompletionOptions` /
  `requestOptions`). The active chat/agent model is the file's `default` entry,
  else the first entry with role `chat`, else the first entry. `sangam models`
  lists the configured entries merged with the ids advertised by probed local
  runtimes (Ollama, LM Studio, and every configured `apiBase`) via `/v1/models`,
  and every probed model is **selectable** under its `source-model-name`
  (`ollama-qwen3:0.6b`). An entry's optional `thinkingLevels` declares the
  `reasoning_effort` values that model accepts, so the thinking level is a
  per-model knob (`/thinking`, `--thinking-level`, the webapp chip, the
  `thinking_level` request field). sangam never installs a model runtime.
- **Typed tool registry** (`core/tools/`) — `read_file`, `write_file`,
  `edit_file`, `list_files`, `search`, `run_python`, `run_js`, `run_container`,
  `run_shell`, `install_deps`, `type_check`, `run_tests`, `suggest_fixes`,
  `web_search` (web search + page fetch), each with a JSON schema for LLM
  function-calling.
  - **MCP servers are tools too** (`core/mcp/`) — configure servers in
    `.mcp.json` (or `~/.sangam/mcp.json`) and their tools join the same
    registry as `mcp__<server>__<tool>`, callable by the model from every
    surface. The MCP client is dependency-free (JSON-RPC 2.0 over stdio or
    HTTP, stdlib only). **Nothing starts without consent**: a server is
    listed but not launched until the model calls one of its tools, or you
    name it with `sangam mcp ping <server>` / `/mcp ping <server>` /
    `/mcp enable <server>`. `"autoStart": false` means *do not start
    unprompted*, not *never start*; `mcp.enabled: false` and
    `sangam mcp disable` still refuse, and a server that fails is retried
    only by `/mcp reload`. Verified end to end against the official
    `chrome-devtools-mcp` server — 30 real tools and a real Chrome — in
    [docs/features/mcp-support.md](docs/features/mcp-support.md).
  - `read_file` reads a whole file **or a slice of it** — `head` (first N
    lines), `tail` (last N), `start_line`+`end_line` (1-based, inclusive), or
    `line_range` in `sed -n` form (`"120,180p"`, `"120p"`, `"120,$p"`). Slices
    stream, so they work on files too big to read whole, and come back numbered
    with the file's line count. Example:
    `{"path": "app.log", "tail": 20}` or `{"path": "app.py", "line_range": "120,180p"}`.
  - `search` greps like `grep -in`: pass `path` for one file
    (`{"path": "app.py", "pattern": "max_file_size"}` → `120:    …`) or
    `directory` to walk a tree, with `literal` (`-F`), `whole_word` (`-w`),
    `ignore_case` (`-i`, the default), `context_lines` (`-C`) and `max_results`.
  - From the REPL you can call either directly: `/tool read_file {"path": "app.py", "head": 20}`
    and `/tool grep {"path": "app.py", "pattern": "TODO"}`.
- **Session store** (`core/session_store.py`) — SQLite-backed sessions, messages,
  model/token usage, and FTS5 search (`~/.sangam/sessions.db` by default).
- **Fourteen CLI subcommands** — `agent`, `exec`, `serve`, `web`, `models`,
  `session`, `run`, `restore`, `agents`, `plugin`, `mcp`, `doctor`, `config`
  and `history`, plus the legacy single-task positional, the interactive REPL
  and batch mode. Every one takes `--help`; they are matched on the *exact*
  first positional, so `sangam config the deployment` still runs the task
  engine rather than the config command. [Every subcommand, with its flags →
  below](#cli).
- **Headless HTTP API server** (`sangam/server/`, Flask) — exposes the agent,
  tools, runtimes, and session store over local HTTP with SSE streaming; the
  single backend for the VS Code extension, local webapp, and desktop app.
- **Local webapp** (`webapp/` + `sangam web`) — a single-page UI (chat, file
  tree, diagnostics, agent-step viewer, palette, status bar) served on
  localhost with the API proxied on the same origin — no build step, no CDN.
  Its presentation layer is built on the DeepSeek Harness design system
  (three-layer `--sgw-*` tokens with light/dark themes, the `AppFrame` shell
  geometry) and it renders markdown, syntax-highlighted code, unified diffs
  and collapsed reasoning rows. It also carries an **interactive CLI dock** —
  a real pseudo-terminal in the browser — and every shell command the webapp
  runs itself goes through it.
- **`install.sh`** — pulls sangam from GitHub `main`, installs it into a venv at
  `~/.sangam/venv`, and runs the headless server. Before starting it probes for a
  local model runtime (Ollama / LM Studio); if none is reachable it errors out
  with "no model available/selected" rather than installing one.
- mypy integration wired through the Executor backends — analyze a file or a
  whole directory, JSON output parsing, and fix suggestions. Defaults to
  `SubprocessExecutor` (`sys.executable -m mypy`); opt-in `use_venv=True` →
  `VenvExecutor` with mypy installed; custom backend via `executor=`.
- Regex-based task engine — natural-language commands mapped to file operations
  and mypy analysis.
- File operations — read, write, append, delete, list, and extract
  functions/classes/imports.
- JSONL task history with search.
- Interactive REPL and batch mode.

**Roadmap**

- Cross-platform desktop app — React UI over the Python core behind a native
  shell, packaged for macOS (`.app`/`.dmg`) and Windows (`.exe`/`.msi`); reuses
  the headless server as its backend. Full breakdown below.
- VS Code extension polish — chat panel, diagnostics, inline actions (extension
  scaffolded and installable; UI activation pending a reload).

## Local Webapp

The **local webapp** is a zero-install surface for the same harness: eight
static files (`index.html`, `app.css`, `app.js`, `markdown.js`, `diff.js`,
`trajectory.js`, `channels.js`, `terminal.js`) plus a vendored `xterm.js`
bundle — no build step, no bundler, no CDN, no runtime dependency — served on
localhost by a small Python static server, talking to the core over HTTP. It reuses the
headless HTTP API server from Phase 4 as its backend — another thin surface
over the same core, not a separate engine.

Item 6.8 restyled its presentation layer onto the design system and app-shell
geometry of [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)
(tokens re-prefixed `--sgw-*`; attribution and the full token/geometry tables in
[`docs/features/webapp-dsh-restyle.md`](docs/features/webapp-dsh-restyle.md)).
No DeepSeek branding is used and no DSH asset is redistributed — only the
design system, re-implemented in plain CSS.

```bash
sangam web                    # UI on http://127.0.0.1:8899, API proxied on one origin
sangam web --port 9000        # different UI port (API stays on 8765)
sangam web --no-browser       # don't open the browser
sangam web --webapp-dir ./webapp   # explicit webapp directory (or $SANGAM_WEBAPP_DIR)
sangam web --theme dark       # pin the page's theme (or $SANGAM_WEB_THEME)
sangam web --no-terminal      # no interactive CLI: every /api/cli/* route 404s
```

> `--no-terminal` (or `$SANGAM_WEB_TERMINAL=0`) turns the browser's CLI off
> entirely. The webapp then keeps working — its shell calls fall back to
> `/api/exec` — and the **▸ CLI** button is disabled with the reason in its
> tooltip, rather than offering a panel that could only fail.

The page follows your **system** theme by default and repaints when the OS
changes it — no reload. Use the ☀ / ☾ / ⌘ switch in the top bar to override;
the choice is remembered in the browser. `--theme` pins it from the command
line for a fresh profile, a kiosk or a screen share, where there is no stored
value and no OS signal to trust. See [Theming](#theming).

What you get in the browser (mirroring the sangam desktop layout):

- **Chat** — send goals in **Agent** mode: the plan → act → observe loop
  streams live from `/api/agent/stream` (step cards for every tool action and
  observation, the final answer, and a stop button). **Exec**, **Type check**,
  and **Tests** composer modes map to `/api/exec`, `/api/type_check`, and
  `/api/run_tests` for one-shot tool runs (per-mode hints + tooltips explain
  what your text becomes and which endpoint it hits).
- **Model selector** — the composer's model chip lists the catalog from
  `GET /api/models` (`~/.sangam/models.yaml` entries; `auto · server default`
  unless you pick one), plus the models auto-probed from Ollama / LM Studio,
  labelled `· probed` and named `<source>-<model>` (2.51); the choice rides the
  agent run as a per-run override and persists in `localStorage`.
- **Attachments** — 📎 or drag-&-drop files onto the composer; they render as
  removable chips and are delivered to Agent prompts as `context.attachments`
  (client-side read, 32 k chars/file cap — no server writes).
- **Explorer** — a lazy file tree (directories via `exec ls`, files via the
  `list_files` tool) with git-status badges (`git status --porcelain`); click a
  file to preview it (HTML renders in a sandboxed iframe).
- **Changes** — type-check diagnostics (errors/warnings with `file:line`) and
  test output from the Changes tab or the composer's Type check / Tests modes,
  **plus a unified-diff view of the working tree** (per-file `+n / −n` counts
  from `git diff --numstat`, the whole diff or one file at a time).
- **Rich replies** — assistant answers render as **markdown** (headings, lists,
  nested lists, tables, blockquotes, task lists) with **syntax-highlighted
  code blocks** (Python, JS/TS, shell, JSON, HTML/CSS, diffs). Consecutive tool
  steps collapse into one disclosure row with a counts summary. A `<think>…</think>`
  block renders as a collapsed **reasoning row** that expands to the full
  reasoning.
- **Light / dark / system theme** — a switcher in the top bar plus a 10–22px
  content font-size stepper. Both persist in `localStorage`
  (`sangam.theme`, `sangam.fontSize`) and the choice is applied by a
  synchronous inline `<head>` bootstrap, so a dark-mode user never sees a white
  flash. `system` follows the OS live.
- **Output-channel backgrounds** — tool calls, the `<think>` reasoning block
  and the final reply each sit on their own themed background, the same way
  the interactive CLI separates its channels (items 6.11 / 2.43). The **reply
  is deliberately the loudest** — a stronger fill, an elevated panel and the
  full content size — while tool calls and thinking stay quiet, so a long
  turn reads as three regions at a glance instead of one mass. The fills
  follow the theme automatically; a **Channels** button in the top bar turns
  them off.
- **App shell** — a 280px sidebar (drag-resizable 264–420px, collapsing to a
  56px icon rail automatically below 1024px, `Mod+B` to toggle) and a
  right-hand panel that opens at 45% of the window, keeps your dragged width up
  to a 70% cap, squeezes to 300px, then closes rather than squeezing the chat
  below its 400px floor.
- **Agents** — the last run's goal, plan steps with status chips, and outcome.
- **Trajectory** — a turn-aware event ledger over the sessions DB (item 6.9):
  virtualized User/Assistant/Tool rows with turn and step boundaries, a
  "Between turns" section for compaction records, an interactive timing
  **Overview** (real start/duration left→right, TTFT divided from decoding,
  hover for the exact clock, drag to focus the ledger on an interval, wheel to
  zoom, right-click to clear) and a **record inspector** (token usage,
  duration, input/output, timing, attachment summaries, and a Code view for
  tool arguments and output). It reads
  `GET /api/sessions/<id>/trajectory` and live-appends from the additive
  `record` / `token` SSE frames — see
  [session-trajectory-view](docs/features/session-trajectory-view.md).
- **CLI** — a bottom-docked **interactive terminal** (item 6.13): a real
  pseudo-terminal, so prompts, pagers, `vim` and colours all behave. Press
  **▸ CLI** in the top bar or `` Ctrl-` ``. Drag (or arrow-key) its top edge to
  resize; Restart gives you a fresh shell, Clear wipes the view without
  stopping it. It runs on a **stdlib PTY** server-side and the vendored
  `xterm.js` browser-side — no CDN, no build step, no new Python dependency.
  Every shell command the webapp issues (Exec mode, `git diff`, the Explorer
  listing, the git-status badges, the +/- counts) goes through
  **`POST /api/cli/exec`** rather than `/api/exec`, so it runs on the same
  terminal and is visible in the dock — see
  [webapp-interactive-cli](docs/features/webapp-interactive-cli.md).
- **⌘K palette** — FTS5 search over all chat history (`/api/search`) plus quick
  commands (new session, toggle panel, refresh changes, …).
- **Status bar** — core health/version, session + tool counts.

Everything is rendered through DOM nodes with `textContent` — `innerHTML` is
used nowhere in the webapp — so markup inside a model reply (or a diff) is
displayed as text, never executed.

Notes: since item 6.9 the agent endpoints accept an optional `session_id`, so
web turns are persisted into SQLite (the user prompt, every assistant step and
tool result, with token/latency/first-token accounting) and the Trajectory view
has something to show; the `localStorage` transcript remains as a fallback for
sessions created before that change. The composer's Exec / Type check / Tests
modes never run the agent loop, so they append through
`POST /api/sessions/{id}/messages` and land in the same ledger. Server sessions
(CLI/agent runs) load read-only. `sangam web` binds to `127.0.0.1` only.
`?api=<url>` (or `localStorage.sangam.apiBase`) points the UI at a different
core, but full support requires the `sangam web` same-origin proxy (the API
sends no CORS headers).

Full task breakdown in [`todo.md`](todo.md), Phase 6.

### Testing the webapp

- `pytest tests/test_webapp_terminal.py` — the interactive CLI: a real PTY
  and a real child shell (echo, `TIOCSWINSZ` verified by running `stty size`
  inside the shell, bounded replay, idempotent teardown), every `/api/cli/*`
  route against a live Flask app and a live SSE socket, the disabled build,
  and the static contract — including *zero* remaining `state.api.exec(` call
  sites in `app.js`.
- `pytest tests/simulation/test_sim_webapp_cli.py -m simulation` — the same
  feature driven in real Edge: the dock, a command typed through xterm's real
  input path into a real shell, `Ctrl-C`, the replay frame, mirroring, theme
  following, and a `fetch` spy proving page-load `git status` hits
  `/api/cli/exec` and **not** `/api/exec`.
- `pytest tests/test_webapp_surface.py` — the HTTP surface: static serving of
  all five assets, same-origin proxy (including the SSE agent stream), the
  client contract, the `--sgw-*` token values, the app-frame geometry numbers
  and the session-description serializer.
- `pytest -m simulation tests/simulation/test_sim_webapp_browser.py` —
  **browser-driven simulations**: launches the real stack and drives
  `index.html` in a real Chromium-family browser (Edge/Chrome, found
  automatically; `$SANGAM_SIM_BROWSER` to point elsewhere) over a
  stdlib-only DevTools-Protocol client — no Playwright/Selenium. Prompts of
  different types (agent goal, exec command, type check path, tests path,
  history search) are typed into the composer and validated against the
  rendered DOM; a real-LLM tier runs the agent prompt through a local model
  endpoint when one is reachable (skips otherwise). Item 6.8 added theme
  switching, markdown/diff/reasoning rendering, the responsive ladder and an
  XSS regression to that suite (39 scenarios). Every test also fails on
  uncaught page JS errors.

## Desktop App — Roadmap

A **cross-platform native desktop app** (macOS + Windows) is a target surface for
the same harness. It keeps the existing **Python core** as the engine and adds a
**React** UI on top — the same "core is a library, surfaces are thin" split used
for the CLI and the VS Code extension:

- **Python core** — the `sangam` package, frozen into a standalone per-OS
  binary and run headless (`sangam serve`, the same HTTPS API server the VS
  Code extension talks to). All agent/tool/runtime logic stays in Python.
- **React UI** — a TypeScript + React frontend (chat panel, file tree,
  diagnostics, agent-step viewer, command palette) that speaks HTTPS to the
  core over localhost (optionally with TLS/cert configuration) — the same
  wire protocol as the VS Code extension.
- **Native shell** — a desktop wrapper that bundles the React build and spawns
  the frozen Python core as a sidecar process. Ships a native `.app`/`.dmg` on
  macOS and `.exe`/`.msi` on Windows (Tauri preferred for the OS-webview native
  feel and small binary; Electron as a fallback).

Planned packaging steps (full task breakdown in [`todo.md`](todo.md), Phase 7):

1. **React UI** — scaffold `desktop-ui/` (Vite + TypeScript + React); build the
   app shell and a typed HTTPS client to the core server.
2. **Native shell** — scaffold `desktop-app/` (Tauri or Electron); configure it
   to bundle and spawn the frozen Python core sidecar; wire frontend↔core IPC over
   localhost HTTPS.
3. **Python core packaging** — freeze the core with PyInstaller/Nuitka per OS
   (including mypy + deps); confirm the headless `serve` entry point runs the
   agent loop end-to-end.
4. **macOS build** — universal `.app` (arm64 + x86_64), `.dmg` installer,
   Developer ID code signing, Apple notarization + stapling, clean-launch test.
5. **Windows build** — `.exe` + `.msi` installer (WiX/Tauri), WebView2 runtime
   handling, code signing, clean-launch test.
6. **Release CI** — GitHub Actions matrix (macOS arm64/x86_64, Windows x64):
   build → sign → notarize (macOS) → package → attach to GitHub Releases.
7. **Updates & polish** — auto-update, native menus/tray, first-run onboarding
   (LLM creds + runtime detection), native window-state persistence.

This builds on the headless HTTPS API server from Phase 4 — the desktop app is
another thin surface over the same server, not a separate backend.

## Installation

```bash
pip install -e .
```

Requires Python >= 3.9, `mypy>=0.900` and `rich>=13.0` (both installed
automatically). `rich` renders the interactive CLI's markdown — tables, lists,
code blocks — and the CLI falls back to its built-in renderer if the import is
unavailable.

### Headless server install (`install.sh`)

To install sangam and run it as a headless HTTP API server (the backend for the
VS Code extension, local webapp, and desktop app), use `install.sh`:

```bash
./install.sh                 # install (or update) and run the server
./install.sh --install-only  # install/update but do not start the server
./install.sh --port 8765     # bind a specific port (default 8765)
./install.sh --local         # install from THIS checkout (current branch,
                             #   working tree as-is) instead of GitHub
./install.sh --local ~/path/to/sangam   # ...from an explicit checkout
```

`install.sh` pulls the source from GitHub `main`, creates a venv at
`~/.sangam/venv`, installs the package, and runs `sangam serve`. With
`--local` it skips the clone entirely and installs **editable** (`pip -e`)
from the local checkout — the current branch and uncommitted changes — so
edits there are picked up by the installed `sangam` without reinstalling
(ideal for testing a feature branch; run `sangam web` afterwards to try it).
The server
needs a model to drive the agent, so before starting it `install.sh` **probes
for a local model runtime**:

- It queries the OpenAI-compatible `/v1/models` endpoint of **Ollama**
  (`127.0.0.1:11434`) and then **LM Studio** (`127.0.0.1:1234`).
- If a reachable endpoint advertises a model, the server is started using that
  local model.
- **sangam never installs a model runtime.** If no Ollama or LM Studio endpoint
  is reachable, `install.sh` errors out with **"no model available/selected"**
  and exits without starting the server.

Override the probe with environment variables:

```bash
SANGAM_MODEL_ENDPOINT=http://127.0.0.1:12345 SANGAM_MODEL_NAME=bonsai-27b-mlx ./install.sh
```

`--install-only` installs without requiring a model.

## Usage

### Surfaces at a glance

Every surface is a thin shell over the same core. This table is the index;
each row is documented below or linked.

| Surface | Command | What it is |
|---|---|---|
| **Agent** | `sangam agent "<goal>"` | The plan → act → observe loop, one-shot |
| **Exec** | `sangam exec --runtime <rt> "<cmd>"` | One command on a chosen runtime |
| **Chat REPL** | `sangam -i` | The interactive session — the richest surface |
| **One-shot chat** | `sangam run "<prompt>"` | One turn of the chat pipeline, then exit |
| **Sessions** | `sangam session`, `sangam restore <id>` | List, describe and resume stored chats |
| **History** | `sangam history [query]` | FTS5 search over messages + token totals |
| **Models** | `sangam models` | The catalog, merged with probed local runtimes |
| **Runtimes** | `sangam plugin` | Inspect/enable/disable runtime plugins |
| **MCP** | `sangam mcp` | MCP servers; nothing starts without consent |
| **Instructions** | `sangam agents` | The `AGENTS.md` loaded into the system prompt |
| **Config** | `sangam config` | Four layers, with the winning source per key |
| **Diagnostics** | `sangam doctor` | What this machine can actually do |
| **HTTP API** | `sangam serve` | The headless backend every GUI surface talks to |
| **Webapp** | `sangam web` | The local single-page UI + its PTY terminal dock |
| **VS Code** | `sangam serve` + the extension | Chat panel and diagnostics over the same API |
| **Task engine** | `sangam "<task>"` | The original natural-language task mode |
| **Batch** | `sangam --batch tasks.txt` | One task per line, from a file |

---

### CLI

Every subcommand takes `--help`. The top-level parser only knows the legacy
task positional, so the subcommand list lives in its epilog:

```console
$ sangam --help
usage: sangam [-h] [--batch] [--config CONFIG] [--verbose] [--interactive]
              [--model PATH] [--theme {light,dark,system}]
              [task]

Subcommands: agent, exec, serve, web, models, session, run, restore, agents,
plugin, mcp, doctor, config, history (each takes --help). With no subcommand, a
task positional runs the legacy single-task mode, and -i starts the interactive
chat.
```

```bash
# --- the legacy task engine, still the default with no subcommand -----------
sangam "read main.py"                 # one task, through the regex task engine
sangam --interactive                  # the chat REPL (alias: -i)
sangam --batch tasks.txt              # one task per line, from a file
sangam --verbose "type check src/"    # DEBUG logging
sangam --theme light --interactive    # pin the terminal theme
# There is deliberately no `--version`: `pip show sangam` is the answer.

# --- agent: the plan → act → observe loop, one-shot -------------------------
sangam agent "fix the type errors in src/ and run the tests"
sangam agent --max-steps 5 "summarise the public API"   # default: 20
sangam agent --model ./models.yaml "add a docstring"    # catalog override
sangam agent --thinking-level high "plan the migration" # reasoning_effort
sangam agent --mcp-config ./path/.mcp.json "…"          # explicit MCP config

# --- exec: one command on a chosen runtime ---------------------------------
sangam exec --runtime shell "ls -la"        # the default runtime
sangam exec --runtime python "print('hello')"
sangam exec --runtime node "console.log('hello')"
sangam exec --runtime container "uname -a" --image alpine:3
sangam exec --cwd ./src --timeout 30 "pwd"   # working directory, wall clock
sangam exec --node-version 20 --runtime node "node -v"

# --- serve: the headless HTTP API, the backend of every GUI surface --------
sangam serve                             # http://127.0.0.1:8765
sangam serve --host 127.0.0.1 --port 9000
sangam serve --certfile cert.pem --keyfile key.pem   # HTTPS

# --- web: the local webapp on the same origin as its API -------------------
sangam web                               # UI on http://127.0.0.1:8899
sangam web --port 9000                   # different UI port (API stays 8765)
sangam web --no-browser                  # don't open the browser
sangam web --no-terminal                 # no interactive CLI: /api/cli/* 404s
sangam web --theme dark                  # pin the page's theme
sangam web --webapp-dir ./webapp         # explicit webapp directory

# --- models: the catalog, merged with what is reachable right now ----------
sangam models                            # configured + auto-probed, marked
sangam models --no-probe                 # configured only (no network)
sangam models --json                     # machine-readable listing
sangam models --select fast-local        # show the client for a named entry
sangam models --select ollama-qwen3:0.6b # …or an auto-probed one
sangam models --role agent               # role used to pick the active model

# --- session / restore: stored chats ---------------------------------------
sangam session list                      # id, updated, surface, description
sangam session list --limit 10 --json
sangam session generate-description <session-id>
sangam session generate-description --all         # only those missing one
sangam session generate-description --all --force # regenerate existing ones
sangam restore <session-id>              # continue it interactively

# --- run: one turn of the chat pipeline, then exit -------------------------
sangam run "summarize the failures in the last test run"
sangam run --model fast-local "what changed since the last commit?"
sangam run --print-session-id "…"        # print the persisted session id

# --- history: searchable chat-session history ------------------------------
sangam history                          # recent sessions + per-session tokens
sangam history retriever                # FTS5 over messages and session titles
sangam history refactor --today         # …filtered by date
sangam history --days 7 --role user     # last week, user turns only
sangam history --session 12             # one session's messages
sangam history planner --json           # machine-readable

# --- agents: the project instructions loaded into the system prompt ---------
sangam agents                           # path, size, truncation notice, preview
sangam agents --path                    # just the resolved path (script-friendly)
sangam agents --preview 0               # the whole file
sangam agents --json                    # machine-readable
sangam agents ./OTHER.md                # inspect an explicit file instead

# --- plugin: the runtime plugins -------------------------------------------
sangam plugin list                      # manifests only; imports no plugin code
sangam plugin list --probe              # import each and report availability
sangam plugin list --json
sangam plugin info python               # one runtime's full manifest
sangam plugin enable deno               # opt in, persisted in the config
sangam plugin disable container         # opt out, persisted
sangam plugin reload                    # re-run discovery, drop cached handles
sangam plugin path                      # folders scanned for dropped-in plugins
sangam plugin scaffold ./deno-plugin    # write a working folder plugin

# --- mcp: MCP servers (.mcp.json) — nothing starts until you ask -----------
sangam mcp list                         # servers, transport, state, source
sangam mcp list --json                  # machine-readable (env values masked)
sangam mcp tools [server]               # the tools a server advertises
sangam mcp ping [server]                # connect + handshake; exit 1 on failure
sangam mcp ping --all                   # including `autoStart: false` entries
sangam mcp enable <server>              # allow it to start (persisted)
sangam mcp disable <server>             # never start it (persisted)
sangam mcp reload                       # re-read the MCP configuration
sangam mcp --config path/.mcp.json list

# --- doctor: what this machine can actually do -----------------------------
# Nothing is installed and no secret is ever printed. Exits 0 even when
# something is missing (use --strict for CI).
sangam doctor                           # every capability
sangam doctor --section node            # one section: python shells node
                                       #   containers runtimes models config
sangam doctor --probe                   # add live `node -v` / `docker version`
sangam doctor --json                    # machine-readable
sangam doctor --strict                  # exit 1 when a capability is missing

# --- config: four layers, and every key says which one is winning ----------
# environment > ./.sangam-config.json > ~/.sangam/config.json > default
sangam config                           # the effective view: value + source
sangam config list --prefix chat.       # narrow to a namespace
sangam config get sandbox.enabled       # one key, with every layer shown
sangam config get sandbox.enabled --raw # just the value, for scripts
sangam config set log_level DEBUG       # write the global file
sangam config set chat.tool_timeout 30 --project   # project-only override
sangam config unset chat.tool_timeout
sangam config path                      # every layer and whether it exists
sangam config list --json               # machine-readable (secrets masked)
```

Three things about `exec` that are easy to get wrong, because the runtimes do
**not** all take the same thing:

- `shell`, `bash`, `zsh`, `sh` and `python`/`venv`/`py` take a **command
  string**, which the runtime hands to your shell or to the interpreter.
- `subprocess`/`direct` takes a **single executable name**. There is no `--`
  separator, so `sangam exec --runtime subprocess "ls -la"` fails with
  `Executable not found: ls -la` — use `--runtime shell` for a command line.
- `container`/`docker`/`podman` and `node`/`js`/`npm` resolve the runtime and
  run the string through them; `--image` and `--node-version` only apply to
  those two.

`--timeout` is a wall-clock ceiling and exits 1 with `[timed out] …`; a CPU-time
or memory kill reports the limit that fired instead.

### Slash commands (inside `sangam -i`)

`sangam -i` is the richest surface, and its vocabulary is declared **once**
(`sangam/cli/chat.py:COMMANDS`) — `/help`, the autocomplete dropdown and
`GET /api/commands` are all rendered from that one table, so a command cannot
be documented without being dispatchable. There are twelve:

| Command | Aliases | What it does |
|---|---|---|
| `/model` | | Pick a model interactively; `/model <name>` switches without losing the transcript |
| `/config` | | Endpoints, models and chat settings; `/config set <k> <v>` writes them |
| `/thinking` | | Show or set the active model's thinking level (`/thinking default` clears it) |
| `/tool` | | List registered tools; `/tool <name> {json}` runs one with **no model round** |
| `/agents` | | Show, switch, drop (`/agents off`) or re-discover (`/agents on`) the `AGENTS.md` |
| `/mcp` | | Configured MCP servers; `/mcp ping|enable|disable <server>`, `/mcp reload` |
| `/plugins` | | The runtime plugins (name, version, source, state) |
| `/compact` | | Compact the history into a digest now, with optional steering |
| `/history` | | The legacy task history (the REPL's own, not `sangam history`) |
| `/search` | | Search the legacy task history |
| `/help` | | The command table, rendered from it |
| `/exit` | `/quit`, `exit`, `quit` | Leave the session |

```bash
# Free text always goes to the model; tools run directly only via /tool,
# and the result is display-only — it is not added to the transcript.
/tool                                 # list registered tools + aliases
/tool read_file {"path": "app.py", "head": 20}
/tool web_search {"query": "…"}
/config set agent_workers 8           # tool-round pool width, 1–16
/compact keep the failing-test details
/thinking high
```

### Slash-command autocomplete

Type `/` at the `sangam -i` prompt and the commands appear below it, with
their descriptions, re-filtered fuzzily as you type (`/cfg` → `/config`):

```
[model] sangam › /cfg
> (•) /config   — Show API call endpoints, the configured models
  ( ) /mcp      — Show the configured MCP servers and their tools
(up/down moves, tab or enter selects, esc dismisses)
```

`Tab` (or `↓` then `Enter`) fills the command in with a trailing space and
returns to the line, so `/model ` can be finished by hand. **It never gets in
the way**: `Enter` with the highlight untouched submits exactly what you
typed, `Esc` dismisses, and a word that matches nothing simply keeps being
your message. The same list appears as a popup over the composer in
`sangam web`, fed by the core's `GET /api/commands`.

### Project instructions (`AGENTS.md`)

sangam loads the repository's standing instructions into the model's
system prompt, so every chat session and every `sangam agent` run follows
your project's rules instead of a generic persona alone. Discovery is
nearest-first (`AGENTS.md`, then `AGENT.md`): the search starts at the
current directory and walks up to the filesystem root, so a nested
package's own file shadows the repo root's.

Resolution order — the same for every surface (chat session, agent loop,
`sangam agents`):

1. an explicit path (`ChatSession(agents_path=…)`, `Agent(agents_path=…)`,
   `/agents <path>`, `sangam agents <path>`),
2. the `SANGAM_AGENTS_FILE` environment variable,
3. walk-up discovery from the current directory.

The content is injected with a framing line that marks it as standing
rules for this repository (without it the model treats the file as
reference material), and is capped at 20 000 characters — head and tail
are kept, the middle is replaced by a `[... truncated N chars ...]`
marker — so a huge file cannot crowd out the conversation. The session
banner names the loaded file, and `/agents` shows, switches, drops or
re-discovers it for the live session.

```bash
export SANGAM_AGENTS_FILE=~/projects/myrepo/CONVENTIONS.md   # override discovery
```

### Streamed thinking

When the backend streams reasoning, `sangam -i` shows it as one
contiguous block instead of one ragged line per token:

```
│ thinking · qwen3:0.6b the user asked about the failing login test, so I
│ should look at the auth module first and check the session store schema

Here is the answer…
```

Deltas are buffered and a line is emitted only once it is complete, then
wrapped to the terminal width — measured in rendered cells, so ANSI
escapes and double-width glyphs count correctly — with a `│ ` gutter on
every line and the model name on the first. Inline markdown in the
reasoning (`code`, **bold**, lists) renders the same way it does in the
answer, and fenced code is left verbatim. The block always ends with a
blank line, so the answer starts on its own line — including when a turn
is cancelled with Ctrl+C.

Turn it off with `/config set thinking_stream off`. Under `NO_COLOR`, or
when output is not a terminal, the block is still wrapped and guttered but
carries no colour escapes, so captured logs stay clean.

### Output channels

The three regions of a turn — tool calls, the streamed thinking block and
the final reply — each print on their own background fill, so a long turn
reads as three separable blocks instead of one mass of text:

```
[ navy   ]  > Tool call · web_search — `{"query": "urvasi song"}`
[ navy   ]  > Tool result · web_search
[ indigo ]  │ thinking · qwen3:0.6b
[ indigo ]  │ the user wants a song, so I should search and summarise
[ green  ]  I found three matches for that song:
```

The fills come from the terminal's 6-level colour cube, so a 256-colour
terminal renders them exactly and the three channels stay distinct. Colour
appears only on a colour terminal: under `NO_COLOR`, `TERM=dumb`, in CI or
when output is piped, the blocks are byte-identical to the plain text (the
gutters and labels still separate them).

```bash
/config set channel_backgrounds off      # back to the glyph-only look
/config set channel_theme light          # for a light terminal background
SANGAM_THEME=light sangam -i             # same, without touching the config
```

Display only — transcripts, `sangam session list` and `sangam restore` are
unaffected. See
[`docs/features/cli-output-channels.md`](docs/features/cli-output-channels.md).

### How many tool calls run in parallel

When the model asks for several tools in one round — typically a fan-out of
`run_agent` sub-agents — the chat session runs up to **4** of them at a time.
That is yours to change:

```bash
/config set agent_workers 8        # up to 8 calls in parallel per round
/config set agent_workers 1        # run a fan-out sequentially (debugging)
/config set agent_workers default  # back to 4
```

The value takes effect **from the next tool round** — no restart — and is
persisted in `~/.sangam/config.json` under its own key `agent.max_workers`
(not `chat.*`: it is an execution knob, not a display preference). A
project-level `.sangam-config.json` can still override it, so a repository
can pin a lower ceiling without touching your global file. The accepted range
is 1–16: `1` disables parallelism, which is how an order-sensitive race gets
debugged, and beyond ~16 the model API's rate limits and cost, not local
threads, are what a fan-out runs into. A value outside the range is an error
that names it — the session keeps going. `/config` shows the live value as
`agent_workers: 8 (default 4)`.

Results are still recorded in the order the model asked for them, whatever the
pool width, and a round containing `question_and_answer` never enters the pool
at all. The knob sizes the *chat session's* pool; `sangam agent` and the
headless server run their tool rounds sequentially.

### When the conversation outgrows the context window

Every send is token-budgeted first (2.29): the prompt is estimated, the limit
comes from the catalog's `maxContextTokens` → `chat.max_context_tokens` → a
conservative 131,072 default, and an over-budget prompt is trimmed — dropping
the oldest exchanges in whole tool-call blocks and truncating oversized tool
results — with a one-line `context trimmed to fit …` notice. If the API
rejects the request anyway with `HTTP 400 … maximum context length`, the turn
retries once against a halved window instead of ending.

Trimming loses the earlier work outright, so **2.30 compacts before that
happens**: once the prompt reaches `auto_compact_threshold` percent of the
budget (**on**, at **70%**), the oldest history is replaced by a single
**digest** message written by the session's own model — and you see a one-line
notice before the reply:

```
compacted 42 older message(s) into a summary (est. 12.4k → 3.1k tokens)
```

The system prompt and the last six messages always survive, and repeating a
compaction folds the previous digest into the new one instead of dropping it.
Run it yourself at any time with `/compact`, optionally steering what the
digest keeps:

```
/compact keep the failing-test details
```

This is **prompt-side only**: `Memory` and the session transcript keep every
message, so `sangam history`, search and resume are unaffected — only what the
model is sent shrinks. If the summarisation call fails (or there is no model
client) a stdlib extractive digest is used instead and the notice says so, so
compaction can never leave you worse off than the trim.

```bash
/config set auto_compact off              # back to plain trim-to-fit
/config set auto_compact_threshold 50     # compact earlier
/config set max_context_tokens 200000     # explicit cap; 0 = resolve
```

All three persist under `chat.*` in `~/.sangam/config.json`, read per turn, and
a bad value is an error that names the range — the session keeps going.

### When a tool call or API request fails

A transient failure — a `429`, a `5xx`, a dropped connection, a read timeout —
is **retried** instead of ending the turn, and you watch it happen:

```
> Tool call · web_search — `{"query": "urvasi song"}`
> Retrying · `web_search`
> connection reset by peer — retrying in 0.5s (attempt 2 of 3)
> Tool result · web_search
```

A tool call that never returns is bounded by a host-level ceiling
(`tool_timeout`, default 120 s) and turned into a failure the model can react
to, instead of wedging the turn with nothing on screen. When retries are
exhausted, the error is printed in full rather than as one run-on line:

```
Model request failed
  type     : ModelClientError
  status   : 500 Internal Server Error
  endpoint : http://127.0.0.1:12345/v1/chat/completions
  response : {'error': {'message': 'CUDA out of memory'}}
  hint     : this looks transient — raise /config set tool_retries
```

A deterministic failure is *not* retried — a `401` or a tool with a bug in it
will fail identically every time — and the hint says so rather than leaving
you to guess. Ctrl+C still wins: a cancelled turn is never re-sent.

```bash
/config set tool_timeout 300      # seconds; 0 = no limit
/config set tool_retries 5        # 0 = never retry
SANGAM_DEBUG=1 sangam -i          # append the traceback to a diagnostic
```

Model-request retries are tuned per model through `requestOptions:
{retry_attempts: N}` in the catalog, or globally with `SANGAM_RETRY_ATTEMPTS`
/ `SANGAM_RETRY_BASE_DELAY` / `SANGAM_RETRY_MAX_DELAY`. Setting
`logging.getLogger("sangam").setLevel(DEBUG)` records every round, call,
attempt and retry decision. See
[`docs/features/tool-call-resilience.md`](docs/features/tool-call-resilience.md).

### Model catalog (`~/.sangam/models.yaml`)

`~/.sangam/models.yaml` is the source of truth for which models sangam can use.
It follows Continue's `config.yaml` `models` shape:

```yaml
default: fast-local          # optional: the active chat/agent model

models:
  - name: fast-local
    provider: ollama
    model: qwen3:0.6b
    apiBase: http://127.0.0.1:11434/v1
    roles: [chat, agent]
    capabilities: [tool_use]
    defaultCompletionOptions:
      temperature: 0.2
      maxTokens: 2048
    requestOptions:
      timeout: 60  # seconds

  - name: remote-gpt
    provider: openai
    model: gpt-4o-mini
    apiKey: $OPENAI_API_KEY   # literal, $VAR, or ${VAR}
    roles: [chat]

  - name: gpt-5-mini
    provider: openai
    model: gpt-5-mini
    apiKey: $OPENAI_API_KEY
    thinkingLevels: [minimal, low, medium, high]   # this model's ladder (2.52)
```

Selection order: the file's `default` entry, else the first entry with role
`chat`, else the first entry. `sangam models` merges those configured entries
with the model ids advertised by probed OpenAI-compatible runtimes (Ollama on
`127.0.0.1:11434`, LM Studio on `127.0.0.1:1234`, and every configured
`apiBase`) via `/v1/models`. Probing is read-only and best-effort — sangam never
installs a model runtime. Override the catalog path with `SANGAM_MODELS_FILE` or
`sangam models --catalog <path>`.

**Probed models are selectable too (2.51).** A model found on a running Ollama
or LM Studio is offered *on top of* the catalog, not instead of it, under its
**`source-model-name`**: `ollama-qwen3:0.6b`, `lmstudio-llama-3.2-3b`. Pick one
from the `/model` picker, type `/model ollama-llama3:0.6b`, use it in
`sangam models --select`, or choose it in the webapp's model chip — every
surface reads the same set. A model the catalog already covers is not
duplicated, and a probed model never becomes the session's default:
`models.yaml` still decides which model you start on and how it is configured
(roles, key, completion options). A probed model is never written back to the
file.

**Thinking levels are declared per model (2.52).** `thinkingLevels:` lists the
`reasoning_effort` values that model accepts — a level one provider accepts
another rejects, and a fixed-reasoning model accepts none, so the ladder is a
catalog fact, not a global setting. Omit it (or leave it empty) and that model
takes no level control at all: every surface hides the selector rather than
offering values the API would reject. The selected level is sent as the
top-level `reasoning_effort` field:

| Where | How |
|---|---|
| `sangam -i` | `/thinking` shows the active level and this model's ladder; `/thinking <level>` sets it; `/thinking default` clears it. Also `/config set thinking_level <level>\|default`, persisted as `chat.thinking_level`. |
| `sangam agent` | `--thinking-level LEVEL`, validated against the resolved model before the run starts (an unusable one is a startup error, exit 2). |
| `sangam web` | a `#thinkingSel` chip beside the model selector, filled from the active model's ladder and sent per request. |
| `sangam serve` | `"thinking_level"` in the `/api/agent/run` and `/api/agent/stream` body; a level the model does not list is **HTTP 400 naming its ladder**. `GET /api/models` reports the same ladder per entry, so the API advertises exactly what it accepts. |

With **no** level selected nothing is sent at all and the provider's own
default applies — which is *not* the same as the `none` level: `none`
explicitly disables thinking and is sent. Switching models re-checks the level
(kept when the new model supports it, stopped with a one-line notice when it
does not), and a level stored in `chat.thinking_level` that the active model
does not accept is simply not sent — persistence is a preference, not a
promise.

`requestOptions` carries transport-level settings for the entry's HTTP calls.
`timeout` (seconds) is honoured on every chat-completions request and overrides
the `SANGAM_LLM_TIMEOUT` environment variable (which defaults the timeout to
120 s). Invalid or non-positive values are ignored.

### Configuration (`~/.sangam/config.json`)

Configuration is read from four layers, and later layers win:

```
environment  >  ./.sangam-config.json  >  ~/.sangam/config.json  >  built-in default
```

`sangam config` shows **which value is winning and where it came from**, which
is otherwise unanswerable from the value alone — a project override and the
global default are the same value.

```bash
sangam config                              # every key: value + source
sangam config list --prefix chat.          # one namespace
sangam config list --prefix sandbox.       # sandbox policy keys
sangam config get sandbox.enabled          # one key, every layer shown
sangam config set log_level DEBUG          # write ~/.sangam/config.json
sangam config set chat.tool_timeout 30 --project   # project-only override
sangam config unset chat.tool_timeout
sangam config path                         # every layer, present or not
```

The namespaces it covers are the ones the rest of the product reads —
`runtimes.*` (the plugin enable/disable lists, 8.5), `mcp.*` (2.44),
`sandbox.*` (the host-enforced policy, 8.4), `chat.*` (the display settings,
2.19/2.46) and `agent.max_workers` (the tool-round worker cap, 2.54). Values
are parsed as JSON when they are valid JSON (`true`, `12`, `["a","b"]`) and
stored as a string otherwise, so `log_level DEBUG` stays a string.

**A key's environment spelling is imported from the module that reads it**, so
a knob added to `sangam/core/sandbox.py` shows up here instead of needing a
second edit — and a test fails if one is ever missed (2.59):

```bash
SANDBOX_MEMORY=536870912 sangam config get sandbox.memory --json
# {"key": "sandbox.memory", "value": null, "source": "env",
#  "layers": {"env": "536870912", ...}}
```

**The printed value and its source are one lookup (2.60).** `resolve()` returns
`(value, source)` together and every layer is coerced to the key's declared
type, so the two cannot contradict each other — which they used to:

```bash
SANDBOX_ENABLED=true sangam config get sandbox.enabled
# sandbox.enabled = true      <- used to print the default, false, beside
#   source: env               <- this line, in the same breath
```

**A `set` never writes the merged view.** `set --project` writes
`./.sangam-config.json` and only that one key, so an override stays a
*difference* from the global file rather than a second copy of it.

**Secrets are never printed.** A key whose name looks secret (2.19's markers:
`key`, `token`, `secret`, `authorization`, `password`) renders as `<set>` /
`(not set)`, and nested dictionaries are masked with the same recursion the
`/config` chat view uses — the two surfaces import the same helpers rather
than each declaring their own. That includes the confirmation line of `set`:
the value is written to the file, but not echoed to the terminal.
`--show-secrets` is the explicit opt-in and says so in its output.

### As a library

```python
from sangam import SangamCLI

cli = SangamCLI()
result = cli.execute_task("type check main.py")
print(result)
```

The runtime contracts are importable for writing a runtime plugin:

```python
from sangam.core import Executor, RunSpec, RunResult
```

The agent harness is importable too:

```python
from sangam.core.agent import Agent
from sangam.core.planner import Planner
from sangam.core.tools import ToolRegistry
from sangam.core.session_store import SessionStore
from sangam.core.model import (
    ModelClient,
    OpenAIModelClient,
    LocalModelClient,
    build_model_client,
)
```

## Theming

Every sangam surface resolves its theme through **one ordered chain**:

```
--theme flag  →  SANGAM_THEME env  →  stored preference  →  system
```

**`system` is the fallback.** With nothing stored and nothing passed you get
whatever your operating system is set to, and the webapp *follows it live* —
flip your OS to dark mode and the page repaints without a reload.

| Surface | How to set it | Remembered in |
|---|---|---|
| **CLI** | `SANGAM_THEME=light sangam -i`, or `sangam --theme light` | `~/.sangam/config.json` → `theme` (`/config set theme dark`) |
| **Webapp** | the ☀ / ☾ / ⌘ switcher, or `sangam web --theme dark` (`$SANGAM_WEB_THEME`) | the browser (`localStorage`) |
| **VS Code** | nothing to set — the panel follows the editor | — |

A few rules worth knowing:

- **`NO_COLOR` wins over everything** on the terminal — no colour at all,
  whatever the theme says.
- **The CLI's effect is deliberately small.** `dark` (and the `system`
  fallback, since a terminal can't ask the OS) is exactly how sangam has
  always looked. `light` only lifts the *dim* tier one step so secondary text
  stays legible on a light-background terminal. No palette was rewritten.
- **Nothing throws.** A typo like `SANGAM_THEME=sepie` warns once and falls
  through to the next source instead of breaking your shell session.
- **The webapp flips instantly** — a theme change is a repaint, never a
  cross-fade.

Full contract, the per-surface precedence table and how to add a palette
token: [`docs/features/theming.md`](docs/features/theming.md).

## Project Structure

```
sangam/
├── __init__.py            # Package exports
├── cli/
│   ├── main.py            # CLI surface (argparse, REPL, batch mode)
│   ├── theme.py           # the shared theme contract: resolution order,
│   │                      #   NO_COLOR + TTY detection, the dim tier
│   ├── chat.py           # the interactive session: COMMANDS, /tool, /config,
│   │                     #   the thinking stream, auto-compact, the worker pool
│   ├── channels.py       # the three output channels and their backgrounds
│   ├── complete.py       # fuzzy candidate ranking (autocomplete, /api/commands)
│   ├── runtimes_setup.py # opt the core into the runtime plugins, once, here
│   ├── spinner.py, theme.py, markdown.py, markdown_rich.py,
│   │   markdown_builtin.py, text_width.py, think_time.py,
│   │   thinking_stream.py, metrics.py, context.py, project_context.py
│   └── commands/         # one module per subcommand
│       ├── agent.py      # sangam agent "<goal>"
│       ├── agents.py     # sangam agents  (the AGENTS.md in the system prompt)
│       ├── config.py     # sangam config  (the four-layer configuration surface)
│       ├── doctor.py     # sangam doctor  (capability probing)
│       ├── exec.py       # sangam exec --runtime <rt> <command>
│       ├── history.py    # sangam history [query]
│       ├── mcp.py        # sangam mcp     (MCP server control)
│       ├── models.py     # sangam models  (list/select from models.yaml)
│       ├── plugin.py     # sangam plugin  (runtime plugin control)
│       ├── restore.py    # sangam restore <session-id>
│       ├── run.py        # sangam run "<prompt>"
│       ├── serve.py      # sangam serve   (headless HTTP API server)
│       ├── session.py    # sangam session list | generate-description
│       └── web.py        # sangam web [--theme] [--no-browser] [--no-terminal]
├── core/
│   ├── engine.py          # Core engine (config + logger + task engine wiring)
│   ├── agent.py           # Agent orchestrator (plan → act → observe → reflect)
│   ├── planner.py         # Goal → step plan
│   ├── memory.py          # Conversation history + working-set
│   ├── session_store.py   # SQLite sessions/messages/usage + FTS5 search
│   ├── sandbox.py         # SandboxPolicy, SandboxedExecutor, resource limits
│   ├── resilience.py      # retry/backoff and structured error diagnostics
│   ├── processes.py       # process helpers shared by the tools
│   ├── mcp/               # the dependency-free MCP client (JSON-RPC 2.0)
│   ├── model/
│   │   ├── __init__.py    # exports + build_model_client(config) factory
│   │   ├── client.py      # ModelClient protocol + data contracts
│   │   ├── catalog.py     # ~/.sangam/models.yaml catalog (load/select/probe)
│   │   ├── model.py       # OpenAIModelClient (OpenAI-compatible)
│   │   └── local.py       # LocalModelClient (local providers, e.g. Ollama/LM Studio)
│   ├── tools/
│   │   ├── registry.py    # Tool, ToolResult, ToolRegistry, JSON-schema gen
│   │   ├── file_tools.py  # read_file, write_file, edit_file, list_files, search
│   │   ├── exec_tools.py  # run_python, run_js, run_container, run_shell, install_deps
│   │   ├── type_check_tools.py # type_check, run_tests, suggest_fixes
│   │   └── web_tools.py   # web_search — web search + page fetch
│   └── plugins/
│       ├── api.py         # the plugin contract: RuntimeManifest, RuntimePlugin,
│       │                  #   Executor, RunSpec, RunResult, EnvironmentStore
│       ├── registry.py    # RuntimeRegistry: discovery, aliases, fallbacks,
│       │                  #   enable/disable, the single sandbox wrap point
│       └── loader.py      # builtin / entry-point / filesystem sources
├── runtimes/              # the five bundled runtime plugins
│   ├── plugins.py         # BUILTIN_MANIFESTS + lazy factories + probes
│   ├── venv.py            # VenvExecutor (isolated venv + pip + run Python)
│   ├── nvm.py             # NvmExecutor (nvm/npm, JS/React)
│   ├── shell.py           # ShellExecutor (native bash/zsh, detect_shell)
│   ├── container.py       # ContainerExecutor (Docker/Podman) — the real
│   │                      #   isolation boundary (--cpus/--memory/--network)
│   ├── subprocess.py      # SubprocessExecutor (plain subprocess fallback)
│   ├── limits.py          # classify_termination / annotate_run_result (5.4):
│   │                      #   names the limit that fired, reports the ones
│   │                      #   the platform cannot enforce
│   └── environments.py    # EnvironmentRegistry (~/.sangam/environments.json)
├── server/                # Headless HTTP API server (Flask)
│   ├── __init__.py        # Flask app exports
│   ├── http_api.py        # create_app factory, per-request SessionStore, SSE,
│   │                      #   /api/cli/* (the interactive CLI, item 6.13)
│   └── terminal.py        # the CLI's PTY layer: TerminalSession + TerminalManager
├── task_engine.py         # Natural-language task parsing and execution
├── sangam_integration.py  # mypy subprocess wrapper and JSON parsing
├── file_ops.py            # File read/write/analysis operations
├── logger.py              # Logging and JSONL task history
├── config.py              # Configuration management
└── mypy_cli.py            # Backward-compatible shim (re-exports cli/main + core)
webapp/                    # local webapp — no build step, no CDN
├── index.html             # markup + the inline SangamClient fetch client
├── app.css / app.js       # the --sgw-* token system, render functions, wiring
├── markdown.js / diff.js  # streaming GFM + syntax highlighting, diff renderer
├── trajectory.js          # the Trajectory ledger, Overview and inspector
├── channels.js            # output-channel vocabulary + master switch
├── terminal.js            # the interactive CLI dock (SgwCli) — item 6.13
└── vendor/                # @xterm/xterm 5.5.0 + addon-fit, vendored (MIT)
tests/                     # pytest test suite
docs/                      # Architecture docs
```
## Running Tests

The test suite uses [pytest](https://docs.pytest.org/). Install the test
dependencies if you don't have them:

```bash
python3 -m venv venv
./venv/bin/python -m pip install -r requirements-dev.txt
```

Then run the tests from the project root — the wrapper runs them **in
parallel** with [pytest-xdist](https://pytest-xdist.readthedocs.io/):

```bash
./run_tests.sh                    # pytest -q -n <4 x CPU> --dist load
```

**Always go through `./run_tests.sh`** rather than calling `pytest` yourself:
the wrapper resolves the venv interpreter, checks that `pytest-xdist` is
importable, and picks the worker count. That count is **4 × the CPU count**,
not one per core — a worker in this suite spends most of its time in
`subprocess`, `urllib`, SQLite and `time.sleep` rather than on the CPU, so one
worker per core leaves cores idle waiting on I/O. Measured on a 10-core
machine (unit subset): 73.4 s at `-n 10`, 64.5 s at `-n 20`, **50.3 s at
`-n 40`**, with the same pass/fail set at every count.

Runner options:

| Command | What it runs |
|---------|--------------|
| `./run_tests.sh` | the whole suite, parallel (`-m` unset, so see the caveat below) |
| `./run_tests.sh -m "not simulation"` | the **unit suite** only — what CI runs |
| `./run_tests.sh --simulation` | the live simulation suite (`-m simulation`, `--dist loadfile`) |
| `./run_tests.sh -k session` | extra args are passed straight to pytest, last |
| `./run_tests.sh -n 4` | fixed worker count |
| `./run_tests.sh --dist loadfile` | group all tests from one file on one worker |
| `./run_tests.sh --serial` | one worker, no xdist (same as plain pytest) |
| `./run_tests.sh --ci` | **headless-runner mode** — see below |
| `./run_tests.sh --ci --print-command` | echo the pytest argv and the CI diagnostics, run nothing |
| `./run_tests.sh --print-command` | echo the pytest argv, run nothing |

> **A bare `./run_tests.sh` runs the simulation tests too.** `pytest.ini`
> carries no `addopts` — deliberately, so plain `pytest tests/` keeps working
> in an environment without `pytest-xdist` — so the marker is not deselected
> for you. Pass `-m "not simulation"` when you want the unit suite.

`--dist` selects how work is handed out: `load` (default, any pending test to
any free worker — fastest), `loadfile` (a whole file stays together),
`loadscope` (a whole class/module stays together), `each` (runs everything on
every worker, for race testing). `SANGAM_TEST_WORKERS`, `SANGAM_TEST_DIST` and
`SANGAM_TEST_SIM_DIST` set the defaults from the environment.

Plain pytest still works, and is what to reach for when you want a single
process (`--pdb` disables xdist automatically):

```bash
python -m pytest tests/ -v
pytest tests/test_file_ops.py
pytest tests/test_file_ops.py::TestFileOperations::test_read_file
```

The mypy integration tests mock the `mypy` subprocess, so they run even if mypy is not installed.

### CI mode (`--ci`)

`./run_tests.sh --ci` (or `SANGAM_CI=1`, the same shape as
`SANGAM_TEST_WORKERS` ← `-n`) is the contract for a runner with **no TTY, no
interactive prompt, no browser, no docker daemon and no local model endpoint**,
and a CPU count that is not the machine the 4× default was measured on. It is a
*mode*, not a second runner: every other flag still works on top of it.

```bash
./run_tests.sh --ci -m "not simulation"     # the unit suite, headless
./run_tests.sh --ci --simulation            # the live suite, headless
SANGAM_CI=1 ./run_tests.sh                  # the same, via the environment
./run_tests.sh --ci --print-command         # see exactly what it would exec
```

It changes four things, and prints each one to **stderr** before pytest starts
(stdout stays the argv, so `--print-command` remains machine-readable):

1. **Nothing can block on stdin.** `exec < /dev/null` before pytest starts, so
   an accidental `input()` gets EOF at once instead of hanging the job until
   the runner's timeout — the difference between a red run and a cancelled one.
2. **A conservative worker default — 2 × CPU, not 4 ×.** The 4× number was
   measured on an *idle* 10-core machine; a CI runner is typically 2 cores
   **already contended by the runner itself**. `-n`, `--serial` and
   `SANGAM_TEST_WORKERS` still win over it.
3. **`-ra --color=no`.** `-ra` prints a *reason* for every skip, which is the
   only way to tell "the Chrome simulation skipped: no Chrome" from "it
   silently never ran".
4. **A machine-readable JSON result**, written to `SANGAM_CI_REPORT` /
   `--ci-report PATH` (default `tmp/ci-result.json`, already covered by
   `.gitignore`'s `tmp/*`), from an EXIT trap — so a **failing** run still
   produces one. It carries the argv, `exit_code`, `duration_s`, a timestamp,
   and every deselected node id **with its reason**:

   ```console
   $ cat tmp/ci-result.json
   {
     "command": ["…/venv/bin/python", "-m", "pytest", "-q", "-n", "4", "…"],
     "deselected": [
       {
         "nodeid": "tests/simulation/test_sim_shell_executor.py::TestRealLifeLogAnalysis::test_logs",
         "reason": "reads tests/fixtures/real_life/logs/sample.log, which .gitignore's *.log rule leaves untracked"
       }
     ],
     "duration_s": 388.55,
     "exit_code": 0,
     "generated_at": "2026-10-03T20:32:51+0530",
     "schema": "sangam-ci-result/1"
   }
   ```

   pytest's own exit code is still the process exit code, unchanged.

`--ci` deliberately does **not** decide whether the simulation suite runs —
that stays `--simulation`'s job, and the workflow states which one it asked
for. What it does guarantee is that a simulation suite asked for on a runner
without Chrome, Ollama or docker **skips** rather than fails: those guards
already exist in the suite (`tests/simulation/conftest.py` for the model
endpoint, `container_executor`/`nvm_executor` for docker and node, and
`launch_browser`'s `CDPError` for Chrome).

Nothing needs deselecting: every fixture the simulation tier reads is tracked.
One used not to be — `TestRealLifeLogAnalysis::test_logs` reads
`tests/fixtures/real_life/logs/sample.log`, which the inherited `.gitignore`'s
`*.log` rule swallowed, so the test failed on every fresh clone. The fixture is
**tracked now** (`!tests/fixtures/**/*.log`, scoped so real `*.log` files
elsewhere stay ignored). `--ci` keeps a conditional deselect for it as a
fallback for a partial or older checkout; it is inert whenever the file is
present, and the condition is on the file's existence rather than on anything
about CI.

### The GitHub Actions workflow

`.github/workflows/ci.yml` is deliberately two jobs, because a CI file nobody
can follow is a liability.

| Job | Matrix | Command |
|---|---|---|
| `unit` | `ubuntu-latest` × `macos-latest` × Python `3.9` × `3.14` | `./run_tests.sh --ci -m "not simulation"` |
| `simulation` | `ubuntu-latest`, Python `3.9` | `./run_tests.sh --ci --simulation --ignore=…webapp_browser.py --ignore=…webapp_cli.py` |

- **Both OSes** because the suite shells out to pty, subprocess, symlinks and
  POSIX resource limits, and those behave differently on each.
- **`3.9` is the floor** `setup.py` declares (`python_requires=">=3.9"`);
  **`3.14` is what `actions/setup-python` resolves as newest today** — it is
  deliberately *ahead* of the classifiers (3.12), so the matrix surfaces drift
  rather than hiding it.
- **The live suite runs on Linux only**, because it drives a real venv, shell,
  rlimits and npm, and a macOS runner costs several times the minutes for no
  extra signal. It runs *explicitly* rather than by accident — see the caveat
  above about a bare `./run_tests.sh` collecting the simulation tests too.
- The two **Chrome-CDP modules** are `--ignore`d there with the reason in the
  step: `ubuntu-latest` has Chrome preinstalled, so they would *run* rather than
  skip, and they are the flakiest tests in the repository. They are run
  locally with `./run_tests.sh --simulation`.
- `pytest-xdist` is installed from `requirements-dev.txt`, the only place the
  suite's test dependencies are declared. All three actions are pinned to an
  explicit patch tag (`checkout@v7.0.1`, `setup-python@v7.0.0`,
  `upload-artifact@v7.0.1`), so one commit cannot get two different workflows.
- There is **no linter, no formatter and no type-check step**. AGENTS.md rule
  16 treats anything needing a compiled toolchain as a dependency this project
  avoids, and the mypy story is the agent shelling out to a tool, not a build
  gate.

Full contract: [`docs/features/test-runner-and-ci.md`](docs/features/test-runner-and-ci.md).

## Simulation Tests

Beyond the unit tests, `sangam` ships a **live simulation suite** that drives
real runtimes end-to-end — no mocking of venv, subprocess, or the model
endpoint. These live in `tests/simulation/` and are tagged with the
`simulation` pytest marker, so they can be selected with `-m simulation`.

**What they cover**

- `test_sim_venv_executor.py` — `VenvExecutor`: real venv creation (idempotent
  `prepare`), real pip installs of a local package, venv-vs-subprocess isolation
  proof, `run_python` convenience + env passthrough, live timeout enforcement,
  failure reporting, and sandbox-wrapped runs (allowed/disallowed cwd, proxy
  clearing).
- `test_sim_shell_executor.py` — `ShellExecutor`: real-life `.sh` fixture
  scripts (`simulate_release.sh` release pipeline, `analyze_logs.sh` log
  analysis), one-off commands (pipelines, env, stdin, cwd), multi-line
  `run_script` constructs, live timeouts, and sandbox policy.
- `test_sim_subprocess_executor.py` — `SubprocessExecutor`: real command
  execution (PATH/absolute resolution, mypy `--version`), failure reporting,
  stdin/env/cwd, live timeouts, sandbox enforcement (filesystem scope, default/
  max timeout clamping, proxy clearing), and **live POSIX resource-limit
  enforcement** (`fsize`, `cpu_time`, `nofile`).
- `test_sim_container_executor.py` — `ContainerExecutor`: a real docker **or**
  podman run. Skips when neither is usable.
- `test_sim_nvm_executor.py` — `NvmExecutor`: real node/npm. Skips when no
  Node toolchain is on PATH.
- `test_sim_sandbox_limits.py` — which limit actually fired, per platform, with
  POSIX/Windows and Linux/macOS `skipif` splits.
- `test_sim_file_read_ranges.py` — the file-tool slice forms against a real
  model endpoint.
- `test_sim_openai_endpoint.py` — a real OpenAI-compatible endpoint (the
  device's local Ollama at `http://localhost:11434/v1`, standing in for any
  OpenAI-compatible provider): `/v1/models`, chat completions (single/multi-turn,
  system prompt, usage accounting), SSE streaming, tool-calling, and
  error-path validation.
- `test_sim_webapp_browser.py`, `test_sim_webapp_cli.py` — **browser-driven**
  simulations: they launch the real stack and drive `index.html` in a real
  Chromium-family browser (Edge/Chrome, found automatically;
  `$SANGAM_SIM_BROWSER` to point elsewhere) over a stdlib-only
  DevTools-Protocol client — no Playwright/Selenium.

**Fixtures** — real-life usage-case files live in `tests/fixtures/real_life/`
(`simulate_release.sh`, `analyze_logs.sh`, `report_env.py`, and a small `src/`
tree). The endpoint and model are overridable via
`SANGAM_OLLAMA_BASE_URL` and `SANGAM_OLLAMA_MODEL` (default `qwen3:0.6b`).

> **Fixture tracking:** `tests/fixtures/real_life/logs/sample.log` was
> swallowed by the generic Python `.gitignore`'s `*.log` rule and so was
> **untracked** — `test_sim_shell_executor.py::TestRealLifeLogAnalysis::test_logs`
> failed on every fresh clone. It is **tracked now**, via a tightly-scoped
> `!tests/fixtures/**/*.log` negation, so the test runs for everyone.

**Running the suite**

```bash
# All live simulation tests (parallel, grouped by file so each file keeps one worker)
./run_tests.sh --simulation

# A single backend
./run_tests.sh --simulation tests/simulation/test_sim_shell_executor.py

# Headless (CI): stdin closed, skip reasons printed, JSON result written
./run_tests.sh --ci --simulation

# Verbose runner with per-test traces, live logs, and a timestamped log file
./run_live_tests.sh
```

The simulation suite drives real shared resources (real venv/npm installs, a
real Ollama server, a real Chrome instance), so the runner defaults it to the
conservative `--dist loadfile` grouping; use `--serial` or `--dist load` if you
want to override that. It gains the least from a wide worker pool — on a
machine already busy with other work, drop to the core count
(`./run_tests.sh --simulation -n "$(sysctl -n hw.ncpu)"`) or go `--serial`.

The endpoint tests **skip** (never fail) when Ollama is not reachable or no
model is pulled, and so do the container, node and browser tiers when their
toolchain is missing — so the default suite stays green on a machine with none
of them. A full run is ~3900 tests; the simulation tier is 236 of them
(3733 unit tests, 236 simulation tests).

---

## License

MIT
