# Palm Engine — LLM Guide

> Lightweight Behavior Tree orchestration for wizards, durable workflows, and human-in-the-loop processes in Python.

**Version:** 0.14.12 (shipping) · **0.14 MCP** (operator adapter) · **PyPI:** `palmengine` · **Import:** `palm` · **CLI:** `palm`

## Philosophy

Palm grows where the sun meets the sea. Orchestration should feel alive, truthful, and humane. The core stays pure; capability grows at the edges through registries.

## Recommended Entry Points

For most applications, use **ApplicationHost** — it wires CQRS, projections, outbox, compensation, and recovery:

```python
from palm.app import ApplicationHost, HostProfile

with ApplicationHost(profile=HostProfile.all_in_one()) as host:
    job = host.submit_flow("onboard")
    views = host.list_instance_views(include_terminal=False)
```

CLI bootstrap uses the same path via `create_cli_host()` (collapsed `all_in_one` profile).

**PalmApp** remains valid for low-level embedding when you need direct runtime registry control without host overhead.

## Architecture (0.10+)

Layered, registry-driven model with a strictly pure core:

```
User / CLI / Services
        ↓
palm/app/          ApplicationHost (recommended), PalmApp (infra), PalmSettings
        ↓
palm/common/       Executions, plans, hooks, CQRS, outbox, compensation, transforms
        ↓
palm/core/         PURE engines — Behavior Tree, Orchestration, Context, Storage, Resource, Event, Auth, Transform
```

**Core purity rule:** `palm/core/` must never import from any other Palm package.

**Extension:** patterns, providers, storages register at bootstrap via registries — never by editing core.

## Key Capabilities

| Area | Highlights |
|------|------------|
| **Wizard** | Validation, collection steps, transform steps, backtracking, summary + commit |
| **Parallel** | Concurrent wizard branches with isolated scopes and merge strategies |
| **Transforms** | 22 built-in rules; pipelines, wizard steps, or `TransformExecutor` |
| **State** | `DictStateSchema`, scoped blackboards, schema-aware snapshots (`__palm:meta`) |
| **Persistence** | Durable `ProcessInstance` records; resume across restarts |
| **CQRS** | Command/query buses; host projections (`instance_index`, `job_status_board`); wizard CQRS in `patterns/wizard/bindings/cqrs/` |
| **Reliability** | Transactional outbox, compensation handlers, optional webhook dispatch |
| **Runtimes** | Embedded, Daemon, Server (HTTP), CLI + REPL (host-backed) |
| **Resources (0.12)** | `ResourceDefinition`, `ResourceEngine.invoke()`, `ResourceLeaf`, `ResourceCatalog`, `palm` provider |
| **Wizard REST (0.13)** | `/v1/wizards` — submit, status, input, backtrack by `instance_id` |
| **Palm Explorer** | SSR hub at `/explorer` — wizard workspace (HTMX), collection UI, **resources**; `GET /` redirects here |
| **MCP (0.14)** | 26 tools, 4 prompts, 10 resources — agent operator adapter; see `docs/MCP.md` |
| **Dashboard** | `palm status` — projection-backed Rich overview |
| **Reliability** | Resource compensation via `register_for_resource()`; `ResourceInvocationProjection` |

## Quick Commands

```bash
pip install palmengine[cli]
palm doctor
palm status
palm flow start onboard
palm flow start schema-onboard
palm flow start todo-builder
palm flow start parallel-demo
palm resource list
palm resource describe fetch-customer
palm resource invoke fetch-customer customer_id=42
```

From source: `uv sync --group dev --extra cli && uv pip install -e ".[cli]"`

Server + Explorer:
```bash
python -c "from palm.runtimes.server import ServerRuntime, run_server; run_server(ServerRuntime())"
# → http://localhost:8080/explorer (root / redirects here)
curl -s -X POST http://localhost:8080/v1/wizards -H 'Content-Type: application/json' -d '{"flow_name":"todo-builder"}'
# → open /explorer/instances/<instance_id> for wizard workspace + collection UI
```

## Agent development with MCP (0.14)

**Prerequisite:** `palm server` running on `:8080` before connecting MCP.

```bash
uv sync --extra mcp && just palm-server    # terminal 1
just mcp-inspector                         # terminal 2 (optional)
```

Grok: `.grok/config.toml` registers `palm-mcp` (stdio). Cursor/Claude: stdio command `palm-mcp` or `uv run --extra mcp palm-mcp`.

### Operator loop (memorize this)

`definitions → submit → inspect → input → wait on children → resume`

### Rules for agents

1. **Instance-first** — use `instance_id` for wizards (`palm_inspect_instance`, `palm_wizard_input`). `job_id` is a handle, not a durable instance key.
2. **Plain input** — `palm_wizard_input(instance_id, input="yes")` not JSON blobs. `yes`/`no` coerce to boolean on confirm steps.
3. **Compact inspect** — default slim responses; `format="verbose"` only when debugging.
4. **Resources = read, tools = write** — MCP read URIs (`palm://definitions/*`, `palm://agent/guide`) via FetchMcpResource; REST resource definitions (`knowkey.search_nodes`, `fetch-customer`) via `palm_invoke_resource`. Never pass `palm://` to invoke.
5. **Compositional wizards** — check `waiting_for_child`, read `palm://instances/{id}/tree`, call `palm_resume_child_wait` or inspect child.
6. **Collection steps** — use `palm_wizard_collection_action`, not guessed strings.
7. **Interactive entry** — `palm_submit_wizard(flow_name=…)` for wizards. Never `palm_submit_process` on catalog processes (e.g. `knowkey_compose`); read `palm://definitions/processes/{name}` for `submit_hint` / `mcp_default_entry`.
8. **Batch drive** — `palm_wizard_drive` accepts plain strings in `inputs`, `payload={…}` alone for one structured step, or both. After commit, `palm_compose_status` includes `result` / `result_summary`.

### Typical session

```
palm_doctor
palm_submit_wizard(flow_name="todo-builder")     → instance_id
palm_inspect_instance(instance_id)                 → step, prompt, choices
palm_wizard_input(instance_id, input="…")          → plain string
# child-wait: palm_resume_child_wait(instance_id)
# stuck: prompt debug-wizard-block + palm_compose_status(instance_id)
```

### Tool cheat sheet

| Need | Tool / resource |
|------|-----------------|
| Project context | `palm://agent/guide` (this file) |
| Flow catalog | `palm://definitions/flows` |
| Start wizard | `palm_submit_wizard` |
| Current step | `palm_inspect_instance` |
| Send answer | `palm_wizard_input(input=…)` |
| Parent/child stack | `palm://instances/{id}/tree`, `palm_compose_status` |
| Collection UI | `palm_wizard_collection_action` |
| Dry-run flow | `palm_validate_flow` |
| Health | `palm_doctor` |
| Read catalogs / guide | `palm://definitions/*`, `palm://agent/guide` (FetchMcpResource) |
| Invoke resource definition | `palm_invoke_resource` (definition name, not `palm://`) |
| Commit result / node IDs | `palm_compose_status`, `palm_fetch_job` |
| Multi-step burst | `palm_wizard_drive` (`payload` for structured JSON) |

**Prompts:** `debug-wizard-block`, `drive-wizard-to-step`, `explain-compositional-stack`, `operator-handoff`

Full inventory: `docs/MCP.md`

Durable local storage:
```bash
export PALM_STORAGE_BACKEND=filesystem
export PALM_DATA_DIR=./data
```

## Package Layout

```
src/palm/
├── app/            # ApplicationHost, PalmApp, HostProfile, create_cli_host()
├── core/           # Pure engines (no external palm imports)
├── common/         # CQRS, hooks, persistence, transforms, runtimes base
├── patterns/       # PatternApp + bindings/ + flow/ (wizard, parallel, pipeline, dag, etl)
├── providers/      # rest, graphql, postgres (extensible)
├── storages/       # memory, filesystem, postgres, mongodb (extensible)
├── definitions/    # FlowDefinition, ProcessDefinition, ResourceDefinition
├── instances/      # ProcessInstance, StateSnapshot
└── runtimes/       # Thin CLI, embedded, daemon, server surfaces
```

## Key Documents

| Document | Purpose |
|----------|---------|
| AGENTS.md | Constitution for AI agents and contributors |
| STATUS.md | Living current state |
| ARCHITECTURE.md | Layers, ApplicationHost, CQRS, reliability |
| DEVELOPMENT.md | Contributor setup and patterns |
| docs/PATTERN-APPS.md | PatternApp manifests, bindings/flow layout, common boundary |
| docs/PROVIDER-APPS.md | ProviderApp manifests, bindings/flow layout for providers; palm reference |
| docs/adr/002-pattern-apps-and-common-boundaries.md | ADR for pattern/common split |
| EXPLORER-WIZARD.md | Explorer wizard workspace + collection UI guide |
| docs/MCP.md | **Agent MCP guide** — setup, workflows, full tool inventory |
| DEVELOPMENT.md | Contributor setup + MCP development workflow |
| MIGRATION-0.12.md | Upgrade from 0.11.x wizard action steps |
| MIGRATION-0.10.md | Upgrade from 0.9.x bootstrap paths |
| docs/VISION-0.13.md | Wizard Experience release vision |
| docs/VISION-0.12.md | Compositional Power vision and best practices |
| examples/README.md | Runnable flow definitions |

## Extension Patterns

| Add | Where | How |
|-----|-------|-----|
| Pattern | `palm/patterns/<name>/` | `pattern.py` + `app.py` + `bindings/definitions/builder.py` + `registry.py`; add to `INSTALLED_PATTERNS`. See `docs/PATTERN-APPS.md` |
| Provider | `palm/providers/<name>/` | `provider.py` + `app.py` (ProviderApp) + `registry.py` + `bindings/` + `flow/`; add to `INSTALLED_PROVIDERS`. See `docs/PROVIDER-APPS.md` |
| Storage | `palm/storages/<name>/` | Same structure; add to `INSTALLED_STORAGES` |
| Transform rule | `palm/common/transforms/rules/` | `BaseTransformRule` + `register_transform()` |
| CQRS handler | `palm/patterns/<name>/bindings/cqrs/` | Register via `register_cqrs_contributor()` in `PatternApp.ready()` |

## Resources (0.12 — Compositional Power)

Resources are first-class definitions (symmetric with flows/processes):

- `ResourceDefinition` in `DefinitionRepository` — register via bootstrap or CLI
- `ResourceLeaf` — core BT node; wizard `step_kind: resource` with `resource_ref`
- `ResourceEngine.invoke()` — bind `{{ state.* }}` params, emit `resource.*` events
- `palm` provider — `submit_flow`, `submit_process`, `invoke_resource`, `fetch` (local or remote HTTP)
- `enrich_resource` transform accepts `resource_ref`; compensation via `register_for_resource()`
- Explorer: `/explorer/resources` catalog, detail, Try Invoke; timelines on instances/jobs

**Best practices:** define once, reference by `resource_ref`; cache definitions not mutating results; use `palm doctor` / Explorer before invoke.

Vision: `docs/VISION-0.12.md` · Migration: `MIGRATION-0.12.md` · ADR: `docs/adr/001-compositional-power-resources.md`

## Invariants (Never Break)

- Job state transitions only through `RunResult` + `OrchestrationEngine.apply_result()`
- Registries use `threading.RLock`; populate at bootstrap, not in hot paths
- Resume authority: `ProcessInstance.state_snapshot` (not historical `state_snapshots[]`)
- Never import from `archive/` in new code

## Repository

https://github.com/JGabrielGruber/palmengine