Metadata-Version: 2.4
Name: stemmata
Version: 1.1.0
Summary: Hierarchical prompt resolver CLI
Author: Pedro Javier Martos Velasco
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/pjmartos/stemmata
Project-URL: Repository, https://github.com/pjmartos/stemmata
Project-URL: Issues, https://github.com/pjmartos/stemmata/issues
Keywords: prompt,yaml,inheritance,composition,llm,cli
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Markup
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: PyYAML==6.0.3
Requires-Dist: jsonschema==4.26.0
Dynamic: license-file

# stemmata

[![CI](https://github.com/pjmartos/stemmata/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/pjmartos/stemmata/actions/workflows/tests.yml)
[![Audit](https://github.com/pjmartos/stemmata/actions/workflows/audit.yml/badge.svg?branch=main)](https://github.com/pjmartos/stemmata/actions/workflows/audit.yml)
[![PyPI](https://img.shields.io/pypi/v/stemmata?v=2)](https://pypi.org/project/stemmata/)
[![Python Version](https://img.shields.io/badge/python-3.12%2B-blue)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)

Prompt reuse across repositories is a mess. You copy a YAML prompt into a new project, tweak it, and within a week the original and the copy have diverged. Multiply that by a dozen services and you're maintaining the same boilerplate in twenty places.

`stemmata` fixes this with hierarchical composition: prompts declare ancestors (by relative path or by registry coordinate), and the CLI resolves the full inheritance chain into a single, deterministic YAML document. Ancestor prompts are distributed as npm packages through any private registry you already run.

## Table of Contents

- [Features](#features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [CLI Reference](#cli-reference)
- [Prompt Format](#prompt-format)
- [Merge Semantics](#merge-semantics)
- [Exit Codes](#exit-codes)
- [Configuration](#configuration)
- [Testing](#testing)

## Features

- **Hierarchical composition**: prompts declare `ancestors` as paths or `(package, version, prompt)` coordinates; the full transitive closure is resolved eagerly via breadth-first search.
- **Deterministic merging**: nearest-wins for scalars and lists, deep-merge for maps, with breadth-first search distance plus reference occurring-ordering (for breaking ties) so the output is reproducible.
- **Placeholder interpolation**: `${path}` references resolve against the merged namespace, with structural, textual, and list-splat forms.
- **Abstract placeholders**: a mid-graph prompt may declare required "holes" via `${abstract:<dotted-path>}`, and any descendant is free to fill them — like the template-method pattern in OOP. `resolve` hard-fails on unfilled holes (exit `16`); `publish`, `describe`, `tree`, and `validate` report them as contract information and keep working.
- **Text resource embedding**: packages may ship resource payloads alongside prompts — `markdown`, `text`, `xml`, `json`, or `yaml` — and splice them in as opaque text via `${resource:...}`, resolved eagerly on the same cache and registry rails as prompts.
- **npm registry transport**: speaks the standard npm REST API; credentials read from `~/.npmrc`.

## Installation

```
pip install stemmata
```

Requires **Python 3.12+** (for `tarfile.data_filter`). Third-party dependencies: `PyYAML` and `jsonschema`.

## Quick Start

```bash
# You have a local prompt that inherits from a base — resolve it:
stemmata resolve ./prompts/onboarding.yaml

# Or resolve a prompt published to your registry by coordinate:
stemmata resolve '@acme/prompts-core@1.2.3#onboarding'

# Describe every prompt in a published package (or a single one):
stemmata describe '@acme/prompts-core@1.2.3'
stemmata describe '@acme/prompts-core@1.2.3#onboarding'

# Print the ancestor DAG as an ASCII tree:
stemmata tree ./prompts/onboarding.yaml
stemmata tree '@acme/prompts-core@1.2.3#onboarding'

# Need machine-readable output for a script or pipeline:
stemmata --output json resolve ./prompts/onboarding.yaml

# Validate a prompt (or an entire directory) against its $schema:
stemmata validate ./prompts/onboarding.yaml
stemmata validate ./prompts/

# Wipe the local cache (by default stored under ~/.cache/stemmata):
stemmata cache clear

# Scaffold a package.json for an existing directory of prompts and resources:
stemmata init ./my-package

# Install a local package into the cache so it can be resolved offline by coordinate:
stemmata install ./my-package
```

## CLI Reference

```
stemmata [GLOBAL FLAGS] <subcommand> [ARGS]
```

### Global flags

| Flag                        | Default                    | Description                                                                                  |
|-----------------------------|----------------------------|----------------------------------------------------------------------------------------------|
| `--output {yaml,json,text}` | `yaml` (`text` for `tree`) | Output format.                                                                               |
| `--verbose`                 | off                        | Timestamped diagnostics on stderr.                                                           |
| `--offline`                 | off                        | Forbid network access; exit `22` if a fetch would be needed.                                 |
| `--refresh`                 | off                        | Re-fetch artifacts even if cached.                                                           |
| `--cache-dir <path>`        | `~/.cache/stemmata`        | Override the cache root. Honours `$PROMPT_CLI_CACHE_DIR` when the flag is absent.            |
| `--npmrc <path>`            | `~/.npmrc`                 | Override the npmrc file used for registry routing and credentials.                           |
| `--version`                 | —                          | Print version and exit.                                                                      |

### `resolve <target>`

Resolves a single prompt. Target is either a local path (`./prompts/onboarding.yaml`) or a registry coordinate (`@<scope>/<name>@<version>#<prompt-id>`).

Resource limits: `--max-prompts` (default 1000), `--max-depth` (default 50), `--max-download-size` (default 64 MiB per package), `--max-total-size` (default 512 MiB per invocation), `--http-timeout` (default 30s), `--timeout` (default 5m).

`--set <dotted-path>=<yaml-value>` overrides the merged value at that path. Repeatable; last-wins on duplicate paths. Values are parsed as YAML, so `--set port=5432` is an int, `--set enabled=true` is a bool, `--set tags=[a,b,c]` is a list, and `--set body=` is `null`. Overrides merge at BFS distance `-1` — nearer than the root prompt — so they beat every ancestor and can satisfy `${abstract:…}` markers. They show up in the JSON envelope's `ancestors[]` as `{canonical_id: "<overrides>", distance: -1}`. `--set` is only accepted by `resolve`.

On success, stdout carries the resolved YAML (or a JSON envelope with `{root, content, ancestors[]}`). On failure, stdout carries a JSON error envelope regardless of `--output`, and stderr gets a one-line human-readable summary.

### `publish [path]`

Builds and uploads the package at `path` (default `.`) to the registry routed by `~/.npmrc`. Before any bytes leave the machine, every prompt listed in `package.json` is checked for: (1) ancestor cycles, (2) intra-document type conflicts, (3) placeholder resolvability against the fully resolved namespace, (4) `dependencies` consistency with the cross-package references found in the prompts (including those inside `${resource:...}` placeholders), (5) manifest closure under relative-path references — every local `ancestors` entry must resolve to a path that is itself declared in `prompts`, since only manifest-listed files are bundled, (6) `$schema` validation against the prompt's content contract, and (7) resource-graph integrity — every `${resource:...}` occupies an allowed position, every local resource reference resolves to an entry in the `resources` array, and the Markdown-embedding graph contains no cycles. All errors discovered in the pass are aggregated into a single envelope; the headline exit code is the most severe one (cycle > schema > reference > merge > placeholder).

Abstract placeholders (`${abstract:<dotted-path>}`) are **not** treated as failures by `publish`: a library package whose prompts contain unfilled abstracts is the whole point. For every prompt that still has unfilled abstracts, `publish` logs a `warning:` line to stderr listing the holes, records them under `abstracts` in the success payload, and defers `$schema` content validation for that prompt (per the per-prompt all-or-nothing rule). Real placeholder / reference / merge errors found alongside abstracts still fail publish as before.

Flags: `--dry-run` (build the tarball but skip upload), `--tarball <path>` (write the built tarball to `path`). The tarball is deterministic: identical inputs produce byte-identical output.

`$schema` enforcement is always on. An unfetchable `$schema` URI (missing local file, offline with no cache, or network failure) aborts with exit code `10`.

### `validate <target>`

Validates prompt files against their `$schema`. Target is a file path or a directory (recursively discovers `.yaml`, `.yml`, `.json` files). For YAML prompts with ancestors, the full resolve → merge → interpolate pipeline runs before validation so inherited and interpolated values participate.

Multi-document YAML files (separated by `---`) are supported — each sub-document is validated independently against its own `$schema`. Files without `$schema` are silently skipped.

All violations are collected and reported together. Error payloads include the natural source line number of the offending value.

Flags: the same resource-limit flags as `resolve`. An unfetchable `$schema` URI (missing file, offline with no cache, or network failure) aborts with exit code `10`.

`$schema` enforcement supports `file://`, `http://`, and `https://` URIs, as well as bare relative paths (resolved against the validated file's directory).

### `describe <target>`

Resolves every prompt in a published package, or a single prompt inside it. Target is either `@<scope>/<name>@<version>` (describe the whole package) or `@<scope>/<name>@<version>#<prompt-id>` (describe one prompt). Each prompt is resolved with ancestors merged and placeholders interpolated, using the same pipeline as `resolve`.

Default YAML output emits one document per prompt, separated by `---` start markers. Each sub-document is prefixed with a `# <canonical-id>` comment (e.g. `# @acme/prompts-core@1.2.3#onboarding`) so the reader can tell which prompt is which. `--output json` returns an array of `{root, content, ancestors[]}` entries in manifest declaration order (length 1 when targeting a single prompt). Package artifacts are fetched through the usual cache (`~/.cache/stemmata` by default), so repeated invocations reuse downloaded tarballs; `--offline` and `--refresh` behave as with `resolve`.

Resource-limit flags match `resolve`.

### `tree <target>`

Prints the ancestor DAG rooted at `<target>`, which takes the same two forms as `resolve` (a local YAML/JSON path or a `@<scope>/<name>@<version>#<prompt-id>` coordinate). The resolver runs the same eager pipeline as `resolve`, so cycles, missing ancestors, and version conflicts surface with the usual exit codes; `--offline` / `--refresh` and the resource-limit flags all apply.

Default `--output text` produces an ASCII tree (`|-- ` / `` `-- `` connectors). Text resources reached via `${resource:...}` are rendered inline under the prompt (or resource) that references them and are prefixed with `resource:` to disambiguate them from prompt coordinates. Diamond inheritance — across both ancestor and resource edges — is rendered once in full and subsequent visits are marked `(seen)` so the output stays finite:

```
root.yaml
|-- a.yaml
|   `-- x.yaml
|-- b.yaml
|   `-- x.yaml  (seen)
`-- resource:@acme/prompts-core@1.2.3#playbook
    `-- resource:@acme/prompts-core@1.2.3#safety
```

`--output yaml` / `--output json` emit a `{root, nodes[], edges[]}` envelope instead, with each node carrying its canonical id, source file, BFS distance from the root, and `kind` (`prompt` or `resource`). Edges carry `kind` (`ancestor` or `resource`).

### `init [path]`

Scaffolds (or updates) a `package.json` at `path` (default `.`). Scans `./prompts` recursively for `.yaml`, `.yml`, and `.json` files and `./resources` recursively for `.md`, `.txt`, `.xml`, `.json`, `.yaml`, and `.yml` files, deriving each entry's `id` from the basename and setting `contentType` from the extension (`.md` → `markdown`, `.txt` → `text`, `.xml` → `xml`, `.json` → `json`, `.yaml`/`.yml` → `yaml`). Entries are sorted alphabetically by path and rendered one-per-line.

### `install [path]`

Installs the package at `path` (default `.`) into the local cache.

### `cache clear`

Evicts every cached entry.

## Prompt Format

A prompt is a structured mapping (YAML or JSON) with reserved envelope keys plus arbitrary content:

```yaml
ancestors:
  - ../base.yaml                         # relative path (within package)
  - package: "@acme/common"              # cross-package coordinate
    version: "1.0.4"
    prompt: "defaults"
$schema: "https://schemas.example/foo.v1.json"   # optional, enforced at publish time if present

database:
  host: "db.internal"
  port: 5432
body: |
  Region is ${vars.region}; DB is ${database.host}:${database.port}.
```

`ancestors` and `$schema` are stripped from the namespace; every other key is addressable via dotted path.

### Package manifest (`package.json`)

```json
{
  "name": "@acme/prompts-core",
  "version": "1.2.3",
  "license": "UNLICENSED",
  "dependencies": { "@acme/common": "1.0.4" },
  "prompts": [
    { "id": "base",       "path": "prompts/base.yaml",             "contentType": "yaml" },
    { "id": "onboarding", "path": "prompts/extra/onboarding.yaml", "contentType": "yaml" }
  ],
  "resources": [
    { "id": "overview", "path": "resources/overview.md", "contentType": "markdown" }
  ]
}
```

`name` must be `@<scope>/<n>`. `version` is strict SemVer, no ranges. `prompts` is non-empty; `id` defaults to basename without extension and must match `[a-z0-9][a-z0-9_-]*`. `resources` is optional; ids, paths, and case-folded path uniqueness are shared across the union of `prompts` and `resources`.

### Text resources

Prompts may embed opaque text payloads via `${resource:<POSIX-relative-path>}` (same-package) or `${resource:@<scope>/<name>@<version>#<id>}` (coordinate). A resource entry's `contentType` may be `markdown`, `text`, `xml`, `json`, or `yaml`; the runtime treats every payload as opaque text — the `contentType` is metadata for downstream consumers. The reference must stand alone — either as the sole content of a line inside a block scalar or a resource file, or as the entire trimmed text of a flow-style YAML scalar or JSON string. Resource payloads are substituted verbatim after ancestor-namespace interpolation; they do not contribute keys to the merged namespace and any `${...}` sequences inside them are left literal.

### Abstract placeholders

An author can mark a dotted-path as a required "hole" that any descendant must fill before the graph becomes resolvable. **Every newly introduced abstract MUST be documented** in the prompt's top-level `abstracts:` block:

```yaml
# @acme/prompts-core#persona (a reusable mid-graph prompt)
abstracts:
  persona.tone:
    description: persona's conversational tone (e.g. "friendly", "formal")
    type: string                           # default; may be omitted
    example: friendly
  persona.steps:
    description: ordered subroutine names this persona executes
    type: list

persona:
  tone: "${abstract:persona.tone}"         # required: any descendant must set persona.tone
  greeting: "Hi — my tone is ${abstract:persona.tone}."
  steps: "${abstract:persona.steps}"
```

A descendant fills the hole by providing a concrete value at the same dotted path:

```yaml
# a concrete child
ancestors:
  - package: "@acme/prompts-core"
    version: "1.0.0"
    prompt: "persona"

persona:
  tone: "friendly"
  steps: ["greet", "ask", "answer"]
```

Annotation fields:

- **`description`** (required, non-empty string). Human-readable prose describing the contract step. Surfaces in `describe`, `tree`, and `validate` output so downstream callers know what to provide.
- **`type`** (optional, defaults to `"string"`). One of `"string"` or `"list"`. A `type: "list"` marker MUST appear in structural position only — embedding a list-typed marker inside a larger string scalar is a publish-time error.
- **`example`** (optional, any). Informational value satisfying the declared type; not validated.

Semantics:

- **Syntax.** `${abstract:<dotted-path>}` — the prefix mirrors `${resource:…}` so dispatch is unambiguous and the form is JSON-safe.
- **Usable positions.** A `string`-typed marker may appear as the sole content of any scalar (flow or block) or positionally inside a larger string scalar (`"prefix ${abstract:x} suffix"`). A `list`-typed marker may appear only in structural position; the resolved sequence then participates in list-splat rules.
- **A hole is unfilled** iff, after BFS merge, the nearest value at the referenced dotted path is (a) absent, (b) explicit `null` (null shadowing does **not** satisfy an abstract), or (c) itself another `${abstract:…}` marker (an abstract does not satisfy another abstract). The per-case `reason` is reported in the error envelope as `not_provided`, `null_shadow`, or `abstract_inherited`.
- **Type-shape gate.** When the resolved value's JSON type contradicts the annotation `type` (a `list`-typed marker resolved with a string, or vice versa), the resolve fails with exit `16` and `reason: "type_mismatch"`; the envelope carries `declared_type` and `actual_type`.
- **`$schema` consistency.** When the introducing prompt also carries `$schema`, every subcommand that loads the prompt verifies that the schema's constraint at the abstract's dotted path is consistent with the annotation `type`; a contradiction is exit `10` (`reason: "schema_type_mismatch"`).
- **Declaration coupling.** Annotations belong to the originating declarer: the prompt that first introduces a marker MUST annotate it; descendants that re-use the inherited marker MUST NOT re-annotate it.
- **Per-prompt all-or-nothing validation.** For any given prompt, if its merged namespace has zero unfilled abstracts, placeholder interpolation and `$schema` content-contract validation run as normal. If one or more abstracts remain unfilled, both checks are deferred for that prompt — nothing about that prompt's content contract can be enforced until the contract is fulfilled.

Subcommand behaviour:

| Command    | Unfilled abstracts present                                                                                                                                                                                                                                                                                                                                      |
|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `resolve`  | Hard-fails with exit `16`. The resolved artefact is not produced while any hole remains.                                                                                                                                                                                                                                                                        |
| `validate` | Does **not** fail. Structural checks and cycle detection still run. Abstracts are reported under `abstracts` in the success payload, each entry carrying the originating declarer's `annotation`.                                                                                                                                                               |
| `publish`  | Does **not** fail. A `warning:` line is logged to stderr listing the unfilled abstracts, and each one is recorded under `abstracts` in the success payload. Schema validation is deferred for any prompt that still has holes.                                                                                                                                  |
| `describe` | Always works. Emits two labelled buckets per prompt: `abstracts.declared` (markers introduced by *this* prompt) and `abstracts.inherited` (declared in an ancestor and still unfilled here). Each entry carries the declarer's `annotation`. The default YAML output adds one `# abstract <path> (<type>): <description-first-line>` comment per surfaced hole. |
| `tree`     | Always works. Each prompt node is annotated with `[abstracts: <path>: <type>, ...]` listing the markers it introduces with their declared type; the JSON/YAML envelope adds an `abstracts` array of `{path, annotation}` to each node.                                                                                                                          |

## Merge Semantics

Reachable prompts are layered by breadth-first search distance from the root (distance 0 = root, wins everything). Ties at the same distance break by enqueue order.

**Maps** are deep-merged, with the nearer value winning at each leaf:

```yaml
# ancestor (distance 1)           # root (distance 0)
database:                         database:
  host: "base.internal"             host: "override.internal"
  port: 5432                        ssl: true
```

Resolved: `database.host` = `"override.internal"` (nearer wins), `database.port` = `5432` (survives from ancestor), `database.ssl` = `true` (only root provides it).

**Lists** replace wholesale — no element-level merge. **`null`** at a nearer layer shadows the entire subtree beneath it.

For the full interpolation reference (structural vs. textual placeholders, list splat, non-splat `${=...}` form, escaping, version conflict resolution), see [`docs/interpolation.md`](docs/interpolation.md).

## Exit Codes

| Code | Meaning                                                       |
|------|---------------------------------------------------------------|
| `0`  | Success                                                       |
| `1`  | Generic / unexpected failure                                  |
| `2`  | Usage error                                                   |
| `10` | Schema validation error                                       |
| `11` | Unknown ancestor, prompt, or resource reference               |
| `12` | Cycle detected (ancestor or resource graph)                   |
| `14` | Unresolvable placeholder                                      |
| `15` | Merge / interpolation type mismatch                           |
| `16` | Abstract placeholder unfilled or wrong-typed (from `resolve`) |
| `20` | Network / registry error                                      |
| `21` | Cache error                                                   |
| `22` | Offline-mode violation                                        |

On failure, stdout always carries a JSON error envelope with `{status, exit_code, command, error: {code, category, message, ...}}` regardless of `--output`. Stderr gets a single-line human summary.

## Configuration

Registry routing and credentials come from `~/.npmrc` for both fetch and publish.

## Testing

```
PYTHONPATH=src python -m pytest tests/ -q
```
