Metadata-Version: 2.4
Name: persevere
Version: 0.1.1
Summary: The Perseverance Composition Engine
Project-URL: Homepage, https://leithdocs.com
Project-URL: Documentation, https://codeberg.org/leithdocs/persevere#readme
Project-URL: Repository, https://codeberg.org/leithdocs/persevere
Author-email: William Waites <ww@leithdocs.com>
License: AGPL-3.0-or-later
License-File: LICENSE
Keywords: agents,composition,document-generation,llm,mcp,multi-agent,plumbing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: General
Requires-Python: >=3.14
Requires-Dist: aiosqlite>=0.22.1
Requires-Dist: alembic>=1.18.1
Requires-Dist: ddgs>=9.10.0
Requires-Dist: genai-prices<0.1,>=0.0.55
Requires-Dist: gitpython>=3.1.46
Requires-Dist: greenlet>=3.3.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: mammoth<2.0,>=1.8.0
Requires-Dist: mcp>=1.25.0
Requires-Dist: opentelemetry-api>=1.20.0
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.20.0
Requires-Dist: opentelemetry-sdk>=1.20.0
Requires-Dist: pydantic-ai>=1.44.0
Requires-Dist: pypdf>=6.0.0
Requires-Dist: python-magic>=0.4.27
Requires-Dist: sqlalchemy>=2.0.0
Requires-Dist: sqlite-vec>=0.1.6
Requires-Dist: structlog>=25.5.0
Requires-Dist: trafilatura>=2.0.0
Requires-Dist: uv>=0.9.26
Requires-Dist: uvicorn==0.40.0
Requires-Dist: xkcdpass>=1.30.0
Provides-Extra: dev
Requires-Dist: black>=24.1.0; extra == 'dev'
Requires-Dist: pre-commit>=3.6.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest>=7.4.0; extra == 'dev'
Requires-Dist: ruff>=0.1.14; extra == 'dev'
Provides-Extra: plumbing
Requires-Dist: persevere-plumbing>=1.0.0; extra == 'plumbing'
Description-Content-Type: text/markdown

# Perseverance Composition Engine (PCE)

> *"Persevere" is the motto of the city of Leith, where this software was produced. As mottos go, it is not bad.*

The **Perseverance Composition Engine (PCE)** is a multi-agent AI system designed for high-quality document generation through the lens of [**Artificial Organisations**](https://eprints.soton.ac.uk/508768/).

Unlike systems that rely solely on individual agent alignment, PCE achieves reliability through **structural constraints**: separation of duties, adversarial review, and information compartmentalisation. It is an "Artificial Organisation" that produces reliable collective behaviour from individually unreliable components.

---

## Philosophy: Artificial Organisations

Alignment research often focuses on making individual agents "helpful, harmless, and honest." PCE takes a different path, drawing from organisational theory (March & Simon, 1958). It assumes that agents have bounded rationality and handles risk through **structure**:

- **Separation of Duties**: Different agents handle drafting, fact-checking, and quality review.
- **Adversarial Review**: A dedicated Corroborator looks for fabrications, while a Critic (blinded to sources) evaluates argumentative quality.
- **Information Compartmentalisation**: "Chinese Walls" are enforced at the architectural level. An agent cannot see documents it is not permitted to see, regardless of instructions.

---

## Architecture

PCE is built on a three-layer architecture:

1. **Document Catalogue**: A metadata-driven persistent store. Documents are governed by **policy-defined visibility tiers** that control which agents can read or write each document.
2. **Agent Network**: A set of specialised agents, each with a specific institutional role and restricted tool access.
3. **Workflow Orchestrator**: A state machine (implemented via `PerseveranceGraph`) that manages iterative refinement loops and asynchronous execution.

### Agents

| Agent | Institutional Role | Primary Responsibility |
| :--- | :--- | :--- |
| **Consul** | Client Boundary | Interface for task specification and requirement elicitation. |
| **Curator** | Memory | Metadata management, categorisation, and institutional memory. |
| **Composer** | Production | Drafting and synthesis based on available source materials. |
| **Corroborator** | Verification | Fact-checking with full source access (adversarial). |
| **Critic** | Review | Independent quality evaluation without source access (blind review). |

The **Consul** serves as the client-facing interface and is not part of the internal agent graph. The four internal agents (Curator, Composer, Corroborator, Critic) participate in the iterative refinement cycle.

---

## The CCC Loop (Composer-Corroborator-Critic)

The core of PCE is the iterative refinement cycle:

1. **Drafting**: The **Composer** generates text from sources.
2. **Fact-Checking**: The **Corroborator** (with source access) verifies every claim.
   - If fabricated: returns to Composer with specific remediation steps.
3. **Expert Review**: If substantiated, the **Critic** (without source access) assigns a quality score (0–100).
   - If score < threshold: returns to Composer for improvement.
4. **Convergence**: The cycle continues until the draft is both well-substantiated and well-argued (typically score ≥ 85).

---

## Key Features

- **Action-Scope Permissions**: Architectural enforcement of privacy using an action-scope model. Every permission maps an action (`read`, `write`, `tasks`, `search`, `chat`) to a list of scope values. Agents and users see only documents matching their read permissions, can only modify documents matching their write permissions, and can only dispatch tasks to agents in their `tasks` scope.
- **Comprehensive Telemetry**: Granular tracking of LLM costs, tokens, and durations.
- **Granular Provenance**: Full audit trails using W3C PROV-O (`wasDerivedFrom`, `alternateOf`, `wasInfluencedBy`).
- **Atomic Versioning**: Git-backed document snapshots. Changes within a task are batched into a single atomic commit at completion, providing a clean audit trail with agent-specific authors and comments.

---

## Quick Start

### Prerequisites

- Python 3.14+
- [uv](https://github.com/astral-sh/uv) (recommended for dependency management)

### Install

```bash
pip install persevere
```

Or from source:

```bash
git clone https://codeberg.org/leithdocs/persevere
cd persevere
uv pip install -e .
```

To include plumbing support (typed agent composition):

```bash
pip install persevere[plumbing]
```

In all command-line examples, we assume that if you use `uv` as recommended, you know to prefix all commands with `uv run`.

Your data is all under `data/`, either in the SQLite database or in the tree under `data/files/`. You will normally be paying for LLM API access to use PCE, which means this data has cost money to produce.

### Initialise

```bash
pce -c config.yaml init
```

This sets up the local configuration, populates default prompts, and initialises a Git repository in the filestore for version control. There are reasons why you might want to run init on an already-populated directory. init tries hard to be non-destructive, so you should not lose the information under files/.

### Run

```bash
pce -c config.yaml cli
```

---

## Configuration

Create a `config.yaml` in your project root. Use `examples/config.yaml` as a template.

```yaml
pce:
  owner: you@example.org
  database:
    url: sqlite+aiosqlite:///./data/persevere.db
  filestore:
    path: ./data/files
  policies: policies/agent-config.yaml

agents:
  default:
    provider: anthropic
    model: claude-haiku-4-5

  composer:
    provider: anthropic
    model: claude-sonnet-4-6
    caching: true

  critic:
    provider: anthropic
    model: claude-sonnet-4-6
```

If you are running in cli mode for other than a brief test, you will want a logfile. By default logging output is sent to stdio, which makes a mess of the CLI:

```yaml
logging:
  filename: mypce-workspace.txt
```

### Visibility Tiers and Permissions

Document visibility and agent permissions are **policy-defined**, not hardcoded. The `policies/agent-config.yaml` file specifies which tiers exist, which agents can read from each tier, and which agents can write to each tier. This means the set of tiers and their names are entirely configurable for your deployment. See `policies/agent-config.yaml` for the defaults and [doc/FEATURES.md](doc/FEATURES.md) for a full explanation.

---

## Deployment Modes

### Interactive CLI

The primary interactive mode. Use `pce cli` to open a conversation with the Consul agent directly in your terminal.

```bash
pce -c config.yaml cli
```

Any number of cli sessions can be run on one computer at the same time, although only one session should be used with any one user context, or workspace as PCE calls it. It is always best to isolate multiple cli sessions. A good way to do this is with [Bubblewrap](https://github.com/containers/bubblewrap), for example:

```
#!/bin/bash

source ./setup # API keys and other environment variables

bwrap \
  --ro-bind /usr /usr \
  --ro-bind /lib /lib \
  --ro-bind /lib64 /lib64 \
  --ro-bind /bin /bin \
  --ro-bind /etc /etc \
  --ro-bind /etc/resolv.conf /etc/resolv.conf \
  --proc /proc \
  --dev /dev \
  --share-net \
  --new-session \
  --tmpfs /tmp \
  --bind $(pwd) /app \
  --chdir /app \
  uv run --link-mode=copy pce -c ./config.yaml cli
```

### Stdio MCP Server

For use with Claude Desktop, Claude Code, [Goose](https://github.com/block/goose) or any MCP-compatible client:

```bash
pce -c config.yaml mcp --transport stdio
```

### Network MCP Server with OAuth

For multi-user or remote deployments, the HTTP transport serves both legacy SSE (`/sse`) and streamable-HTTP (`/mcp`) endpoints. OAuth 2.1 is required for network deployments; see the frontend proxy (`persevere-frontend`) for the authentication layer.

```bash
pce -c config.yaml mcp --transport http --host 0.0.0.0 --port 8000
```

---

## Example Interaction

PCE is built for delegation. Long-running tasks are handled asynchronously via MCP tools:

```python
# Create a task and get its ID
task = task_create(
    title="Cover letter for Alice",
    remit="Compose a cover letter for Alice using her CV and the job advert.",
    start_agent="composer",
    success_threshold=85,
)

# Check progress
status = task_get(task_id=task.id)
# {"status": "active", "step": 2, ...}

# List all tasks
task_list(status="complete")
```

The Consul agent accepts natural language and translates it into `task_create` calls on your behalf. You can also call the tools directly.

---

## MCP Tool Reference

The following tools are exposed to authorised clients via the Consul:

| Tool | Description |
| :--- | :--- |
| `create` | Create a document from text content, an existing file, or a URL |
| `read` | Read a document's text content |
| `update` | Update a document's content or metadata |
| `edit` | Partially edit a document by line range or pattern replace |
| `delete` | Delete a document |
| `metadata` | Return metadata for a document |
| `move` | Move a document to a new path |
| `copy` | Copy a document with provenance tracking |
| `history` | Git history for a document or the entire project |
| `diff` | Diff a document between two versions |
| `version` | Get detailed information about a specific document version |
| `search` | Full-text and semantic search across documents |
| `grep` | Regexp search within document contents |
| `websearch` | Web search via DuckDuckGo |
| `newssearch` | News search via DuckDuckGo |
| `dispatch` | Create and dispatch a new agentic task |
| `task` | Get status and details of a task by ID |
| `tasks` | List tasks, optionally filtered by status |
| `cancel` | Abort a running task |
| `reindex` | Rebuild the SQLite index from sidecar files in the filestore |
| `verify` | Verify consistency between SQLite and sidecar files |
| `whoareyou` | Identify the connected user and their role |

---

## Documentation

- [Features](doc/FEATURES.md): Detailed look at PROV-O, visibility, and tool design.
- [Evaluation](doc/EVALUATION.md): How to use the test harness and metrics.
- [Technical Notes](doc/NOTES.md): Architecture background and substrate choices.
- [Testing](doc/TESTING.md): Strategy for unit and E2E tests.

---

## Licence

Copyright 2025–2026 William Waites / Leith Document Company Limited.

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

See [LICENSE](LICENSE) for the full text.
