Metadata-Version: 2.4
Name: pisama
Version: 0.5.6
Summary: Multi-agent failure detection for production AI systems
Project-URL: Homepage, https://pisama.ai
Project-URL: Documentation, https://docs.pisama.ai
Project-URL: Repository, https://github.com/Pisama-AI/pisama-python
Project-URL: Issues, https://github.com/Pisama-AI/pisama-python/issues
Project-URL: Changelog, https://github.com/Pisama-AI/pisama-python/blob/main/CHANGELOG.md
Author-email: Pisama Team <team@pisama.ai>
License-Expression: MIT
License-File: LICENSE
Keywords: agents,ai,failure-detection,monitoring,multi-agent,observability
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Requires-Dist: click>=8.0
Requires-Dist: httpx>=0.27
Requires-Dist: pisama-core<2,>=1.8.2
Requires-Dist: rich>=13.0
Provides-Extra: auto
Requires-Dist: opentelemetry-api>=1.20.0; extra == 'auto'
Requires-Dist: opentelemetry-sdk>=1.20.0; extra == 'auto'
Requires-Dist: pisama-auto>=0.1.0; extra == 'auto'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: langgraph
Requires-Dist: langchain-core>=0.3.0; extra == 'langgraph'
Requires-Dist: langgraph>=0.2.0; extra == 'langgraph'
Provides-Extra: mcp
Requires-Dist: mcp<2,>=1.0.0; extra == 'mcp'
Description-Content-Type: text/markdown

# Pisama

**Find and fix failures in AI agent systems. No LLM calls required.**

[![PyPI](https://img.shields.io/pypi/v/pisama?color=blue)](https://pypi.org/project/pisama/)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)

Pisama ships 32 core heuristic detectors. They apply across frameworks including n8n, LangGraph, Dify and OpenClaw, with per-platform gating (for example `coordination` runs only on multi-agent platforms). They run locally with zero LLM cost on the heuristic tier. An archived, in-distribution run on the [TRAIL benchmark](https://arxiv.org/abs/2505.08638) reports 59.9% joint accuracy (span and category). It is not a held-out result: 144 of the 148 traces appeared in calibration material, and the published archive does not contain the prediction-level data needed to recompute joint accuracy. The public confusion counts do reproduce the reported macro-F1 (0.7535) and micro-F1 (0.7463). See the [benchmark evidence and its reproducibility boundary](https://github.com/Pisama-AI/pisama-detectors/tree/main/benchmarks).

## Install

```bash
pip install pisama
```

## Usage

```python
from pisama import analyze

result = analyze("trace.json")  # also accepts dicts and JSON strings

for issue in result.issues:
    print(f"[{issue.type}] {issue.summary} (severity: {issue.severity})")
    print(f"  Fix: {issue.recommendation}")
```

## CLI

```bash
pisama analyze trace.json          # Analyze a trace
pisama watch python my_agent.py    # Watch a live agent (pip install pisama[auto])
pisama replay <trace-id>           # Re-run detection on stored traces
pisama smoke-test --last 50        # Batch test recent traces
pisama detectors                   # List all 32 core detectors
pisama mcp-server                  # Start MCP server (pip install pisama[mcp])
```

## MCP Server

Works in Cursor, Claude Desktop, and Windsurf. No API key is needed:

```json
{
  "mcpServers": {
    "pisama": { "command": "pisama", "args": ["mcp-server"] }
  }
}
```

## Detectors

32 core detectors, gated per platform (n8n, LangGraph, Dify, OpenClaw and others). A representative selection:

| Detector | What It Catches |
|----------|----------------|
| `loop` | Infinite loops, retry storms, stuck patterns |
| `coordination` | Deadlocked handoffs, message storms |
| `hallucination` | Factual errors, fabricated tool results |
| `injection` | Prompt injection, jailbreak attempts |
| `corruption` | State corruption, type drift |
| `persona_drift` | Persona drift, role confusion |
| `derailment` | Task deviation, goal drift |
| `context` | Context neglect, ignored instructions |
| `specification` | Output vs. requirement mismatch |
| `communication` | Inter-agent message breakdown |
| `decomposition` | Poor task breakdown, circular dependencies |
| `workflow` | Unreachable nodes, missing error handling |
| `completion` | Premature completion, unfinished work |
| `withholding` | Suppressed findings, hidden errors |
| `convergence` | Metric plateau, regression, thrashing |
| `overflow` | Context window exhaustion |
| `propagation` | Silent error propagation across steps |
| `citation` | Fabricated citations and source misattribution |
| `routing` | Inputs misrouted to the wrong specialist agent |
| `mcp_protocol` | MCP tool-communication failures |

## Benchmark Results

**TRAIL** (trace-level failure detection, 148 traces). Archived April 2026 run, **in-distribution and not held out**: 144 of 148 traces appeared in calibration material.

| Method | Joint Accuracy |
|--------|---------------|
| Pisama archived run | 59.9% |

Reproducible from the archive: macro-F1 0.7535, micro-F1 0.7463 (`benchmarks/verify_report.py`).
Joint accuracy is **not** recomputable from the public archive, which lacks prediction-level
data. LLM-judge baselines are omitted here because
[`trail_llm_baselines.json`](https://github.com/Pisama-AI/pisama-detectors/blob/main/benchmarks/trail_llm_baselines.json)
carries confusion counts but `"result": null` for every model, so no comparable joint-accuracy
figure exists in the artifact. See the
[reproducibility boundary](https://github.com/Pisama-AI/pisama-detectors/tree/main/benchmarks).

**Who&When** (ICML 2025, multi-agent attribution, 58 hand-crafted cases):

| Method | Agent Accuracy | Step Accuracy |
|--------|---------------|---------------|
| GPT-5.4 Mini | 60.3% | 22.4% |
| **Pisama + Sonnet 4** | **60.3%** | **24.1%** |

## Links

- [Documentation](https://docs.pisama.ai)
- [GitHub](https://github.com/Pisama-AI/pisama-python)
- [Platform](https://pisama.ai)

## License

MIT

## Source boundary

This repository is the public source for the MIT-licensed `pisama` Python
package. It does not contain the Pisama Cloud backend, dashboard, calibration
data, managed detection tiers, or paid automation.
