Metadata-Version: 2.4
Name: tensile
Version: 0.10.1
Summary: Improve AI agent reliability through LLM trajectory logging, analysis, and remediation
Author: DataRobot
Project-URL: Repository, https://github.com/datarobot-oss/tensile
Project-URL: Issues, https://github.com/datarobot-oss/tensile/issues
Keywords: llm,agents,trajectory,logging,reliability,testing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: AUTHORS
Requires-Dist: click>=8.1.0
Requires-Dist: fastapi>=0.133.0
Requires-Dist: httpx>=0.24.0
Requires-Dist: python-dotenv>=0.19.0
Requires-Dist: petname>=2.6
Requires-Dist: pydantic>=2.0
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: tqdm>=4.67.1
Requires-Dist: uvicorn[standard]>=0.34.0
Dynamic: license-file

# tensile

> **ten·sile** *(adjective)* — describing a material's ability to be stretched or pulled without breaking; capable of withstanding tension.
>
> The tensile strength of AI agents should be enough to handle unexpected inputs, tool failures, and adversarial conditions — and recover gracefully when pushed to their limits.

## What is Tensile?

Tensile records an agent's LLM interactions and helps analyze and remediate mistakes. It installs a set of [Agent Skills](#skills) and a [CLI](#cli-commands),
and captures traffic one of two ways:

- **Proxy (recommended)** — run your agent behind a local capture proxy with
  `tensile run -- <your command>`. No code changes, works with agents in any
  language. Install Tensile as a standalone CLI tool.
- **Python library** — add Tensile as a project dependency and wrap
  your LLM client in the `TrajectoryLogging` context manager. For Python agents
  that can't run behind the proxy.

Either way, Tensile helps improve agent reliability by:
- **Recording** agent interactions as replayable [trajectories](#trajectories)
- **Analyzing** trajectories to identify mistakes and failures
- **Creating** [motifs](#motifs) — reusable pattern detectors that measure recurring mistakes
- **[Remediating](#remediations)** issues through system prompt updates, tool improvements, and contextual hints

⚠️ **WARNING - Tensile [replays](#replays) LLM responses to validate remediations. These replays can consume substantial numbers of tokens!**

## Quickstart (proxy)

The proxy path needs no code changes and works with agents in any language.

```bash
# 0. Install uv (https://docs.astral.sh/uv/getting-started/installation/)
curl -LsSf https://astral.sh/uv/install.sh | sh

# 1. Install tensile
uv tool install tensile

# 2. From your agent's project directory, initialize tensile (configures preferences & installs skills)
tensile init

# 3. Run your agent through Tensile to record trajectories
tensile run -- <your agent's run command>   # e.g. tensile run -- python my_agent.py
```

`tensile run` starts a local proxy, points your agent's LLM endpoint at it and records every LLM interaction. For agents that reach their LLM through a custom endpoint (e.g. `DATAROBOT_ENDPOINT`), the first run may record nothing and suggest possible environment variable to monitor. See [proxy-docs](docs/proxy.md#3-a-nonstandard-endpoint-encoded-in-an-env-var-proxymirror).

4. `tensile auth` sets up authentication to enable [replays](#replays). Run only after a trajectory has been successfully recorded.

5. Analysis of trajectories is done through your coding agent's skills. Open your coding agent in the same directory that you ran `tensile init` and invoke the [/tensile](src/tensile/skills/tensile/tensile/SKILL.md) skill to get started.

6. `tensile ui` starts the UI:

![Tensile UI](docs/images/tensile-ui.png)

7. `tensile uninstall` removes all tensile-related files in the current directory.

### Python library (legacy)

For Python agents that can't run behind the proxy, install Tensile as a project
dependency and instrument the LLM client in-process instead:

```bash
# 1. Install tensile
uv add tensile

# 2. Initialize tensile - this configures preferences and installs skills
tensile init
```

3. Fire up your coding agent (Claude/Cursor/VSCode/etc) and invoke the [/tensile-instrument-agent](src/tensile/skills/tensile/tensile-instrument-agent/SKILL.md) skill to add `TrajectoryLogging` to your codebase. Follow the instructions provided by your coding agent until it confirms that TrajectoryLogging is succesfully setup. From there, run your agent normally — trajectories are recorded automatically — and continue with `tensile ui` as above.

## Skills

- [/tensile](src/tensile/skills/tensile/tensile/SKILL.md) The default, primary tool for inspecting recorded agent trajectories. Use this first for any lightweight analysis, debugging agent behavior, identifying behavioral patterns, or performing latency/token analysis.
- [/tensile-remediate](src/tensile/skills/tensile/tensile-remediate/SKILL.md) Executes a rigorous 3-phase workflow: mistake identification, motif creation, and statistical replay remediation. The agent will guide you through each phase of this workflow.
 - [/tensile-autopilot](src/tensile/skills/tensile/tensile-autopilot/SKILL.md) Run the full tensile-remediate loop end-to-end. Collects agent trajectories, orchestrates a subagent through analyze → create motifs → remediate phases, reviews each phase's output, applies validated fixes to the codebase, and optionally iterates to verify improvements.
 - [/tensile-instrument-agent](src/tensile/skills/tensile/tensile-instrument-agent/SKILL.md) Instrument an agent for trajectory logging with Tensile.

## Key Concepts

### Trajectories

Similar to a trace, a **trajectory** is a recorded log of every LLM interaction that occurred during a single agent run. Each step in a trajectory contains the messages sent to the LLM and the LLM's response (text and tool calls). Trajectories are saved as `.jsonl.gz` files and serve as the raw data for all downstream analysis. You generate trajectories by running your agent under the capture proxy (`tensile run`) or by instrumenting it in-process with Tensile's `TrajectoryLogging` context manager.

Multi-agent and concurrent agents are supported. Tensile commits only to the structure it can determine with certainty — **threads** (continuation chains: a linear agent, a multi-turn subagent, or a main agent resumed after fan-out) — and reliable agent labels where an exact signal exists (an explicit handoff, or a name you supply). It deliberately does **not** guess at fuzzy links or group threads into runs; that judgment is surfaced to the analyzer along with the raw evidence (timing, connection identity). For exact attribution, name agent scopes in-process with `tensile.branch("subagent-name")`; those names carry straight through to the analysis.

### Sessions

A **session** groups one or more trajectories together for analysis. When you start a session, Tensile extracts key artifacts from each trajectory (system prompt, user prompt, tool definitions, and a human-readable summary) and scans them against any existing motifs. Sessions are identified by memorable petnames (e.g., "bright-falcon") and track which trajectories have been analyzed, making it easy to focus on new runs.

### Motifs

A **motif** is a durable, reusable pattern detector that identifies a specific class of agent behavior — typically a recurring mistake. Each motif is a directory containing:

- **`detect.py`** — a Python function that returns `True` when the behavioral pattern is present at a given step
- **`expect.py`** (optional) — a Python function that returns `True` when the *correct* behavior is present instead
- **`metadata.yaml`** — a description of the pattern and when it applies
- **`annotations.jsonl`** — an append-only log of (trajectory, step) pairs where the pattern was observed

Motifs let you measure how often a mistake occurs and whether a fix actually works. They accumulate annotations over time, building a dataset of known instances across many trajectories.

### Replays

A **replay** takes an annotated step from a trajectory, feeds the conversation history up to that point to the LLM, and samples a fresh response. By comparing detect/expect rates across many replays — with and without a remediation applied — you can measure whether a proposed fix actually changes the agent's behavior. Replays are the mechanism that makes Tensile's feedback loop quantitative rather than anecdotal.

### Remediations

A **remediation** is a change to the agent's system prompt, tool definitions, or contextual hints designed to fix a detected mistake. During the remediation phase, you apply changes and test them via replays to measure their effect before modifying your actual agent code. Effective remediations reduce the detect rate (bad behavior occurs less) and increase the expect rate (correct behavior occurs more).

### Contextual Hints

**Contextual hints** are targeted messages injected into an agent's conversation at inference time. Unlike system prompt changes (which are always present), hints fire only when a trigger condition is met — for example, when a tool returns an error or when the agent is about to repeat a known mistake. Hints are optional but can be highly effective for situational fixes.

## Workflow Overview

Tensile follows a structured loop: **capture → run → analyze → create motifs → remediate**.

```
┌──────────────┐    ┌───────────┐     ┌─────────────┐     ┌────────────────┐     ┌─────────────┐
│ 1. Capture   │───▶│ 2. Run    │────▶│ 3. Analyze  │────▶│ 4. Create      │────▶│ 5. Remediate│
│    Traffic   │    │   Agent   │     │ Trajectories│     │    Motifs      │     │   Mistakes  │
└──────────────┘    └───────────┘     └─────────────┘     └────────────────┘     └───────┬─────┘
                         ▲                                                               │
                         └───────────────── Re-run agent to verify ──────────────────────┘
```

1. **Capture** — Run your agent under the proxy (`tensile run -- <command>`, no code change), or add `TrajectoryLogging` to your agent's LLM client (a one-line code change)
2. **Run** — Execute your agent; trajectories are recorded automatically
3. **Analyze** — Start a session, review trajectory summaries, and identify mistakes and root causes
4. **Create Motifs** — Build detect/expect functions for recurring issues, scan trajectories, and annotate affected steps. Establish baseline rates via replays.
5. **Remediate** — Propose fixes (prompt edits, tool changes, contextual hints), test them via replays, and measure improvement against the baseline. Apply successful fixes back to your codebase.

After remediation, re-run your agent to generate new trajectories and verify the fixes work in practice. Repeat the loop to address additional issues.

## CLI Commands

For direct command-line usage without skills:

```bash
# Installation & Configuration
tensile init                     # Initialize Tensile (install skills & configure)
tensile check                    # Check if deployed skills are up to date
tensile uninstall                # Remove Tensile skills and configuration

# Capture
tensile run -- <command>         # Run a command behind the capture proxy (e.g. tensile run -- python my_agent.py)

# Credentials
tensile auth                     # Configure credentials for all trajectories
tensile auth <trajectory>        # Configure credentials for a single trajectory
tensile auth <session-id>        # Configure credentials for a session

# Trajectories
tensile ls                       # List trajectories with metadata
tensile inspect <trajectory>     # View trajectory contents (no LLM needed)
tensile extract <trajectory>     # Extract system prompt, tools, and summary
tensile replay <trajectory> --step N  # Replay a specific trajectory step

# Sessions
tensile session start <traj>...  # Start an analysis session
tensile session ls               # List all analysis sessions

# Motifs
tensile motif create <name>      # Create a new motif with skeleton files
tensile motif scan <motif> <traj>  # Scan trajectories for motif detections
tensile motif test <motif> -n 10 # Test a motif against trajectory/replay data
tensile motif annotate <motif> <traj> --step N  # Record a motif annotation
tensile motif show-step <traj> --step N  # Show the step dict detect/expect receive

# Contextual Hints
tensile hint create <name>       # Create a new hint with skeleton files
tensile hint ls                  # List all hints
tensile hint test <hint> <traj>  # Test a hint trigger against a trajectory

# UI
tensile ui                       # Launch web UI (see below)
```

See `tensile --help` for all commands and options.

## FAQ

Q: What are contextual hints and do I need them?

>Contextual hints are targeted messages that are surfaced to an agent at inference time. They are a powerful way to steer agent behavior. For example, a hint message could be "Always remember to ask the user for permission before deleting a file." We have found contextual hints to be highly effective at improving agent behavior, but they are a new feature, so caution is recommended.

Q: What frameworks does Tensile support?

>Tensile works with any agent that uses an OpenAI-compatible API (OpenAI, Anthropic, Google Gemini, etc.). The proxy path (`tensile run`) is language- and framework-agnostic — it captures any agent whose LLM SDK honors a base-URL environment variable. The Python library path supports direct SDKs, LangChain, Pydantic AI, and LiteLLM through HTTPX transport wrapping.

Q: Does Tensile support multi-agent systems?

>Yes — orchestrator/subagent architectures and concurrent fan-out are supported. Under the `tensile run` proxy this works out of the box: the recording is reconstructed into threads, with agent labels assigned wherever an exact signal (an explicit `transfer_to_*` handoff, or a stamp) establishes them. To label proxy-captured agents exactly, set the `X-Tensile-Agent` header on their LLM requests (e.g. `extra_headers={"X-Tensile-Agent": "researcher"}`) — the language-agnostic equivalent of `tensile.branch`. For in-process capture, give all LLM clients one shared `trajectory_name` so they record into a single trajectory, and wrap subagent call sites in `with tensile.branch("name"):` to label them exactly. Where no exact signal exists (e.g. programmatic fan-out), analysis leaves the threads unlabeled and hands the evidence to the analyzer rather than guessing. The `/tensile-instrument-agent` skill knows how to set up the in-process path. (Subagents running in separate processes are captured only under the proxy.)

Q: What is replay and why does Tensile need my credentials?

>Tensile will replay certain LLM responses in trajectories in order to establish the validity of remediations. Replay works by sending the recorded LLM request to your LLM endpoint and inspecting the new LLM response. Your LLM endpoint's credentials are needed to communicate with the LLM endpoint. `tensile auth` is designed to help configure these credentials.

## Documentation

- [docs/proxy.md](docs/proxy.md) - The capture proxy (`tensile run`): how it works, setup, and nonstandard endpoints
- [docs/credentials.md](docs/credentials.md) - Credential management, `tensile auth`, and source spec types
- [CONTRIBUTING.md](CONTRIBUTING.md) - Contributing guide for developers
- [AGENTS.md](AGENTS.md) - Repository guidelines for AI agents and contributors
- Skills documentation in `src/tensile/skills/*/SKILL.md`

## License

See [LICENSE](LICENSE) file for details.
