Metadata-Version: 2.4
Name: causal-agent-verifier
Version: 0.1.0
Summary: A first-principles evaluation framework for LLM agents based on causal invariants, state transitions, and counterfactuals.
Author: Deepak Sen
Requires-Python: >=3.10,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Provides-Extra: all
Provides-Extra: anthropic
Provides-Extra: gemini
Provides-Extra: openai
Requires-Dist: anthropic (>=1.3.0,<2.0.0) ; extra == "anthropic" or extra == "all"
Requires-Dist: google-genai (>=2.21.0,<3.0.0) ; extra == "gemini" or extra == "all"
Requires-Dist: openai (>=3.6.0,<4.0.0) ; extra == "openai" or extra == "all"
Requires-Dist: pydantic (>=2.13.5,<3.0.0)
Requires-Dist: simpleeval (>=1.0.7,<2.0.0)
Description-Content-Type: text/markdown

# Causal Agent Verifier (CAV)

**Causal Agent Verifier (CAV)** is a first-principles evaluation framework for LLM agents. 

Unlike other evaluation frameworks that rely on brittle "expected trajectories" or subjective "LLM-as-a-judge" heuristics to grade an agent's reasoning, CAV evaluates agents by auditing their telemetry against **Causal Invariants** (deterministic rules) and conducting **Counterfactual Analysis** on critical state mutations.

## Why CAV?

LLM agents are difficult to evaluate because they operate autonomously over long horizons, and their trajectories are non-deterministic. A single mistake compounds over time.

CAV solves this by evaluating agents at 3 distinct causal layers:
1. **Epistemics (Belief vs Reality):** Did the agent hallucinate, or did it perceive the environment correctly?
2. **Invariants (Deterministic Rules):** Did the agent's actions violate any core system rules (e.g., "Never delete a production database")?
3. **Counterfactuals (Causal Rationality):** Given what the agent believed, was its action the safest and most rational choice?

## Features

- **OpenTelemetry / OpenInference Native:** Seamlessly ingests standard OTel traces. Works out-of-the-box if your agent (LangGraph, CrewAI, LlamaIndex, Pydantic AI) uses OpenInference instrumentation.
- **Sandboxed Execution:** Deterministic invariants are executed securely using `simpleeval`, blocking malicious or arbitrary code execution.
- **Multi-Provider LLM Support:** Use **Gemini**, **OpenAI**, or **Anthropic** for Layer 3 Counterfactual analysis and Invariant Generation.

## Making Your Agent Compatible

CAV relies on **OpenTelemetry / OpenInference** for standard telemetry. You **do not** write the `agent_trace.json` manually—it is automatically generated by your agent framework when it runs (e.g., using `openinference-instrumentation-langchain` or `openinference-instrumentation-openai`).

### How to Generate `agent_trace.json` Locally (Without Modifying Prod Code)

Hardcoding an `InMemorySpanExporter` inside your application code is an anti-pattern, because you would have to rewrite your telemetry configuration when you move to production (where you'd use an `OTLPSpanExporter` to send traces to a backend collector).

Instead, your application code should remain completely agnostic to where traces are sent. Just instrument the framework:

```python
# app.py (Your production code)
from openinference.instrumentation.openai import OpenAIInstrumentor
OpenAIInstrumentor().instrument()

def run_agent(prompt: str):
    # Agent logic here...
    pass
```

To capture the traces locally or in your CI/CD pipelines for CAV evaluation, use CAV's `capture_traces` context manager in your test suite. This dynamically intercepts the OTel traces during the test execution and dumps them to a JSON file, leaving your production code completely untouched.

```python
# test_agent.py (Your evaluation suite)
from app import run_agent
from cav.testing import capture_traces

def test_agent_deployment():
    # 1. Dynamically intercept traces just for this run
    with capture_traces("agent_trace.json"):
        run_agent("Deploy the web app")
        
    # 2. Now pass agent_trace.json into CAV...
```

However, to use CAV's **Epistemics (Belief vs Reality) Layer**, you must make two small additions to your evaluation pipeline:

### 1. Emit Agent Belief

Ensure your agent code emits a custom span attribute called `agent.belief`. This allows CAV to evaluate what the agent *thought* was happening. 

**Schema:**
```json
{
  "observed_facts": {"env": "test", "user_auth": "admin"},
  "reasoning": "Since I am an admin in the test env, I will proceed."
}
```

**Prompting Tip:** To get your agent to emit this, simply add this instruction to its system prompt:
> *"Before taking any action, you must output a 'thought' JSON block containing 'observed_facts' (key-value pairs of your current environment state) and 'reasoning' (your rationale)."* 

Extract that JSON in your agent loop and attach it to the OpenInference trace as the `agent.belief` attribute.

### 2. Provide a Ground Truth File (Optional)

To use the Epistemics (Belief vs Reality) layer, you need a `ground_truths.json` file mapping each agent `step_id` to the actual state of the environment.

**You do not have to write this structure manually.** CAV provides a utility to generate a skeleton based on your trace:

```python
from cav.utils import generate_ground_truth_skeleton

# This reads your trace and creates a skeleton ground truth file
generate_ground_truth_skeleton(
    trace_filepath="agent_trace.json", 
    output_filepath="ground_truths.json"
)
```

The generated file will look like this, ready for you to fill in the true values:
```json
{
  "span_001": {
    "_comment": "Enter ground truth environment variables for step at 2026-09-02T10:00:00Z",
    "env": "prod",
    "user_auth": "guest"
  }
}
```
If this file is not provided, the Epistemics check is gracefully skipped.

## Installation

Install CAV via pip:

```bash
pip install causal-agent-verifier
```

To install with specific LLM provider support:

```bash
pip install causal-agent-verifier[gemini]
# or
pip install causal-agent-verifier[openai]
# or
pip install causal-agent-verifier[all]
```

## Quick Start

```python
from cav.engine import CavEngine
from cav.invariants.checker import InvariantChecker
from cav.epistemics import EpistemicEvaluator
from cav.counterfactuals import CounterfactualEvaluator
from cav.telemetry import TelemetryParser

# 1. Initialize Evaluators
checker = InvariantChecker("invariants.json")
epistemics = EpistemicEvaluator(strict_mode=True)
counterfactuals = CounterfactualEvaluator()

# 2. Setup the Engine
engine = CavEngine(
    invariant_checker=checker,
    epistemic_evaluator=epistemics,
    counterfactual_evaluator=counterfactuals,
    goal="Deploy the web application"
)

# 3. Parse your OpenInference Agent Trace and Ground Truth
# The agent trace is generated by your agent framework.
# The ground truth file is generated by your evaluation test suite.
trajectory, ground_truths = TelemetryParser.parse_file(
    trace_filepath="agent_trace.json",
    ground_truth_filepath="ground_truths.json"
)

# 4. Evaluate the Trajectory
report = engine.evaluate_trajectory(trajectory, ground_truths)

# 5. Output Results
from cav.reporters import VerifierReporter
print(VerifierReporter.generate_markdown_report(report))
```

## Evaluating at Scale (Test Suites & CI/CD)

If you have many edge cases or variants to test, you integrate CAV directly into your development cycle (e.g., using `pytest`):

1. **Run your Agent against Test Cases:** Execute your agent against a suite of 50 different prompts/scenarios. This will generate 50 separate `agent_trace.json` files (or one large trace with 50 parent spans).
2. **Generate/Maintain Ground Truths:** Use the `generate_ground_truth_skeleton` utility to scaffold the true states for those 50 runs. You only need to do this once when creating the test suite. As long as your test environments are deterministic, you can reuse these ground truth files for all future runs.
3. **Automate CAV:** Write a simple test script that loops over your 50 traces, passes them to `CavEngine`, and asserts that `report["goal_reached"] == True`. If an agent regression causes an invariant to fail, your CI/CD pipeline will catch it immediately.

## Advanced Customization

### Generating Invariants
CAV provides scaffolding to pre-generate deterministic invariants using an LLM.

```python
from cav.invariants.generator import InvariantGenerator
from cav.llm_providers import GeminiProvider

provider = GeminiProvider(api_key="YOUR_API_KEY")
generator = InvariantGenerator(llm_client=provider)

template = generator.generate_template(
    goal="Clean up test databases",
    context="Only drop tables in the test environment."
)
```

## License
MIT License

