Metadata-Version: 2.5
Name: cauval
Version: 0.1.1
Summary: Sovereign Agent with Deterministic Output via Causal Grounding
Project-URL: Homepage, https://github.com/brighk/causal_autonomy
Project-URL: Repository, https://github.com/brighk/causal_autonomy
Project-URL: Issues, https://github.com/brighk/causal_autonomy/issues
Author-email: Koycho Georgiev <georgievkoycho@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: causal-reasoning,grounding,knowledge-graph,llm,rag,sparql,verification
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: fuzzywuzzy>=0.18.0
Requires-Dist: httpx>=0.25.0
Requires-Dist: loguru>=0.7.0
Requires-Dist: networkx>=3.2.1
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: pydantic>=2.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: python-levenshtein>=0.25.0
Requires-Dist: spacy>=3.7.0
Requires-Dist: sparqlwrapper>=2.0.0
Provides-Extra: api
Requires-Dist: fastapi>=0.115.0; extra == 'api'
Requires-Dist: prometheus-client>=0.19.0; extra == 'api'
Requires-Dist: prometheus-fastapi-instrumentator>=7.0.0; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.30.0; extra == 'api'
Provides-Extra: experiments
Requires-Dist: matplotlib>=3.8.0; extra == 'experiments'
Requires-Dist: numpy<3.0.0,>=2.0.0; extra == 'experiments'
Requires-Dist: pandas>=2.0.0; extra == 'experiments'
Provides-Extra: llm
Requires-Dist: accelerate>=0.25.0; extra == 'llm'
Requires-Dist: bitsandbytes>=0.43.0; extra == 'llm'
Requires-Dist: torch>=2.2.0; extra == 'llm'
Requires-Dist: transformers>=4.56.0; extra == 'llm'
Description-Content-Type: text/markdown

# CAUVAL — Causal AUtonomous Verification And vaLidation

[![CI](https://github.com/brighk/causal_autonomy/actions/workflows/ci.yml/badge.svg)](https://github.com/brighk/causal_autonomy/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/cauval.svg)](https://pypi.org/project/cauval/)

A causal fact-checker for LLMs. CAUVAL prevents language models from hallucinating causal claims by grounding every causal assertion against a formal knowledge base — and answers interventional and counterfactual questions using do-calculus on that knowledge base instead of the model's statistical priors.

---

## The problem

Large language models are trained on observational text. They learn statistical associations — what words follow what words — not causal structure. When you ask an LLM "Does X cause Y?" it pattern-matches against its training distribution. When you ask "What would happen if we forced X?", or "Would Y have occurred if X had been prevented?", it does the same thing, dressed in causal-sounding language. It has no mechanism to distinguish those three questions, and it regularly gets them wrong.

This is not a prompting problem. It is a structural one. Judea Pearl's causal hierarchy formalises it precisely:

| Level | Question type | Example | What it requires |
|-------|--------------|---------|-----------------|
| **1 — Observational** | Does X correlate with Y? | Do CPU spikes co-occur with timeouts? | Pattern matching over data |
| **2 — Interventional** | What if we do(X)? | If we enforce auth at node A, what is the breach rate? | A causal model + do-calculus |
| **3 — Counterfactual** | Would Y have occurred if X had been different? | Would this breach have happened without the rogue base station? | A structural causal model + abduction |

An LLM operates natively at Level 1. It can approximate Levels 2 and 3 in easy cases, but it has no guarantee of correctness and no mechanism for abstention when it doesn't know.

CAUVAL addresses this by keeping the LLM where it belongs — generating and reformulating text — and handing Levels 2 and 3 to formal causal machinery.

---

## How it works

CAUVAL routes every query to the appropriate reasoning mechanism based on its type:

```
Query
  │
  ├─ Level 2/3 pattern detected? ──Yes──► Build causal graph from KB (SPARQL BFS)
  │                                        Apply do-calculus (Pearl's graph surgery)
  │                                        Return formal answer — LLM not called
  │
  └─ No (factual/observational) ──────► LLM drafts a response
                                         Parse response → RDF triplets
                                         SPARQL-verify each triplet against KB
                                         ┌─ All verified? ──► Return response
                                         └─ Contradictions? ──► Inject constraints
                                                                 LLM retries
                                                                 (up to N times)
```

**Level 1 (factual):** the LLM generates a candidate answer. The semantic parser extracts causal assertions as RDF triplets. The truth anchor queries the knowledge base via SPARQL. If a triplet contradicts the KB, the contradiction is fed back as a constraint and the LLM regenerates. This loop continues until the response is fully grounded or the iteration limit is reached.

**Level 2/3 (interventional and counterfactual):** the LLM is bypassed entirely. CAUVAL links the query's entities to KB nodes, walks causal-predicate edges outward via SPARQL traversal to build a local causal graph, applies Pearl's do-calculus (graph surgery: remove incoming edges to the intervened node, propagate through descendants), and returns the formal result. The answer comes from the KB structure, not from the model's priors.

The response always carries a `causal_level` field (1, 2, or 3) so callers know which mechanism answered it.

---

## Quick start

```bash
pip install cauval
python -m spacy download en_core_web_sm   # one-time — see Setup
```

Two background services are required: Fuseki (the knowledge base) and the inference engine (the LLM server). See [Setup](#setup) for details.

```bash
# 1. Start Fuseki
FUSEKI_ADMIN_PASSWORD=secret docker compose -f deployment/docker-compose.yml up -d

# 2. Start the inference engine (separate terminal)
uv run python -m modules.inference_engine.server
```

```python
from cauval import Cauval

caf = Cauval()

# Level 1 — factual query: LLM answers, SPARQL verifies
result = caf.ask("Does high CPU usage cause increased response time?")
print(result.text)                               # verified answer
print(result.verification_status.is_valid)       # True
print(result.causal_level)                       # 1

# Level 2 — interventional query: do-calculus answers, LLM not called
result = caf.ask("Would response time increase if we prevent CPU spikes?")
print(result.text)                               # "No. Under do(cpu_spikes=False)..."
print(result.verification_status.verification_method)  # "do_calculus"
print(result.causal_level)                       # 2
```

Async variant for embedding in an existing async app:

```python
result = await caf.aask("Does rain cause road slipperiness?")
```

---

## What it is for

CAUVAL is useful in any domain where you need causal reasoning you can trust and trace — where "the model said so" is not an acceptable justification.

**6G network security.** A security analyst queries an incident knowledge graph:
- *Level 1:* "Do signal anomalies co-occur with auth failures at node 7?" — SPARQL fact check.
- *Level 2:* "If we enforce beamforming authentication at node 7, what is the downstream breach probability?" — do-calculus on the causal security model.
- *Level 3:* "Would this breach have occurred had quantum-safe encryption been deployed at the time?" — counterfactual forensics against the stored structural model.

**Medical and clinical reasoning.** Feed clinical literature through the companion [causal-discovery](../causal-discovery) extractor to build a causal knowledge graph, then query it. Every accepted answer traces back to the source edge that supports it.

**Policy and economic analysis.** Same architecture. The KB holds causal relationships extracted from policy documents; CAUVAL answers intervention questions ("what is the effect of policy X on outcome Y?") without conflating correlation with causation.

**Verified LLM pipelines.** Any application that uses an LLM to reason about a domain where you have structured causal knowledge. CAUVAL sits between the LLM and the user, ensuring the model's causal claims are grounded before they are accepted.

---

## Architecture

```
cauval/         Public library: Cauval class, CAFPipeline
api/            FastAPI gateway (POST /v1/infer) — same pipeline over HTTP
modules/
  inference_engine/   LLM client (HTTP) and server (vLLM / HuggingFace 4-bit)
  semantic_parser/    LLM text → RDF triplets via spaCy + Fuseki entity linking
  truth_anchor/       SPARQL verification against Apache Jena Fuseki
                      causal_graph_builder.py — BFS KB traversal for Level 2/3
  causal_validator/   Cycle and consistency checks on the verified causal graph
experiments/
  caf_algorithm.py              CAFLoop — the original script-level driver
  intervention_calculus.py      CausalGraph, do-calculus, parse_counterfactual_query
  kb_fvl_with_intervention.py   KnowledgeBaseFVLWithIntervention (experiments path)
  knowledge_base_fvl.py         KnowledgeBaseFVL — base SPARQL verifier
  run_counterbench_experiment.py CounterBench evaluation harness
common/
  llm_integration.py  Backend-agnostic LLM wrapper (Ollama, HuggingFace, 4-bit)
```

**Three entry points:**

1. **`cauval` library** (`from cauval import Cauval`) — `pip install cauval`, talks to Fuseki and to a running inference-engine server over HTTP. The primary interface.

2. **FastAPI service** (`api/main.py`) — the same pipeline exposed as `POST /v1/infer`. Run with `uv run python -m api.main`.

3. **`CAFLoop` directly** (`experiments/caf_algorithm.py`) — for scripting and benchmarks, no FastAPI layer. Use `KnowledgeBaseFVLWithIntervention` as the verifier to get Level 2/3 routing here too.

---

## Setup

`caval` itself is lightweight — spaCy, SPARQLWrapper, httpx, pydantic — no GPU stack. The inference engine, FastAPI gateway, and experiments harness are optional extras:

```bash
uv sync                          # caval only
uv sync --extra api              # + FastAPI gateway
uv sync --extra llm              # + local model loading
uv sync --extra experiments      # + benchmark harness
uv sync --all-extras             # everything
```

`en_core_web_sm` is not on PyPI. For local dev, `uv sync` installs it automatically. If you installed `caval` from PyPI:

```bash
python -m spacy download en_core_web_sm
```

`SemanticParser` raises a clear `RuntimeError` with this exact command if the model is missing.

### Running Fuseki

```bash
FUSEKI_ADMIN_PASSWORD=<password> docker compose -f deployment/docker-compose.yml up -d
```

Verify the dataset is configured (an unconfigured Fuseki silently serves an empty default dataset):

```bash
curl http://localhost:3030/$/ping
curl -G http://localhost:3030/dataset/query --data-urlencode "query=ASK { ?s ?p ?o }"
```

The second command should return a JSON `ASK` result. If it 404s, the dataset config did not load — see the note in `deployment/docker-compose.yml` about the `secoresearch/fuseki` image's mount paths.

### Loading knowledge

A single fact:

```bash
curl -X POST http://localhost:3030/dataset/update \
  -H "Content-Type: application/sparql-update" \
  --data 'INSERT DATA {
    <http://local.caf/rain> <http://causality.org/causes> <http://local.caf/slippery_road>
  }'
```

Bulk load from an N-Triples file:

```bash
curl -X POST http://localhost:3030/dataset/data \
  -H "Content-Type: application/n-triples" \
  --data-binary @my_facts.nt
```

To build that file from unstructured text, use the companion [causal-discovery](../causal-discovery) repo's `populate_kb_from_text.py`.

Clearing the dataset:

```bash
curl -X POST http://localhost:3030/dataset/update \
  -H "Content-Type: application/sparql-update" \
  --data 'DELETE WHERE { ?s ?p ?o }'
```

### Development checks

```bash
uv sync --locked
uv run --no-sync prek install
uv run --no-sync prek run --all-files   # lint + format the whole repo
```

---

## Running the CounterBench benchmark

```bash
uv run python -m experiments.run_counterbench_experiment \
  --input <dataset>.json \
  --use-llm --llm-model <name> \
  --use-real-sparql \
  --sparql-endpoint http://localhost:3030/dataset/query \
  --output results/caf_run
```

Requires `uv sync --extra experiments --extra llm`.

---

## Gotchas

**Nothing verifies — everything lands in "unverifiable":** the failure is in entity linking, not the SPARQL check. `EntityLinker` links mention text to KB URIs via `rdfs:label` / `skos:prefLabel`. A KB of bare `<uri> causes <uri>` triples with no labels means every entity fails to link, `subject_linked=False` on every triplet, and `TruthAnchor` skips the SPARQL query entirely rather than querying a fabricated URI. Check the dataset is configured (see the `ASK` query above) and that your KB entities have label triples — without them, nothing can be verified.

**Level 2/3 routing silently falls back to Level 1:** `CausalGraphBuilder` returns an empty graph if it can't link the query's entities to KB nodes, or finds no causal-predicate edges within `max_hops` (default 2) hops. The response will have `causal_level=1` and the LLM will be called. Increase `max_hops` if your causal chains are longer, at the cost of more SPARQL round-trips.

**The semantic parser is fragile on complex phrasing:** `SemanticParser` does dependency-parse SVO extraction. It handles simple declarative sentences ("X causes Y") reliably. Complex phrasing (passive voice, relative clauses, subordinate clauses) can produce wrong triplets — the LLM will see an unexpected constraint. Ask for short declarative answers in your prompts.

**Entity linking trades false negatives for false positives:** the fuzzy matcher uses `max(ratio, partial_ratio)`, so a short phrase ("habitat destruction") can match a longer KB label that contains it as a substring. Prefer specific multi-word phrases over single generic words, especially on KBs built from non-atomic labels.

**`SemanticParser` has no fallback:** if spaCy or `en_core_web_sm` is missing, `Caval()` raises `RuntimeError` immediately rather than degrading silently — a degraded extractor would quietly undermine verification.

---

## License

MIT — see `LICENSE`.
