Metadata-Version: 2.4
Name: aether-observer
Version: 0.1.4
Summary: Importable SDK for evaluating knowledge-graph builders for intent and behavior drift.
Author: Aether
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.11
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Provides-Extra: graphiti
Requires-Dist: graphiti-core[kuzu]<0.30.0,>=0.29.0; extra == 'graphiti'
Description-Content-Type: text/markdown

# Aether KG Eval

Importable Python SDK for evaluating knowledge-graph builders against benchmark KG datasets.

The core customer flow is:

```text
kg dataset -> kg builder -> built KG / build trace -> kg eval result
```

## Install

```bash
python -m pip install aether-observer
```

For local development from this repository:

```bash
python -m pip install -e ".[dev]"
```

## Python API

```python
from aether_kg_eval import dataset_from_triples, evaluate_kg_builder

dataset = dataset_from_triples(
    [
        ("entity:1", "relation:works_at", "entity:2"),
        ("entity:3", "relation:reports_to", "entity:1"),
    ],
    name="customer-kg-smoke",
)

def builder(dataset):
    return dataset.triples

result = evaluate_kg_builder(dataset, builder)

print(result.overall_score)
print(result.intent_drift.status)
print(result.behavior_drift.status)
print(result.kg_quality.to_dict())
```

The v1 result reports:

- `intent_drift`: whether the builder preserved the intended KG construction task.
- `behavior_drift`: whether builder behavior or KG shape moved away from baseline.
- `orchestration_drift`: reserved as `not_applicable` for single-builder pipelines.
- `kg_quality`: triple, entity, relation, duplicate, invalid, and contradiction metrics.

## Builder Contract

`evaluate_kg_builder(dataset, builder)` accepts builders that return one of:

- `BuiltKG`
- `dict` with `triples` and optional `trace`
- `list[KGTriple]`
- `list[tuple[subject, relation, object]]`

Trace entries can be supplied as existing `EpisodeRecord` objects or dicts matching the legacy
JSONL episode shape. When trace is absent, KG quality is still scored and behavior drift falls back
to KG quality.

## Legacy Episode Format

Each JSONL record is an episode:

```json
{"id":"e1","timestamp":"2026-05-15T00:00:00Z","content":"Alice joined Acme.","content_type":"text","source":"seed","agent_id":"agent-a","task_id":"build-kg","entity_types":["Person","Org"],"relation_types":["JOINS"],"tool_calls":["extract"],"expected_intent":"add employment fact","outcome":"success"}
```

Important optional fields:

- `entity_types`, `relation_types`, `graph_nodes`, `graph_edges`, `embedding`, `contradictions`, `invalidations`
- `tool_calls`, `handoff`, `coordination_failure`, `boundary_violation`, `human_intervention`
- `expected_intent`, `observed_intent`, `reasoning_trace`, `outcome`

The legacy `kg_drift_builder` modules remain available internally for rolling-window scoring and
Graphiti/Kuzu experiments, but the sellable SDK surface is `aether_kg_eval`.

## Optional Graphiti/Kuzu

The old Graphiti/Kuzu ingestion adapter is still present as an optional integration layer. It is not
required to evaluate normal Python KG outputs.
