Metadata-Version: 2.5
Name: openodke
Version: 1.0.0rc1
Summary: An open implementation of the ODKE+ architecture (Apple, arXiv:2509.04696): ontology-guided knowledge extraction into a grounded graph. Independent; not affiliated with Apple.
Project-URL: Homepage, https://github.com/deepskandpal/odke
Project-URL: Source, https://github.com/deepskandpal/odke
Project-URL: Issues, https://github.com/deepskandpal/odke/issues
Project-URL: Changelog, https://github.com/deepskandpal/odke/blob/main/CHANGELOG.md
Author: Deepanshu Kandpal
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: ODKE+,entity-extraction,graphrag,information-extraction,knowledge-extraction,knowledge-graph,llm,neo4j,odke,odke-plus,ontology,rdf,relation-extraction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: pydantic<3,>=2.7
Requires-Dist: typer<1.0,>=0.15
Provides-Extra: all
Requires-Dist: litellm<2,>=1.55; extra == 'all'
Requires-Dist: neo4j<7,>=5.20; extra == 'all'
Requires-Dist: networkx<4,>=3.2; extra == 'all'
Requires-Dist: nltk<4,>=3.8; extra == 'all'
Requires-Dist: opentelemetry-api<2,>=1.24; extra == 'all'
Requires-Dist: pyarrow>=15; extra == 'all'
Requires-Dist: pypdf<7,>=5.0; extra == 'all'
Requires-Dist: python-docx<2,>=1.1; extra == 'all'
Requires-Dist: pyyaml<7,>=6; extra == 'all'
Requires-Dist: rdflib<8,>=7.0; extra == 'all'
Provides-Extra: bench
Requires-Dist: nltk<4,>=3.8; extra == 'bench'
Provides-Extra: docs
Requires-Dist: pypdf<7,>=5.0; extra == 'docs'
Requires-Dist: python-docx<2,>=1.1; extra == 'docs'
Provides-Extra: docx
Requires-Dist: python-docx<2,>=1.1; extra == 'docx'
Provides-Extra: llm
Requires-Dist: litellm<2,>=1.55; extra == 'llm'
Provides-Extra: neo4j
Requires-Dist: neo4j<7,>=5.20; extra == 'neo4j'
Provides-Extra: networkx
Requires-Dist: networkx<4,>=3.2; extra == 'networkx'
Provides-Extra: otel
Requires-Dist: opentelemetry-api<2,>=1.24; extra == 'otel'
Provides-Extra: parquet
Requires-Dist: pyarrow>=15; extra == 'parquet'
Provides-Extra: pdf
Requires-Dist: pypdf<7,>=5.0; extra == 'pdf'
Provides-Extra: rdf
Requires-Dist: rdflib<8,>=7.0; extra == 'rdf'
Provides-Extra: yaml
Requires-Dist: pyyaml<7,>=6; extra == 'yaml'
Description-Content-Type: text/markdown

# openodke

An open implementation of ODKE+ (Apple, [arXiv:2509.04696](https://arxiv.org/abs/2509.04696)) — independent, and not affiliated with Apple.

[PyPI](https://pypi.org/project/openodke/) · [Documentation](https://openodke.dev/) · [Changelog](CHANGELOG.md) — `pip install openodke`

<!-- TAGLINE: the maintainer's to write (DECISIONS #24, #100). It replaces the bold sentence below. -->
**The verification layer between any extractor and your graph store.** It doesn't
matter what produced your triples: LangChain's `LLMGraphTransformer`,
neo4j-graphrag, LangExtract, patterns over a table, or openodke's own extractor.
openodke checks each fact against the text it came from, links its entities,
counts the independent sources behind it and measures the result. Then it writes
the graph to Neo4j, RDF or JSON Lines with the evidence attached.

Every fact carries the document, the character span, the source tier and the
grounding verdict, so *why is this edge in my graph?* is a query, not an
investigation.

> **Status: 1.0.0rc1, a release candidate; 0.2.1 is the last full release on
> [PyPI](https://pypi.org/project/openodke/).** 1.0 is the verification layer
> ([DECISIONS #24](DECISIONS.md)): the Validator runs on any extractor's triples
> (`odke validate`, with adapters for LangChain, LangExtract, neo4j-graphrag and a
> graph already in Neo4j), inside openodke's own pipeline (`odke run`), and on a
> graph openodke did not build (`odke ground`); the Evaluator (`odke eval`,
> `odke bench`) measures it.
>
> On public documents the grounder removed 9–20% of the wrong triples each
> extractor wrote, for 16–37% of extraction's cost; on single sentences it cost
> precision ([Benchmarks](https://openodke.dev/benchmarks/)). No grounder has met
> its [calibration bar](https://openodke.dev/calibration/) yet, so by default
> what it would refuse is marked for review, not dropped. See
> [ROADMAP.md](ROADMAP.md).

## Quickstart

From a clone, with no key, no network and no database:

```bash
git clone https://github.com/deepskandpal/odke && cd odke
pip install -e ".[yaml]"
odke run examples/e2e/odke.yaml
```

That runs a small invented corpus through all thirteen stages on recorded model
responses and writes the graph to `examples/e2e/out/`. The same run from Python:

```python
from openodke import HybridExtractor, LLMExtractor, Ontology, Pipeline, VerdictGate
from openodke.ground import LLMGrounder
from openodke.llm import RecordedClient, ReplayClient
from openodke.loaders import DirectoryLoader
from openodke.sinks import JsonlSink

ontology = Ontology.from_json("examples/e2e/ontology.json")
docs = list(DirectoryLoader().load("examples/e2e/corpus"))
# Recorded responses, so this runs with no key. Drop `client=` to call the models.
extract = ReplayClient("examples/e2e/recorded/extract.json")
ground = RecordedClient.from_fixture("examples/e2e/recorded/ground.json")
kg = Pipeline(
    ontology,
    HybridExtractor(LLMExtractor(client=extract), documents=docs),
    grounder=LLMGrounder(client=ground),  # a second model checks every cited span
    gate=VerdictGate(),  # marks what its span contradicts, below the grounder's bar
    sinks=[JsonlSink("out")],
).run(docs)
print(len(kg.facts), "facts,", kg.stats.get("marked", 0), "marked for review")
```

[`examples/e2e/`](examples/e2e/README.md) goes on into Neo4j and ends with the
Cypher that shows provenance, a `DIFFERENT` link between two companies with one
name, and a cardinality check.

## Does it work?

On the example, `odke eval ablation` runs one config three ways against 36
labelled facts:

| Configuration | Precision | Recall | Model calls |
|---|---|---|---|
| extraction alone | 0.889 | 0.889 | 3 |
| + grounding | 0.889 | 0.889 | 39 |
| + corroboration | 0.944 | 0.944 | 39 |

**This is a demonstration on hand-authored recorded responses, not a benchmark.**
The responses and the labels were written for the example, so the table shows
what the harness reports, not how well any model does. On that fixture the
grounder caught one of four wrong candidates, and the gate marked it for review
rather than dropping it: the default grounder is
[below its calibration bar](https://openodke.dev/calibration/), so nothing is
dropped on its word (refusing it would have given 0.914). Normalisation did the
rest. The paper's 98.8% is neither reproduced nor claimed.
[What the example shows, and does not](examples/e2e/README.md#5-the-ablation-on-this-example).

On public data, [five extractors run with and without the layer](https://openodke.dev/benchmarks/#every-extractor-with-and-without-the-layer),
with ODKE+'s [paper-mode grounder](https://openodke.dev/grounding/#paper-mode-the-odke-grounder-as-written)
reading each whole document (Claude models, uncalibrated), not the Validator's
default span prompt. On Re-DocRED's documents the grounder
raised precision for all four model-based extractors, by 1.8 to 4.3 points, for
one Haiku call per triple ($4.23 to $9.01 per 1,000 documents), with F1 unchanged
within its interval. On Text2KGBench's single sentences it lowered precision for
every extractor, by 0.7 to 2.2 points. The pattern extractor found nothing in prose.

The harness is the point: run the same command on a slice of your own corpus
that you have labelled, and it tells you whether grounding earns its calls there.

## Why this exists

Apple published [ODKE+ (arXiv:2509.04696)](https://arxiv.org/abs/2509.04696), a
production system that extracts open-domain facts at scale and ingests them into a
knowledge graph at 98.8% precision. Apple released no implementation, and nothing
on PyPI, conda, GitLab or Hugging Face implements it. This is an independent
implementation of that architecture, generalised from one company's internal
graph into an SDK anyone can install.

The three ideas worth taking from that paper:

| Idea | What it does | Why the alternatives lose |
|---|---|---|
| **Ontology snippets** | Prompts the model with a small, ranked, per-type schema fragment rather than the whole ontology | A 200-predicate schema does not fit in a useful prompt. Snippets keep prompt size flat as the schema grows |
| **Grounding verification** | A second, cheap model checks each candidate fact against its own evidence span and records `supported`, `contradicted` or `not_found` on the fact; a gate decides what is written | Extraction alone hallucinates. A yes/no check against a quoted span is cheap, and a verdict kept on the fact can be measured and re-gated later |
| **Corroboration** | Merges the same claim across sources, and resolves conflicts on freshness × trust × agreement | Real corpora disagree with themselves. Without it you write both answers and cannot say which to believe |

Other libraries turn text into a graph — [iText2KG](https://github.com/AuvaLab/itext2kg),
[neo4j-graphrag](https://neo4j.com/docs/neo4j-graphrag-python/current/user_guide_kg_builder.html)'s
`SimpleKGPipeline`, LangChain's `LLMGraphTransformer`. They extract, and most can
be handed a schema. None of them has a separate grounding pass, cross-source
corroboration, or character-span provenance carried through to the store.

## Thirteen stages, each one a Protocol

The pipeline is a superset: thirteen stages, each a `typing.Protocol` in
`openodke.stages` with a pass-through default (DECISIONS #20). Take the subset your
corpus needs, supply your own for any stage by writing one method, and leave the
rest out. Nothing domain-specific is in the code; it arrives as an ontology, a
config, or a stage of yours.

| Stage | Built-in | Name in `odke run` | Left out |
|---|---|---|---|
| load | `DirectoryLoader`, `TextLoader`, `MarkdownLoader`, `HtmlLoader`, `PdfLoader`, `DocxLoader`, `CsvLoader`, `TsvLoader`, `JsonLoader`, `JsonlLoader`, `ParquetLoader` | `directory` (every suffix), `text`, `markdown`, `html`, `pdf`, `docx`, `csv`, … | `directory` |
| chunk | `SentenceChunker` — whole sentences, offsets intact | `sentence` | one chunk per document |
| route | yours: skip marketing, policy, narrative | — | extract everything |
| extract | `TriplesExtractor` (any extractor's triples), `PatternExtractor`, `LLMExtractor`, `HybridExtractor` | `triples`, `pattern`, `llm`, `hybrid` | required |
| ground | `SpanGrounder` (free), `LLMGrounder` | `span`, `llm` | verdict `unchecked` |
| normalise | `ValueNormalizer` — dates, numbers, units, name keys | `value` | unchanged |
| resolve | `NativeResolver` — blocking, identifiers, links | `native` | keys as given |
| corroborate | `SignatureCorroborator` | `signature` | every fact its own claim |
| score | `EvidenceScorer` — verdict × support × extractor confidence | `evidence` | **`evidence`** under `odke run`; pass-through in a hand-built `Pipeline` |
| gate | `VerdictGate` — refuses `contradicted` | `verdict` | accept everything |
| sink | `JsonlSink`, `Neo4jSink`, `CypherFileSink`, `Neo4jAdminCsvSink`, `RdfSink`, `NetworkXSink` | `jsonl`, `neo4j`, `cypher_file`, `neo4j_admin_csv`, `rdf`, `networkx` | nothing written |
| constrain | `Neo4jConstrainer` — the ontology as DDL | `neo4j` | no constraints |
| infer | `OntologyInferrer`; `odke ontology infer`, then review and `freeze` | — | never run: inference is a bootstrap (DECISIONS #8) |

## Install

From [PyPI](https://pypi.org/project/openodke/):

```bash
pip install openodke                   # the base install
pip install "openodke[yaml]"           # + YAML configs
pip install "openodke[neo4j,yaml]"     # + the Neo4j sink
pip install "openodke[all]"            # everything
```

For unreleased changes on `main`: `pip install "openodke @ git+https://github.com/deepskandpal/odke"`.

The base install is pydantic and typer, and talks to nothing: the data model,
the ontology compiler, the text, Markdown, HTML and record loaders, the pattern
extractor, the OpenAI-compatible model client, corroboration, evaluation, and
the JSONL and Cypher-file sinks. Each extra is imported on first use, and a
missing one is named in the error.

| Extra | Adds | For |
|---|---|---|
| `llm` | litellm | Model providers that do not speak the OpenAI shape |
| `neo4j` | the Neo4j driver | `Neo4jSink`, `Ontology.from_neo4j` |
| `yaml` | PyYAML | YAML ontologies and run configs |
| `parquet` | pyarrow | `ParquetLoader` |
| `pdf`, `docx` | pypdf; python-docx | `PdfLoader`, `DocxLoader` |
| `rdf` | rdflib | `RdfSink`, `Ontology.from_owl` |
| `networkx` | networkx | `NetworkXSink` |
| `docs` | pypdf, python-docx | The document readers together |
| `bench` | NLTK | `odke bench` on Text2KGBench (its hallucination metrics) |
| `otel` | the OpenTelemetry API | Spans per job, stage and model call |
| `all` | all of the above | |

Python 3.11–3.14.

## `odke run`

One file names the inputs, the ontology, the models, which implementation fills
each stage, and the sink. Here the extractor is any other system's triples, read
from a file, checked against the texts they cite and written to Neo4j:

```yaml
ontology: ontology.json
inputs:
  - path: texts                     # the texts the triples' `doc` names
    loader: {use: directory}
models:
  ground: anthropic/claude-haiku-4-5-20251001
stages:
  extractor: {use: triples, path: triples.jsonl, extractor: my-extractor}
  grounder: llm
  normalizer: value
  corroborator: signature
  gate: verdict
  sink: {use: neo4j, uri_env: NEO4J_URI, password_env: NEO4J_PASSWORD}
  constrainer: neo4j
bootstrap: true
```

`extractor: llm` (or `hybrid`, patterns first) extracts with openodke's own
reference extractor instead, on the `extract` model.

```bash
odke run config.yaml --dry-run     # load, extract and ground; print what would be written
odke run config.yaml
```

A stage of your own is `package.module:Name` with its options alongside. A dry
run opens no sink, so it needs no database. Every stage's own counts — verdicts,
rejections, conflicts, refusals, cost — go into the graph's `stats`, and a
document's id is its path, so a label or a query can name it.
[`examples/run.yaml`](examples/run.yaml) comments every key.

## Evaluate against your own labels

openodke ships the formats and the arithmetic for scoring every stage against
your own labelled dataset; `odke bench` runs the public ones
([Benchmarks](https://openodke.dev/benchmarks/)).

```bash
odke eval extract --describe                           # what to label
odke eval extract --labels gold.jsonl --predictions out/facts.jsonl
odke eval ablation --config config.yaml --labels gold.jsonl
odke eval spans --facts out/                           # no labels at all
```

There is an evaluator for routing, extraction, grounding, resolution, scoring
(Brier, reliability, ECE) and validation, and `openodke.eval.CostMeter` measures
tokens and USD per stage without touching a stage. One evaluator needs no labels:
`odke eval spans` reports cited span width split by grounding verdict, which
scores the too-narrow-citation error with no gold set. The fixtures in the test
suite exercise the arithmetic and are not a benchmark.

## When the platform does a stage itself

Some stores already resolve, prune or constrain after the write. A sink declares
that with a `PlatformProfile`; a stage configured on both sides raises one
`DoubleStageWarning` and still runs, because the two passes are not the same
pass (DECISIONS #21). To hand a stage to the platform, pass
`Delegated(to="neo4j-graphrag:FuzzyMatchResolver")` — or `{use: delegated, to: ...}`
in a config — and openodke stamps who did it wherever the data model has room, so
the platform's work can be read back and scored with the same evaluator.

## Models and providers

Nothing in this library imports a provider SDK or names a vendor. Every model
call goes through one `LLMClient` protocol, and which client serves a model
string is decided by a small, inspectable routing rule you can override.

```python
from openodke.llm import ModelRoles, ModelSpec

ModelRoles()  # Claude by default: Sonnet extracts, Haiku grounds
ModelRoles.single("ollama/llama3.1")  # entirely local — no extras, no key, no network
ModelRoles(
    extract=ModelSpec(model="anthropic/claude-sonnet-5"),
    ground=ModelSpec(model="ollama/qwen2.5:3b"),
)
```

**Two models, not one, by default.** Extraction wants a capable model; grounding
asks thousands of yes/no questions and wants a small one. Anything speaking the
OpenAI chat shape — Ollama, vLLM, LM Studio, llama.cpp, OpenRouter, Groq, a
gateway — works on the base install; everything else goes through litellm with
`[llm]`; `openodke.llm.register("mycorp", factory)` routes a provider through your
own client. `ReplayClient`, `RecordedClient` and `ScriptedClient` ship in the
package, so your own stages are testable offline too.

```bash
odke models                                  # every provider, its key variable, whether it is set
odke run odke.yaml --model openai/gpt-5.5    # override the config's models for one run
```

`odke models` runs on the base install with no key and no network, and never
prints a value: keys are read from the environment and stored nowhere. See
[Models and providers](https://openodke.dev/models/).

## Ontologies

```python
from openodke import Ontology

ontology = Ontology.from_json("examples/e2e/ontology.json")  # or from_owl, from_neo4j
print(ontology.validate())  # diagnostics with dotted paths, never an exception
print(ontology.snippet("Company").render())  # exactly what the model is prompted with
```

```bash
odke ontology validate schema.json
odke ontology diff old.json new.json --fail-on-breaking
odke ontology snippet schema.json Company
```

A bad extraction is usually a bad snippet, and `snippet` shows it before a token
is spent. The ontology also marks which qualifiers bear identity (`percentile`)
and which reconcile (`start_time`), and that decides what counts as one claim.

## Honest limits

- **The fixtures and the example are not benchmarks.** Their model responses are
  hand-authored. The numbers measured with real models are on
  [Benchmarks](https://openodke.dev/benchmarks/).
- **Live Neo4j needs 5.7 or later.** The constraint bootstrap uses relationship
  uniqueness constraints; Community edition is enough.
- **Structured facts are grounded against their own cell.** The cell `Leeds`
  cannot say whose head office it is, so a careful grounder answers `not_found`
  and the call is spent for little. The default gate keeps `not_found` for this
  reason.
- **The default grounder reads the cited span.** That is what keeps it cheap,
  and a span that leaves out the subject cannot support the claim. Widen and
  retry (`widen`, off by default) asks again against the sentence, and
  `context: document` reads the whole document, as paper mode does.
- **Resolution never merges a stored node on names.** Only a shared
  identifier re-keys onto one; a name match is a `SIMILAR` link for someone to
  act on. Asked to (`normalize_batch=True`, off by default), it makes a
  batch's own look-alikes that nothing in their names or context keeps apart
  one entity before anything is written (DECISIONS #43).
- **Inference drafts; it never decides.** `odke ontology infer` proposes a small
  schema for a person to review and freeze, and nothing infers implicitly
  (DECISIONS #8).

## Documentation

- [openodke.dev](https://openodke.dev/) — the documentation site: concepts, the Validator and its inputs, loading documents, grounding, resolution and corroboration, stores, `odke run`, evaluation, labels and benchmarks, models, the ontology and the reference extractor
- [examples/](examples/README.md) — the end-to-end example and the commented run config
- [ROADMAP.md](ROADMAP.md) — milestones, what each one delivers, and the estimate
- [CHANGELOG.md](CHANGELOG.md) — what landed, milestone by milestone
- [DECISIONS.md](DECISIONS.md) — the design calls and why they went that way
- [CONTRIBUTING.md](CONTRIBUTING.md) — `./scripts/verify.sh` is the whole check
- [NOTICE](NOTICE) — the relationship to the ODKE+ paper

## Relationship to the paper

This is an independent implementation of a published architecture. It is not
affiliated with or endorsed by Apple Inc., uses no Apple code, data or models,
and was written from the paper alone. All credit for the architecture belongs to
Khorshidi et al. See [NOTICE](NOTICE).

It departs from the paper where the paper is specific to its deployment. ODKE+
decides what to refresh by watching Wikipedia edits and retrieves its own
evidence; here those two stages are optional protocols, `Initiator` and
`Retriever`, because an SDK is usually handed its documents.

## License

Apache-2.0.
