Metadata-Version: 2.5
Name: vruksha
Version: 0.1.2
Summary: a typed entity graph over a litesearch store: LLM extraction, priming, a guarded merge, and a search leg
Project-URL: Repository, https://github.com/vedicreader/vruksha
Project-URL: Documentation, https://vedicreader.github.io/vruksha/
Author-email: Karthik <karthik.rajgopal@hotmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: entity resolution,knowledge graph,llm,nbdev,search,sqlite
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.10
Requires-Dist: fastcore>=2.2.15
Requires-Dist: litesearch>=0.1.61
Requires-Dist: numpy
Description-Content-Type: text/markdown

# vruksha


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

vruksha (वृक्ष, “tree”) adds a knowledge graph to a [litesearch](https://github.com/vedicreader/litesearch) store, so that search can follow references between passages. A store holds documents cut into chunks. vruksha sends each chunk to a language model, which names the entities in it, such as people, methods, works or articles of a law, and the relations between them, such as “Article 5 refers to Annex III”. These go into three tables that litesearch provides: entities, mentions (which chunk names which entity) and edges (which entity relates to which).

A model asked about one chunk at a time tends to call the same thing by several names. vruksha reduces this in two ways. Before each chunk, it shows the model the names already in the graph that look closest to the chunk and asks it to reuse them. After extraction, it merges entities whose embeddings are close, but only when their names also pass a lexical test, described below.

vruksha does not choose a model. You pass `chat`, any object with a `oneshot(prompt, sp=..., max_tokens=...)` method, and `emb_fn`, a function that turns a list of strings into vectors. A local model run through rishi, an API client or a stub for tests all work.

## How to use

[`build_graph`](https://vedicreader.github.io/vruksha/build.html#build_graph) extracts entities from every chunk not yet in the graph, so it can be run again after new documents are added. `graph_search` then runs hybrid search and adds the chunks one edge away from the best hits.

``` python
from litesearch import database
from vruksha import build_graph          # extract + prime + write + embed + merge, incremental

db = database('corpus.db')               # a litesearch store with a tree
build_graph(db, chat=my_chat, emb_fn=my_encoder)   # a local model does the extraction
db.graph_search('how does resolution work', qemb)  # hybrid + a typed-edge walk
```

## When two names are one entity

Priming prevents most duplicate entities, and [`resolve_entities`](https://vedicreader.github.io/vruksha/build.html#resolve_entities) merges the rest. Merging on embedding similarity alone would join `python 3.11` with `python 3.12`, so every merge must also pass [`_lex_ok`](https://vedicreader.github.io/vruksha/entities.html#_lex_ok). Two names pass when their words overlap enough, when one is the acronym of the other, or when they are one-word names that read the same once romanisation is ignored (`Bhīṣma`, `Bheeshma` and `भीष्म`). The numbers in both names must also be identical, which is why the second pair below is kept apart:

``` python
from vruksha.entities import _lex_ok
_lex_ok('multi-head attention', 'multi head attention'), _lex_ok('python 3.11', 'python 3.12')
```

    (True, False)

## Citations found by a regular expression

The model catches loose references, such as a mention of another author’s method. Structured references, such as `Article 5` or `Annex III` in an EU directive, are found more cheaply and more reliably by a regular expression. For a legal corpus, pass `seed_fn=cites, canon=norm_cite` to [`build_graph`](https://vedicreader.github.io/vruksha/build.html#build_graph). [`cites`](https://vedicreader.github.io/vruksha/entities.html#cites) lists the citations in a chunk, and [`norm_cite`](https://vedicreader.github.io/vruksha/entities.html#norm_cite) writes them in one form, so that `Article 3(1)` and `Article 3` are the same entity. Both are off by default, since only a corpus dense with citations needs them.

``` python
cites('as set out in Article 5 and Annex III, and again in article 5'), norm_cite('Article 3(1)')
```

    (['Article 5', 'Annex III'], 'Article 3')

## What it is worth

The case this graph is for is a bridge query: a query that shares no word with the passage it should find, which is reachable only through a reference in another passage. litesearch measured this on 18 cross-reference-dense chunks of one EU VAT directive, searched within the full regulatory store of 3,585 chunks. Following typed edges reached the referenced article 0.61 to 0.87 of the time, against 0.08 to 0.20 for hybrid search alone. The result tracks the model that extracted the graph. A local Qwen3-4B did as well as Claude, while a 1.5B model reached 0.17 on its own and 0.74 once a regular expression added the article citations, as [`cites`](https://vedicreader.github.io/vruksha/entities.html#cites) does. A graph built from word co-occurrence, which this package built before, did no better than hybrid search. The method and the numbers are in litesearch’s `evals/RESULTS.md`.

## Install

``` sh
pip install vruksha
```
