Metadata-Version: 2.4
Name: N-Seek
Version: 0.1.6
Summary: Explainable neuro-symbolic search for trustworthy document retrieval
Project-URL: Homepage, https://github.com/Arkay92/NeuroSeek
Project-URL: Documentation, https://github.com/Arkay92/NeuroSeek#readme
Project-URL: Repository, https://github.com/Arkay92/NeuroSeek
Project-URL: Issues, https://github.com/Arkay92/NeuroSeek/issues
Author: Arkay92
License: MIT License
        
        Copyright (c) 2026 Arkay92
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
License-File: LICENSE
Keywords: neuro-symbolic,rag,retrieval,search,vector-search
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: numpy>=1.26
Requires-Dist: platformdirs>=4.2
Requires-Dist: pydantic>=2.8
Requires-Dist: typing-extensions>=4.12
Provides-Extra: all
Requires-Dist: docgun>=0.2.1; extra == 'all'
Requires-Dist: faiss-cpu>=1.8; extra == 'all'
Requires-Dist: fastapi>=0.111; extra == 'all'
Requires-Dist: langchain-core>=0.3; extra == 'all'
Requires-Dist: llama-index-core>=0.12; extra == 'all'
Requires-Dist: openai>=1.40; extra == 'all'
Requires-Dist: sentence-transformers>=3; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.30; extra == 'all'
Provides-Extra: api
Requires-Dist: fastapi>=0.111; extra == 'api'
Requires-Dist: uvicorn[standard]>=0.30; extra == 'api'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: mypy>=1.11; extra == 'dev'
Requires-Dist: pytest-cov>=5; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: docgun
Requires-Dist: docgun>=0.2.1; extra == 'docgun'
Provides-Extra: embeddings
Requires-Dist: sentence-transformers>=3; extra == 'embeddings'
Provides-Extra: faiss
Requires-Dist: faiss-cpu>=1.8; extra == 'faiss'
Provides-Extra: integrations
Requires-Dist: langchain-core>=0.3; extra == 'integrations'
Requires-Dist: llama-index-core>=0.12; extra == 'integrations'
Provides-Extra: openai
Requires-Dist: openai>=1.40; extra == 'openai'
Description-Content-Type: text/markdown

# NeuroSeek

<p align="center">
  Explainable neuro-symbolic search for Python and trustworthy RAG pipelines.
</p>

<p align="center">
  <img width="256" height="256" alt="NeuroSeek logo" src="https://raw.githubusercontent.com/Arkay92/NeuroSeek/refs/heads/main/neuroseek.png" />
</p>

<p align="center">
  <a href="https://github.com/Arkay92/NeuroSeek/actions/workflows/publish.yml"><img alt="Publish" src="https://github.com/Arkay92/NeuroSeek/actions/workflows/publish.yml/badge.svg" /></a>
  <a href="https://pypi.org/project/neuroseek/"><img alt="PyPI" src="https://img.shields.io/pypi/v/neuroseek.svg" /></a>
  <img alt="Python" src="https://img.shields.io/pypi/pyversions/neuroseek.svg" />
  <img alt="Downloads" src="https://img.shields.io/pypi/dm/neuroseek.svg" />
  <img alt="License" src="https://img.shields.io/pypi/l/neuroseek.svg" />
</p>

> **Package name:** `neuroseek`  
> **Project name:** NeuroSeek  
> **Install:** `pip install neuroseek`  
> **Import:** `from neuroseek import NeuroSymbolicIndex`

**NeuroSeek** combines dense vectors and BM25 with lexical relevance, typed facts, symbolic
rules, metadata constraints, hierarchy, source trust, evidence support, and contradiction
penalties. It returns validated result models with separate relevance and confidence scores
so a mathematically close chunk does not silently override the user's actual constraints.

- Hybrid dense and BM25 candidate generation with reciprocal-rank fusion.
- Deterministic local hash embeddings and exact NumPy search for tests and offline use.
- Optional Sentence Transformers, OpenAI embeddings, and FAISS indexes.
- Typed facts and constrained, explainable scoring rules.
- Metadata, relationship, regular-expression, and temporal operators.
- Indexed hard constraints, soft preferences, and query-specific symbolic goals.
- Optional CrossEncoder reranking after hard eligibility filtering.
- Contextual embeddings, versioned caches, batched indexing, and tombstone compaction.
- Independent confidence scoring and contradiction diagnostics.
- Complete score attribution and source provenance for every result.
- JSON snapshots and an optional SQLite record store.
- Python API, CLI, and FastAPI application factory.
- DocGun, LangChain, and LlamaIndex integrations.
- Dependency-light core with heavyweight backends kept optional.

---

## Before / After

```python
from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex()
index.add(
    id="notice-period",
    text="Employees must provide four weeks written notice.",
    metadata={"document_type": "contract", "trust": 0.95},
)

result = index.search("How much notice must I give?")[0]

print(result.text)
print(result.score)
print(result.confidence)
```

Example output:

```text
Employees must provide four weeks written notice.
0.864
0.835
```

With `explain=True`—the default—each result includes semantic, lexical, symbolic,
hierarchy, trust, and evidence contributions alongside matched rules, rejected rules,
provenance, and contradictions.

---

## Why Neuro-Symbolic Search?

Dense search can return a semantically similar result that is obsolete, applies to the
wrong user, comes from an untrusted source, or contradicts current policy. NeuroSeek makes
those constraints part of retrieval rather than leaving them for the application to infer:

```text
Query
  -> Normalize query and extract high-confidence constraints
  -> Generate dense, BM25, and metadata-filtered candidates
  -> Fuse rankings with reciprocal-rank fusion
  -> Enforce hard symbolic eligibility rules
  -> Optionally rerank a bounded window with a CrossEncoder
  -> Apply semantic-gated soft preferences
  -> Calibrate confidence, evidence, and contextual contradictions
  -> Return ranked results with a complete explanation
```

The default fused score is 50% semantic, 10% lexical, 15% symbolic, 8% hierarchy,
7% trust, and 10% evidence support, minus configurable contradiction and uncertainty
penalties. Every weight can be replaced through `NeuroSeekConfig`.

Raw BM25 values are saturation-normalized before fusion, cosine scores are bounded, and
symbolic contributions are semantic-gated. `fit_fusion_weights()` can learn non-negative
normalized weights from labelled feature rows, while `ConfidenceCalibrator` provides
dependency-free Platt scaling for held-out confidence labels.

### Retrieval architecture

```mermaid
flowchart LR
    Q[Query] --> X[Constraint extraction]
    Q --> D[Dense retrieval]
    Q --> B[BM25 retrieval]
    X --> M[Metadata postings]
    D --> F[Reciprocal-rank fusion]
    B --> F
    M --> F
    F --> H[Hard symbolic rules]
    H --> R[Optional CrossEncoder]
    R --> S[Vectorized scoring and soft rules]
    S --> C[Confidence and contextual contradictions]
    C --> R[Ranked results]
    C --> X[Explanations and provenance]
```

Dense retrieval handles paraphrases while BM25 preserves exact terms, names, dates, and
identifiers. Symbolic rules enforce eligibility after fused candidate generation; they
refine relevant results and are gated so metadata cannot rescue an irrelevant match.

---

## Install

```bash
pip install neuroseek
```

For accelerated FAISS indexes:

```bash
pip install "neuroseek[faiss]"
```

For neural embedding models or DocGun ingestion:

```bash
pip install "neuroseek[embeddings]"
pip install "neuroseek[docgun]"
```

For the complete optional toolset:

```bash
pip install "neuroseek[all]"
```

For development:

```bash
pip install -e ".[dev,all]"
pytest -q
python -m build
```

---

## Quick Start

### Build an Index

```python
from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex(
    embedding="hash",
    vector_backend="numpy-flat",
)

index.add(
    id="contract-12-termination",
    text="Employees must provide four weeks written notice.",
    metadata={
        "document_id": "contract-12",
        "document_type": "employment_contract",
        "section": "Termination",
        "is_current": True,
        "trust": 0.95,
    },
)
```

### Add Symbolic Rules

```python
from neuroseek import Condition, Rule

index.rules.add(Rule(
    name="prefer_current_policy",
    conditions=(
        Condition(field="metadata.document_type", operator="eq", value="employment_contract"),
        Condition(field="metadata.is_current", operator="eq", value=True),
    ),
    score_delta=0.15,
    reason="Current employment contracts are preferred.",
))
```

Required rules reject candidates that do not satisfy their conditions:

```python
index.rules.add(Rule(
    name="current_only",
    conditions=(Condition(field="metadata.is_current", operator="eq", value=True),),
    required=True,
))

results = index.search(
    "What is my notice period?",
    goals=["current_only"],
    explain=True,
)
```

### Add Facts and Relationships

```python
index.facts.add(
    "contract-12",
    "applies_to",
    "employee-robert",
)

index.rules.add(Rule(
    name="user_relevant",
    conditions=(Condition(
        field="metadata.document_id",
        operator="related_to",
        value={"predicate": "applies_to", "object": "employee-robert"},
    ),),
    required=True,
))
```

Supported condition operators are `eq`, `neq`, `gt`, `gte`, `lt`, `lte`, `in`,
`contains`, `exists`, `before`, `after`, `matches`, and `related_to`.

---

## CLI

Create a local index:

```bash
neuroseek init .neuroseek
```

Index one or more text files:

```bash
neuroseek add .neuroseek ./documents --recursive --document-type policy
```

Search and print complete result models as JSON:

```bash
neuroseek search .neuroseek "What is the current policy?" --top-k 5 --explain
```

Add a fact:

```bash
neuroseek fact .neuroseek contract-12 applies_to employee-robert
```

Run a query benchmark from a JSON list:

```bash
neuroseek benchmark .neuroseek benchmark/queries.json
```

---

## Backends

| Capability | Core default | Optional backends |
|---|---|---|
| Embeddings | Deterministic signed feature hashing | Sentence Transformers, OpenAI |
| Vector index | NumPy exact cosine search | FAISS Flat, HNSW, IVF, IVF-PQ |
| Records | In-memory with JSON snapshots | SQLite record store |
| Rules | Constrained typed conditions | Relationship and temporal predicates |
| API | Python API and CLI | FastAPI application factory |
| Ecosystem | Native document records | DocGun, LangChain, LlamaIndex |

### Neural embeddings and FAISS

```python
from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex(
    embedding_backend="sentence-transformers",
    embedding_model="BAAI/bge-small-en-v1.5",
    vector_backend="faiss-hnsw",
)
```

FAISS vectors are L2-normalized and searched with inner product for cosine similarity.
The core package never installs FAISS or Torch unless their extras are explicitly selected.
Hash embeddings remain the deterministic development backend; BGE with FAISS HNSW is the
recommended production profile. `NeuroSeekConfig.production()` supplies those defaults.

### Retrieve and rerank

```python
from neuroseek import NeuroSymbolicIndex
from neuroseek.retrieval.reranking import CrossEncoderReranker

index = NeuroSymbolicIndex(
    embedding_backend="sentence-transformers",
    embedding_model="BAAI/bge-small-en-v1.5",
    vector_backend="faiss-hnsw",
    reranker=CrossEncoderReranker(),
)
```

The CrossEncoder sees only candidates that satisfy hard filters and required rules.
`rerank_candidates` bounds the expensive window and defaults to 30.

---

## DocGun

DocGun handles quality-aware document ingestion; NeuroSeek preserves its document IDs,
element types, hierarchy, page locations, extraction confidence, and parser provenance for
retrieval:

```python
from docgun import ingest
from neuroseek import NeuroSymbolicIndex
from neuroseek.integrations.docgun import index_document

state = ingest("policy.pdf")
index = NeuroSymbolicIndex()

index_document(index, state.document)
results = index.search("What is the current expenses policy?")
```

This keeps the responsibilities explicit:

```text
DocGun      -> quality-aware document ingestion
NeuroSeek   -> constraint-aware, explainable retrieval
PaperTrail  -> application and interface
```

---

## Persistence

Save records, facts, rules, configuration, and a versioned manifest:

```python
index.save(".neuroseek")
```

Restore the index later:

```python
from neuroseek import NeuroSymbolicIndex

index = NeuroSymbolicIndex.load(".neuroseek")
```

The generated `manifest.json` records the schema version, backend, embedding dimensions,
embedding backend, and record count.

---

## Architecture

```text
src/neuroseek/
  __init__.py          # Small public API
  api.py               # NeuroSymbolicIndex facade
  config.py            # Validated configuration and score weights
  models/              # Records, facts, rules, results, and explanations
  embeddings/          # Hash, Sentence Transformers, and OpenAI backends
  indexes/             # NumPy and FAISS vector indexes
  symbolic/            # Fact store, rule store, and condition evaluation
  retrieval/           # Filtering and explainable score fusion
  inference/           # Lightweight evidence graph
  integrations/        # DocGun, FastAPI, LangChain, and LlamaIndex
  storage/             # SQLite record storage
  cli/                 # Command-line interface
tests/                 # Unit and integration tests
examples/              # Search, symbolic, and DocGun examples
CHANGELOG.md            # Release history
pyproject.toml          # Package metadata and dependency extras
```

---

## FastAPI

The optional application factory exposes health and search routes around an existing index:

```python
from neuroseek import NeuroSymbolicIndex
from neuroseek.integrations.fastapi import create_app

index = NeuroSymbolicIndex.load(".neuroseek")
app = create_app(index)
```

Run it with Uvicorn after installing `neuroseek[api]`.

---

## Development

```bash
pip install -e ".[dev,all]"
python -m pytest
ruff check .
python -m build
```

The test suite covers retrieval, symbolic constraints, explanations, persistence, batch
indexing, deletion, and the duck-typed DocGun adapter.

### Retrieval benchmark

The included synthetic policy benchmark compares dense-only retrieval with the same index
using required rules and metadata constraints:

```bash
python -m benchmarks.retrieval_quality
```

It reports Recall@1/@3/@10, Precision@3, MRR, NDCG@10, constraint accuracy,
forbidden-result rate, explanation fidelity, p50/p95 latency, indexing throughput, and
peak memory per record. Separate suites cover semantic paraphrases, exact identifiers,
constraint collisions, adversarial similarity, and keyword-heavy policies. Results compare
dense-only, hybrid retrieval, and hybrid-plus-rules. Replace
`benchmarks/synthetic_policy.json` or call `run(path)` to evaluate another corpus.

---

## License

MIT

---

## Contributing

Contributions are welcome. Please include a minimal corpus or generated fixture, the
expected ranking and explanation, and a regression test for retrieval or reasoning changes.

---

## Citation

```bibtex
@software{NeuroSeek2026,
  title={NeuroSeek: Explainable Neuro-Symbolic Search for Python},
  author={Arkay92},
  url={https://github.com/Arkay92/NeuroSeek},
  year={2026},
  version={0.1.6},
}
```

---

## Acknowledgments

- [NumPy](https://numpy.org/) for the dependency-light exact vector backend.
- [FAISS](https://github.com/facebookresearch/faiss) for accelerated similarity search.
- [Sentence Transformers](https://www.sbert.net/) for neural embeddings and reranking tools.
- [Pydantic](https://docs.pydantic.dev/) for validated public models and configuration.
- [DocGun](https://github.com/Arkay92/DocGun) for quality-aware structured ingestion.
