Metadata-Version: 2.4
Name: ontocast
Version: 0.4.0
Summary: Agentic ontology and knowledge graph co-generation
License-File: LICENSE
Requires-Python: <4.0,>=3.12
Requires-Dist: asyncio>=3.4.3
Requires-Dist: click>=8.1.8
Requires-Dist: duckduckgo-search>=8.1.1
Requires-Dist: fastapi>=0.115.0
Requires-Dist: fastembed[all]>=0.8.0
Requires-Dist: hdbscan>=0.8.41
Requires-Dist: httpx>=0.27.0
Requires-Dist: langchain-core>=0.3.60
Requires-Dist: langchain-experimental>=0.3.4
Requires-Dist: langchain-huggingface>=0.2.0
Requires-Dist: langchain-ollama>=0.3.3
Requires-Dist: langchain-openai>=0.3.17
Requires-Dist: langchain>=0.3.25
Requires-Dist: langgraph>=0.2.35
Requires-Dist: neo4j>=5.28.1
Requires-Dist: networkx>=3.0
Requires-Dist: owlready2>=0.47
Requires-Dist: oxrdflib>=0.5.0
Requires-Dist: pydantic>=2.11.9
Requires-Dist: pyld>=2.0.4
Requires-Dist: python-multipart>=0.0.22
Requires-Dist: qdrant-client>=1.15.1
Requires-Dist: rapidfuzz>=3.13.0
Requires-Dist: rdflib>=7.1.4
Requires-Dist: rich>=14.0.0
Requires-Dist: simsimd>=6.2.1
Requires-Dist: suthing>=0.4.1
Requires-Dist: umap-learn>=0.5.11
Requires-Dist: uvicorn[standard]>=0.32.0
Provides-Extra: all
Requires-Dist: docling>=2.57.0; extra == 'all'
Requires-Dist: easyocr>=1.7.2; extra == 'all'
Requires-Dist: sentence-transformers>=5.1.1; extra == 'all'
Provides-Extra: dev
Requires-Dist: pre-commit>=4.2.0; extra == 'dev'
Requires-Dist: pytest-dotenv>=0.5.2; extra == 'dev'
Requires-Dist: pytest-order>=1.3.0; extra == 'dev'
Requires-Dist: pytest>=8.3.5; extra == 'dev'
Requires-Dist: requests>=2.32.3; extra == 'dev'
Requires-Dist: ruff>=0.11.2; extra == 'dev'
Requires-Dist: ty>=0.0.14; extra == 'dev'
Provides-Extra: docs
Requires-Dist: griffe<2.0.0; extra == 'docs'
Requires-Dist: mkdocs-gen-files>=0.5.0; extra == 'docs'
Requires-Dist: mkdocs-glightbox>=0.4.0; extra == 'docs'
Requires-Dist: mkdocs-jupyter>=0.25.1; extra == 'docs'
Requires-Dist: mkdocs-literate-nav>=0.6.2; extra == 'docs'
Requires-Dist: mkdocs-material>=9.6.14; extra == 'docs'
Requires-Dist: mkdocs>=1.6.1; extra == 'docs'
Requires-Dist: mkdocstrings-python>=2.0.3; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.29.1; extra == 'docs'
Provides-Extra: plot
Requires-Dist: pygraphviz>=1.14; extra == 'plot'
Description-Content-Type: text/markdown

# OntoCast <img src="https://raw.githubusercontent.com/growgraph/ontocast/refs/heads/main/docs/assets/favicon.ico" alt="Agentic Ontology Triplecast logo" style="height: 32px; width:32px;"/>

### Agentic ontology-assisted framework for semantic triple extraction

![Python](https://img.shields.io/badge/python-3.12-blue.svg) 
[![PyPI version](https://badge.fury.io/py/ontocast.svg)](https://badge.fury.io/py/ontocast)
[![PyPI Downloads](https://static.pepy.tech/badge/ontocast)](https://pepy.tech/projects/ontocast)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![pre-commit](https://github.com/growgraph/ontocast/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/growgraph/ontocast/actions/workflows/pre-commit.yml)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.17796467.svg)](https://doi.org/10.5281/zenodo.17796467)

---

## Overview

OntoCast is a framework for extracting semantic triples (creating a Knowledge Graph) from documents using an agentic, ontology-driven approach. It combines ontology management, natural language processing, and knowledge graph serialization to turn unstructured text into structured, queryable data.

---

## Key Features

- **Ontology-Guided Extraction**: Ensures semantic consistency and co-evolves ontologies
- **Entity Disambiguation**: Resolves references across document chunks (embedding + symbolic alignment)
- **Parallel Processing**: Per-unit ontology and facts loops with configurable worker concurrency
- **Multi-Format Support**: Handles text, JSON, PDF, and Markdown
- **Semantic Chunking**: Splits text based on semantic similarity
- **RDF Output**: Produces standardized RDF/Turtle (optional JSON-LD LLM wire format)
- **RDF 1.2 Provenance**: Quoted-triple / reification support; optional `strip_provenance` on API output
- **Triple Store Integration**: Supports Neo4j (n10s), Apache Fuseki, and filesystem fallback
- **Ontology Context Modes**: Catalog selection, Qdrant vector retrieval, or fixed catalog ontology
- **Tenancy**: Partition Fuseki datasets and Qdrant collections by tenant and project
- **REST API**: Document processing, ontology catalog management, and graph-matching endpoints
- **Hierarchical Configuration**: Type-safe configuration system with environment variable support
- **Automatic LLM Caching**: Built-in response caching for improved performance and cost reduction
- **GraphUpdate Operations**: Token-efficient SPARQL-based updates instead of full graph regeneration
- **Budget Tracking**: Comprehensive tracking of LLM usage and triple generation metrics
- **Ontology Versioning**: Automatic semantic versioning with hash-based lineage tracking

---

## Applications

OntoCast can be used for:

- **Knowledge Graph Construction**: Build domain-specific or general-purpose knowledge graphs from documents
- **Semantic Search**: Power search and retrieval with structured triples
- **GraphRAG**: Enable retrieval-augmented generation over knowledge graphs (e.g., with LLMs)
- **Ontology Management**: Automate ontology creation, validation, and refinement
- **Data Integration**: Unify data from diverse sources into a semantic graph

---

## Installation

```sh
uv add ontocast[doc-processing] 
# or
pip install ontocast
```

### Optional features: document processing (PDFs, PPT, OCR, semantic chunking):

```sh
uv add "ontocast[doc-processing]"
# or
pip install "ontocast[doc-processing]"
```

---

## Quick Start

### 1. Configuration

Copy the example file and edit as needed:

```bash
cp .env.example .env
```

Minimal settings:

```bash
# LLM Configuration
LLM_PROVIDER=openai
LLM_API_KEY=your-api-key-here
LLM_MODEL_NAME=gpt-4o-mini
LLM_TEMPERATURE=0.0

# Server Configuration
PORT=8999
BASE_RECURSION_LIMIT=1000
ESTIMATED_CHUNKS=30
MAX_VISITS=1
RENDER_MODE=ontology_and_facts
ONTOLOGY_MAX_TRIPLES=50000

# Path Configuration
ONTOCAST_WORKING_DIRECTORY=/path/to/working
ONTOCAST_ONTOLOGY_DIRECTORY=/path/to/ontologies
ONTOCAST_CACHE_DIR=/path/to/cache

# Optional: Triple Store Configuration (Fuseki preferred over Neo4j)
FUSEKI_URI=http://localhost:3030
FUSEKI_AUTH=admin/admin
# Datasets default to ontocast--test--facts / ontocast--test--ontologies when unset

# Optional: Web search grounding (search-later mode)
WEB_SEARCH_ENABLED=false
WEB_SEARCH_PROVIDER=duckduckgo
WEB_SEARCH_TOP_K=3
```

See [Configuration Guide](docs/user_guide/configuration.md) and `.env.example` for the full surface (embeddings, Qdrant, aggregation, etc.).

### 2. Start Server

```bash
# Backend automatically detected from .env configuration
ontocast --env-path .env

# Process a specific file (batch mode)
ontocast --env-path .env --input-path ./document.pdf

# Process only the first N chunks (for testing)
ontocast --env-path .env --head-chunks 5
```

Paths and triple-store credentials are configured via `.env` — not CLI overrides.

### 3. Process Documents

```bash
curl -X POST http://localhost:8999/process -F "file=@document.pdf"
```

JSON body example:

```bash
curl -X POST http://localhost:8999/process \
  -H "Content-Type: application/json" \
  -d '{"text": "Your document text here"}'
```

### 4. API Endpoints

The OntoCast server exposes a REST API. Common endpoints:

- **POST /process** — full document pipeline (JSON or multipart file upload)
- **POST /process_unit** — single-unit pipeline (useful for debugging prompts)
- **POST /ontologies** — upload catalog ontologies (Turtle)
- **POST /flush** — flush/clean triple store data
- **GET /health**, **GET /info** — health and service metadata
- **POST /match/entities** (and related match routes) — entity alignment and evaluation

```bash
curl -X POST http://localhost:8999/process -F "file=@document.pdf"

curl -X POST http://localhost:8999/flush

curl -X POST "http://localhost:8999/flush?dataset=my_dataset"
```

For Fuseki, an optional `dataset` query parameter targets a specific dataset; Neo4j ignores it and clears the whole database.

Full reference: [API Endpoints](docs/user_guide/api.md).

---

## Workflow

The extraction pipeline converts documents to Markdown, chunks them, runs parallel per-unit ontology and facts loops, then merges and serializes results.

![Workflow diagram](docs/assets/graph.png)

Landscape layout: [docs/assets/graph.lr.png](docs/assets/graph.lr.png). Per-unit render/critic loops: [Workflow guide](docs/user_guide/workflow.md).

Regenerate diagrams after graph changes: `uv run plot-graph`

---

## LLM Caching

OntoCast includes automatic LLM response caching to improve performance and reduce API costs. Caching is enabled by default and requires no configuration.

### Cache Locations

- **Tests**: `.test_cache/llm/` in the current working directory
- **Windows**: `%USERPROFILE%\AppData\Local\ontocast\llm\`
- **Unix/Linux**: `~/.cache/ontocast/llm/` (or `$XDG_CACHE_HOME/ontocast/llm/`)

### Benefits

- **Faster Execution**: Repeated queries return cached responses instantly
- **Cost Reduction**: Identical requests don't hit the LLM API
- **Offline Capability**: Tests can run without API access if responses are cached
- **Transparent**: No configuration required — works automatically

Details: [LLM Caching](docs/user_guide/llm_caching.md)

---

## Configuration System

OntoCast uses a hierarchical configuration system built on Pydantic BaseSettings:

### Environment Variables

| Variable | Description | Default | Required |
|----------|-------------|---------|----------|
| `LLM_API_KEY` | API key for LLM provider | - | Yes (OpenAI) |
| `LLM_PROVIDER` | LLM provider (openai, ollama) | openai | No |
| `LLM_MODEL_NAME` | Model name | gpt-4o-mini | No |
| `LLM_TEMPERATURE` | Temperature setting | 0.0 | No |
| `ONTOCAST_WORKING_DIRECTORY` | Working directory path | - | Yes (filesystem mode) |
| `ONTOCAST_ONTOLOGY_DIRECTORY` | Ontology seed files | - | No |
| `PORT` | Server port | 8999 | No |
| `MAX_VISITS` | Maximum render/critic visits per unit loop | 1 | No |
| `BASE_RECURSION_LIMIT` | Base recursion limit for workflow | 1000 | No |
| `RENDER_MODE` | `ontology`, `facts`, or `ontology_and_facts` | ontology_and_facts | No |
| `ONTOLOGY_MAX_TRIPLES` | Maximum triples in ontology graph | 50000 | No |
| `ONTOCAST_CACHE_DIR` | Custom cache directory | Platform default | No |
| `WEB_SEARCH_ENABLED` | Optional web grounding (search-later) | false | No |

See [Configuration Guide](docs/user_guide/configuration.md) for chunking, Qdrant, embeddings, aggregation, and web-search variables.

### Triple Store Configuration

```bash
# Fuseki (Preferred)
FUSEKI_URI=http://localhost:3030
FUSEKI_AUTH=admin/admin

# Neo4j (Alternative)
NEO4J_URI=bolt://localhost:7687
NEO4J_AUTH=neo4j/password
```

When multiple triple stores are configured, **Fuseki is preferred over Neo4j**. See [Triple Store Setup](docs/user_guide/triple_stores.md).

### CLI Parameters

```bash
ontocast --env-path .env
ontocast --env-path .env --input-path ./document.pdf
ontocast --env-path .env --head-chunks 5
ontocast --env-path .env --tenant acme --project reports
```

---

## Triple Store Setup

OntoCast supports multiple triple store backends with automatic fallback:

1. **Apache Fuseki** (Recommended) — Native RDF with SPARQL support
2. **Neo4j with n10s** — Graph database with RDF capabilities  
3. **Filesystem** (Fallback) — Local file-based storage

### Quick Setup with Docker

**Fuseki:**
```bash
cd docker/fuseki
cp .env.example .env
# Edit .env with your values
docker compose --env-file .env fuseki up -d
```

**Neo4j:**
```bash
cd docker/neo4j
cp .env.example .env
# Edit .env with your values
docker compose --env-file .env neo4j up -d
```

See [Triple Store Setup](docs/user_guide/triple_stores.md) for detailed instructions.

---

## Documentation

- [Quick Start Guide](docs/getting_started/quickstart.md) — Get started quickly
- [Workflow](docs/user_guide/workflow.md) — Pipeline and diagrams
- [Configuration System](docs/user_guide/configuration.md) — Environment variables
- [API Endpoints](docs/user_guide/api.md) — REST reference
- [Tenancy](docs/user_guide/tenancy.md) — Multi-tenant stores
- [Ontology Context](docs/user_guide/ontology_context.md) — Catalog vs vector retrieval
- [Triple Store Setup](docs/user_guide/triple_stores.md) — Fuseki / Neo4j setup
- [LLM Caching](docs/user_guide/llm_caching.md) — Automatic response caching
- [User Guide](docs/user_guide/concepts.md) — Core concepts
- [API Reference](docs/reference/onto/state.md) — Python API (MkDocs)

Build docs locally: `uv run mkdocs build`

---

## Highlights

### Ontology Management

- **Automatic Versioning**: Semantic version increment based on change analysis (MAJOR/MINOR/PATCH)
- **Hash-Based Lineage**: Git-style versioning with parent hashes for tracking ontology evolution
- **Multiple Version Storage**: Versions stored as separate named graphs in Fuseki triple stores
- **Timestamp Tracking**: `updated_at` field tracks when ontology was last modified

### GraphUpdate System

- **Token Efficiency**: LLM outputs structured SPARQL operations (insert/delete) instead of full TTL graphs
- **Incremental Updates**: Only changes are generated, dramatically reducing token usage
- **Structured Operations**: TripleOp operations with explicit prefix declarations for precise updates

### Budget Tracking

- **LLM Statistics**: Tracks API calls, characters sent/received for cost monitoring
- **Triple Metrics**: Tracks ontology and facts triples generated per operation
- **Summary Reports**: Budget summaries logged at end of processing

See [CHANGELOG.md](CHANGELOG.md) for release-by-release notes.

---

## Examples

### Basic Usage

```python
from ontocast.config import Config
from ontocast.toolbox import ToolBox

config = Config()
tools = ToolBox(config)

# Process documents via tools / workflow graph
```

### Server Usage

```bash
ontocast --env-path .env --input-path ./document.pdf --head-chunks 10
```

---

## Running tests

Tests load settings from the process environment. To run the suite with variables from a project `.env` file:

```bash
bash -c 'set -a; source .env; set +a; uv run pytest test'
```

Integration tests (for example Qdrant) read optional variables such as `QDRANT_URI` and `QDRANT_API_KEY` from that environment; they skip automatically when the service is unreachable.

---

## Contributing

We welcome contributions! Please see our [Contributing Guide](docs/contributing.md) for details.

---

## License

This project is licensed under the Apache License 2.0 — see the [LICENSE](LICENSE) file for details.

---

## Support

- **Documentation**: [docs/](docs/)
- **Issues**: [GitHub Issues](https://github.com/growgraph/ontocast/issues)
- **Discussions**: [GitHub Discussions](https://github.com/growgraph/ontocast/discussions)
