Metadata-Version: 2.4
Name: cognigraph-chunker
Version: 2.2.4
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Topic :: Text Processing :: General
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Summary: Fast text chunking toolkit: fixed-size, delimiter, semantic, cognition-aware, intent-driven, topology-aware, enriched, and adaptive strategies
Keywords: chunking,rag,embeddings,nlp,text-splitting,semantic-chunking
Author-email: Gedank Rayze <info@gedankrayze.com>
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://github.com/gedankrayze/cognigraph-chunker#readme
Project-URL: Homepage, https://github.com/gedankrayze/cognigraph-chunker
Project-URL: Repository, https://github.com/gedankrayze/cognigraph-chunker

# cognigraph-chunker

Fast text chunking toolkit for RAG pipelines, written in Rust with Python bindings.

Eight chunking strategies — fixed-size, delimiter, semantic, cognition-aware,
intent-driven, topology-aware, enriched, and adaptive — plus token-aware
merging, signal-processing primitives, and five intrinsic quality metrics.

## Install

```sh
pip install cognigraph-chunker
```

## Quick start

```python
from cognigraph_chunker import Chunker, OllamaProvider, SemanticConfig, semantic_chunk

# Fixed-size chunking — no external services needed
for chunk in Chunker("Your long document text here...", size=1024):
    print(chunk)

# Semantic chunking with a local Ollama embedding model
provider = OllamaProvider(model="nomic-embed-text")
result = semantic_chunk(open("document.md").read(), provider, SemanticConfig())
for text, offset in result.chunks:
    print(f"[{offset}] {text[:80]}")
```

Cognition-aware chunking (entity/discourse-preserving boundaries with quality
metrics), intent-driven, topology-aware, enriched, and adaptive strategies are
exposed as `cognitive_chunk`, `intent_chunk`, `topo_chunk`, `enriched_chunk`,
and `adaptive_chunk`, alongside `evaluate_chunks` for standalone quality
scoring.

Full documentation, the CLI, and the REST API live in the
[project repository](https://github.com/gedankrayze/cognigraph-chunker).

MIT licensed.

