Metadata-Version: 2.4
Name: judie
Version: 0.0.2
Summary: Judge and evaluate your chunk quality with Judie, the Owl! Quick, easy, and effective!
Author: Bhavnick Minhas, Shreyash Nigam
Maintainer: Bhavnick Minhas, Shreyash Nigam
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/chonkie-inc/judie
Project-URL: Repository, https://github.com/chonkie-inc/judie
Project-URL: Documentation, https://github.com/chonkie-inc/judie#readme
Project-URL: Bug Tracker, https://github.com/chonkie-inc/judie/issues
Keywords: chunking,rag,llm,evaluation,nlp,information-retrieval
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: chonkie[all]<1.1.0,>=1.0.0
Requires-Dist: datasets>=2.0.0
Provides-Extra: dev
Requires-Dist: ruff; extra == "dev"
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Dynamic: license-file

<div align="center">

![🦉 Judie Logo](./assets/judie-logo.png)

# 🦉 judie ✨
_Judge and evaluate your chunk quality with Judie, the Owl! Quick, easy, and effective!_

</div>

Chunking is a crucial step for RAG systems and LLM workflows. Most of the time, chunk quality is rarely evaluated even though it can have a significant impact on the performance of the system. Judie makes it super easy to evaluate your chunks!

## 📦 Installation

Installation is super easy! Just run the following command in your terminal:

```bash
pip install judie
```

## 🧑‍⚖️ Usage

Judie works together with [chonkie](https://github.com/chonkie-inc/chonkie) to evaluate your chunks. It supports all the chunkers that chonkie supports, and as long as any chunker can be wrapped in a `chonkie.BaseChunker` wrapper, Judie will support it.

Here's a simple example of how to use Judie:

```python 
from judie import Evaluator 
from chonkie import RecursiveChunker

chunker = RecursiveChunker(chunk_size=1000, chunk_overlap=200)

# Load your dataset
dataset = ["...", "..."]

# Initialize the evaluator
evaluator = Evaluator(chunker=chunker,
                      dataset=dataset, 
                      embedding_model=embedding_model)

# Evaluate your chunks
metrics = evaluator.evaluate()

# Print the metrics
print(metrics['Recall@10'])
```

## 🧩 Available Benchmarks

| Benchmark | Description |
|-----------|-------------|
| [🧸 Gacha](https://huggingface.co/datasets/chonkie-ai/gacha) | 🧸 Gacha is a corpus of 100 most popular textbooks from Gutenberg Corpus and numerous NIAH-style questions for evaluating chunking algorithms! |


## 📚 Citation

If you use Judie in your work, please cite it as follows:

```bibtex
@software{judie2025,
  author = {Bhavnick Minhas and Shreyash Nigam},
  title = {🦉 Judie: Evaluating chunk quality with LLM Judges},
  url = {https://github.com/chonkie-inc/judie},
  version = {0.1.0},
  year = {2025},
}
```
