Metadata-Version: 2.4
Name: ragone-ai
Version: 0.1.1
Summary: ModernBERT-based RAG evaluation model
Author: Hariharan Suthan
License-Expression: MIT
Keywords: RAG,retrieval-augmented-generation,evaluation,ModernBERT,NLP
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1
Requires-Dist: transformers>=4.45
Requires-Dist: huggingface_hub>=0.25
Requires-Dist: safetensors>=0.4
Dynamic: license-file

# RAGOne

**A lightweight, ModernBERT-based model for evaluating retrieved context in RAG pipelines.**

RAGOne evaluates retrieved context across four tasks: answerability,
evidence quality, contradiction detection, and relevance classification.

- **Hugging Face model:** [Hariharan05/ragone](https://huggingface.co/Hariharan05/ragone)
- **PyPI package:** `ragone-ai`
- **Python import:** `from ragone import RAGOne`

## Installation

```bash
pip install ragone-ai
```

## Quick start

```python
from ragone import RAGOne

model = RAGOne.from_pretrained("Hariharan05/ragone")

result = model.predict(
    query="What is the refund period?",
    context="Customers can request a refund within 30 days.",
    task="answerability",
    options=["yes", "no"],
)

print(result)
```

On first use, the model files are downloaded from Hugging Face.
The package includes the inference code; model weights are hosted
separately in the Hugging Face repository.

## Supported tasks

| Task | Purpose | Example options |
|---|---|---|
| `answerability` | Can the query be answered from the context? | `["yes", "no"]` |
| `evidence_quality` | How strong is the context as evidence? | `["strong", "medium", "weak"]` |
| `contradiction` | Does the statement contradict the context? | `["yes", "no"]` |
| `relevance` | Is the context relevant to the query? | `["relevant", "irrelevant"]` |

Use labels consistent with the model's training conventions.

## Example output

```python
{
    "prediction": "yes",
    "scores": [
        {"label": "yes", "score": 0.89},
        {"label": "no", "score": 0.11}
    ],
    "latency_ms": 85.0,
    "task": "answerability"
}
```

Scores and latency above are illustrative, not guaranteed results.
Scores are normalized across candidate options and are not necessarily
calibrated probabilities of correctness.

## Model and training

RAGOne uses a fine-tuned `answerdotai/ModernBERT-base` encoder and a
neural scoring head to rank candidate labels.

Reported training details:

- Dataset: 5,074 records
- Training: 5 epochs
- Best reported validation loss: 0.2697
- Reported final validation accuracy: 94.591%
- Separate evaluation: 37/40 correct (92.5%)

The separate evaluation used 10 examples per task. These results are
preliminary and do not establish performance on all real-world RAG
systems. The training and evaluation metrics come from different
evaluations and should not be compared as if they were the same test.

## Intended use

RAGOne is intended for research, experimentation, and development
of RAG evaluation workflows. It can help assess retrieved-context
relevance, answerability, evidence quality, and contradiction.

It does not perform document retrieval or answer generation and
should not be treated as a definitive source of truth.

## Limitations

- Performance may vary by domain, language, and retrieval strategy.
- Inputs are limited to the model's configured token length.
- Results depend on the candidate labels supplied.
- Softmax scores should not be interpreted as calibrated confidence
  without additional validation.
- Evaluate on independent, representative datasets before production use.

## License

RAGOne is intended for research use under the applicable OpenRAIL
license terms. Review the license included with the model repository
and the base model's license before using or redistributing model
weights. The package metadata alone does not replace the full license.

## Links

- Model: https://huggingface.co/Hariharan05/ragone
- Author: https://huggingface.co/Hariharan05
