Metadata-Version: 2.4
Name: openenv-halluguard
Version: 2.0.0
Summary: Python SDK for HallucinationGuard-Env — evaluate and train any LLM to avoid hallucinations using OpenEnv. Research-grade grader: ROUGE + BERTScore + AlignScore + NLI-DeBERTa-v3-large. 1M+ training examples across 38 datasets.
Project-URL: Homepage, https://ss-360.github.io/Openenv-halluguard/
Project-URL: HF Space, https://huggingface.co/spaces/SamSankar/hallucination-guard-env
Project-URL: Documentation, https://samsankar-hallucination-guard-env.hf.space/docs
Project-URL: Leaderboard, https://samsankar-hallucination-guard-env.hf.space/leaderboard
Project-URL: Repository, https://github.com/SS-360/Openenv-halluguard
Project-URL: Bug Tracker, https://github.com/SS-360/Openenv-halluguard/issues
Author-email: Sam Sankar P <p.samsankar2005@gmail.com>
License: MIT
Keywords: ai-safety,alignscore,benchmark,bertscore,fact-checking,grounded-generation,grpo,hallucination,hallucination-avoidance,hallucination-detection,llm,llm-evaluation,llm-training,nlp,openenv,question-answering,reinforcement-learning,rouge
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Requires-Dist: requests>=2.28.0
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.18.0; extra == 'anthropic'
Provides-Extra: groq
Requires-Dist: groq>=0.4.0; extra == 'groq'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Description-Content-Type: text/markdown

# openenv-halluguard

**Python SDK for HallucinationGuard-Env** — evaluate and train any LLM to avoid hallucinations using the OpenEnv framework.

[![PyPI](https://img.shields.io/pypi/v/openenv-halluguard)](https://pypi.org/project/openenv-halluguard/)
[![License](https://img.shields.io/badge/License-MIT-green)](https://github.com/SS-360/Openenv-halluguard/blob/main/LICENSE)
[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compatible-blue)](https://github.com/meta-pytorch/OpenEnv)
[![Dataset](https://img.shields.io/badge/Dataset-1M%2B_examples-orange)](https://huggingface.co/datasets/SamSankar/hallucination-guard-cache)

---

## What is this?

HallucinationGuard-Env is a **reinforcement learning environment** that trains AI models to stop making things up. It provides a reward signal based on 9 research-grade components including ROUGE, BERTScore, AlignScore, and NLI-DeBERTa-v3-large — built on 1M+ real-world QA examples across 38 datasets.

This package is the **Python SDK** that lets you evaluate any LLM against the environment in 3 lines of code.

---

## Install

```bash
pip install openenv-halluguard
```

Optional extras for specific model providers:

```bash
pip install openenv-halluguard[openai]     # + openai
pip install openenv-halluguard[anthropic]  # + anthropic
pip install openenv-halluguard[groq]       # + groq (free tier)
```

---

## Quick Start

```python
from openenv_halluguard import HallucinationGuardEnv

# Define your model — must answer from the provided context only
def my_model(question, context):
    # call any LLM here
    return "your answer based on context"

# Evaluate
env = HallucinationGuardEnv()
results = env.evaluate(my_model, episodes=5, model_name="my-model")
env.print_report(results)
```

---

## Works With Any LLM

### OpenAI

```python
import openai
from openenv_halluguard import HallucinationGuardEnv

client = openai.OpenAI(api_key="sk-...")

def gpt4(question, context):
    r = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Answer ONLY from the context provided."},
            {"role": "user", "content": f"Context: {context}\n\nQuestion: {question}"}
        ]
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(gpt4, episodes=10, model_name="gpt-4o")
env.print_report(results)
```

### Anthropic Claude

```python
import anthropic
from openenv_halluguard import HallucinationGuardEnv

client = anthropic.Anthropic(api_key="sk-ant-...")

def claude(question, context):
    msg = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=256,
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return msg.content[0].text

env = HallucinationGuardEnv()
results = env.evaluate(claude, episodes=10, model_name="claude-3-5-sonnet")
env.print_report(results)
```

### Groq (free tier — Llama, Mixtral)

```python
from groq import Groq
from openenv_halluguard import HallucinationGuardEnv

client = Groq(api_key="YOUR_GROQ_KEY")

def llama(question, context):
    r = client.chat.completions.create(
        model="llama-3.1-8b-instant",
        messages=[{"role": "user", "content": f"Context: {context}\n\nQ: {question}\n\nAnswer from context only."}],
        max_tokens=200
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(llama, episodes=10, model_name="llama-3.1-8b")
env.print_report(results)
```

### Ollama (local, fully offline)

```python
import requests
from openenv_halluguard import HallucinationGuardEnv

def ollama(question, context):
    r = requests.post(
        "http://localhost:11434/api/generate",
        json={"model": "llama3", "stream": False,
              "prompt": f"Context: {context}\n\nQ: {question}\n\nAnswer from context only."}
    )
    return r.json()["response"]

env = HallucinationGuardEnv()
results = env.evaluate(ollama, episodes=5, model_name="llama3-local")
env.print_report(results)
```

---

## GRPO Training (5 lines)

```python
from trl import GRPOConfig, GRPOTrainer
from openenv_halluguard import HallucinationGuardEnv

env = HallucinationGuardEnv(verbose=False)

def reward_fn(prompts, completions, **kwargs):
    return [env.step(c).get("reward", 0.0) for c in completions]

trainer = GRPOTrainer(
    model="Qwen/Qwen3-1.7B",
    reward_funcs=reward_fn,
    args=GRPOConfig(output_dir="./halluguard-grpo", num_train_epochs=1),
)
trainer.train()
```

---

## Reward System (v2.0 — Research-Grade)

9 components, each contributing to the final reward in [0.0, 1.0]:

| Component | Weight | Reference |
|---|---|---|
| Factual Correctness | 30% | Semantic similarity + entity overlap |
| Source Grounding | 25% | Word coverage vs context |
| Citation Accuracy | 10% | Fuzzy match of source_quote in document |
| Confidence Calibration | 8% | \|confidence − correctness\| |
| Semantic Consistency | 7% | NLI-DeBERTa-v3-large entailment |
| Hallucination Penalty | 5% | Multi-type fabrication detection |
| ROUGE-L | 5% | Lin (2004) |
| BERTScore | 5% | Zhang et al. (2020) |
| AlignScore | 5% | Zha et al., ACL 2023 |

---

## Datasets

1M+ examples cached permanently across 38 real-world QA datasets including SQuAD, TriviaQA, HotpotQA, HaluEval, TruthfulQA, MedQA, AQUA-RAT, HellaSwag, AdversarialQA, and more. No downloads required at runtime — instant boot.

Full dataset list: [HuggingFace Space](https://huggingface.co/spaces/SamSankar/hallucination-guard-env)

---

## HTTP API

The environment is also available as a live REST API:

```python
import requests

BASE = "https://samsankar-hallucination-guard-env.hf.space"

obs    = requests.post(f"{BASE}/reset").json()
result = requests.post(f"{BASE}/step", json={"answer": "your answer"}).json()

print(result["reward"])           # 0.0 – 1.0
print(result["is_hallucination"]) # True / False
print(result["feedback"])         # Detailed explanation
```

---

## Links

| | |
|---|---|
| 🌐 Site | https://ss-360.github.io/Openenv-halluguard/ |
| 🤗 HF Space | https://huggingface.co/spaces/SamSankar/hallucination-guard-env |
| 📖 API Docs | https://samsankar-hallucination-guard-env.hf.space/docs |
| 🏆 Leaderboard | https://samsankar-hallucination-guard-env.hf.space/leaderboard |
| 💾 Dataset Cache | https://huggingface.co/datasets/SamSankar/hallucination-guard-cache |
| 🐛 Issues | https://github.com/SS-360/Openenv-halluguard/issues |

---

**Build for to Train AI Models to Stop Hallucinating**

MIT License
