Metadata-Version: 2.4
Name: openenv-halluguard
Version: 1.0.0
Summary: Python SDK for HallucinationGuard-Env — evaluate and train LLMs to avoid hallucination using OpenEnv
Project-URL: Homepage, https://huggingface.co/spaces/SamSankar/hallucination-guard-env
Project-URL: Documentation, https://samsankar-hallucination-guard-env.hf.space/docs
Project-URL: Repository, https://huggingface.co/spaces/SamSankar/hallucination-guard-env
Project-URL: Bug Tracker, https://huggingface.co/spaces/SamSankar/hallucination-guard-env/discussions
License: MIT
Keywords: ai-safety,benchmark,evaluation,hallucination,llm,nlp,openenv,reinforcement-learning
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Requires-Dist: requests>=2.28.0
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.18.0; extra == 'anthropic'
Provides-Extra: groq
Requires-Dist: groq>=0.4.0; extra == 'groq'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Description-Content-Type: text/markdown

# openenv-halluguard

> **Python SDK for HallucinationGuard-Env — evaluate and train any LLM to avoid hallucinations using OpenEnv.**

[![PyPI](https://img.shields.io/pypi/v/openenv-halluguard)](https://pypi.org/project/openenv-halluguard/)
[![License](https://img.shields.io/badge/License-MIT-green)](LICENSE)
[![OpenEnv](https://img.shields.io/badge/OpenEnv-Compatible-blue)](https://github.com/meta-pytorch/OpenEnv)

## Install

```bash
pip install openenv-halluguard
```

## Quick Start

```python
from openenv_halluguard import HallucinationGuardEnv

# Define your model function
def my_model(question, context):
    # Call your LLM here — must answer from context only
    return "your answer based on context"

# Evaluate
env = HallucinationGuardEnv()
results = env.evaluate(my_model, episodes=5, model_name="my-model")
env.print_report(results)
```

## What It Does

Connects to the live [HallucinationGuard-Env](https://huggingface.co/spaces/SamSankar/hallucination-guard-env) — an OpenEnv RL environment with **100,000+ QA examples** across 15 real-world datasets. Your model is scored on:

| Component | Weight |
|---|---|
| Factual correctness | 30% |
| Source grounding | 20% |
| Citation accuracy | 15% |
| Confidence calibration | 15% |
| Semantic consistency | 10% |
| Hallucination penalty | 10% |

## Works With Any LLM

```python
# OpenAI
import openai
client = openai.OpenAI(api_key="sk-...")

def openai_model(question, context):
    r = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(openai_model, episodes=5, model_name="gpt-4")
```

```python
# Anthropic Claude
import anthropic
client = anthropic.Anthropic(api_key="sk-ant-...")

def claude_model(question, context):
    msg = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=256,
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return msg.content[0].text

env = HallucinationGuardEnv()
results = env.evaluate(claude_model, episodes=5, model_name="claude-haiku")
```

```python
# Groq (fast inference)
from groq import Groq
client = Groq(api_key="gsk_...")

def groq_model(question, context):
    r = client.chat.completions.create(
        model="llama3-8b-8192",
        messages=[{"role": "user", "content": f"Context: {context}\n\nQuestion: {question}\n\nAnswer from context only."}]
    )
    return r.choices[0].message.content

env = HallucinationGuardEnv()
results = env.evaluate(groq_model, episodes=5, model_name="llama3-groq")
```

## Optional Dependencies

```bash
pip install openenv-halluguard[openai]      # OpenAI
pip install openenv-halluguard[anthropic]   # Anthropic Claude
pip install openenv-halluguard[groq]        # Groq
```

## Links

- 🤗 Live Environment: https://huggingface.co/spaces/SamSankar/hallucination-guard-env
- 📖 API Docs: https://samsankar-hallucination-guard-env.hf.space/docs
- 📦 OpenEnv: https://github.com/meta-pytorch/OpenEnv

## License

MIT
