Metadata-Version: 2.4
Name: ai-qa-sdk
Version: 0.2.0
Summary: Universal AI validation SDK — hallucination, factuality, safety checks for any LLM output
Project-URL: Homepage, https://github.com/Nagam-Naidu/ai-qa-sdk
Project-URL: Documentation, https://nagam-naidu.github.io/ai-qa-sdk
Project-URL: Repository, https://github.com/Nagam-Naidu/ai-qa-sdk
Project-URL: Issues, https://github.com/Nagam-Naidu/ai-qa-sdk/issues
Project-URL: Changelog, https://github.com/Nagam-Naidu/ai-qa-sdk/blob/main/CHANGELOG.md
Author-email: Nagam Naidu <jdnaidu15@gmail.com>
License: Apache-2.0
Keywords: agents,ai,evaluation,hallucination,llm,qa,rag,testing,validation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: anthropic>=0.40
Requires-Dist: click<9.0,>=8.0
Requires-Dist: httpx<1.0,>=0.27
Requires-Dist: jsonschema<5.0,>=4.0
Requires-Dist: openai>=1.50
Requires-Dist: pydantic<3.0,>=2.0
Requires-Dist: pyyaml<7.0,>=6.0
Requires-Dist: rich<14.0,>=13.0
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=8.0; extra == 'dev'
Requires-Dist: twine>=4.0; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25; extra == 'docs'
Description-Content-Type: text/markdown

# ai-qa-sdk

Python SDK for validating AI outputs.

## Installation

```bash
pip install ai-qa-sdk
```

This installs everything you need including support for Anthropic (Claude) and OpenAI (GPT).

## Quick Start

### 1. Deterministic Checks (No API Key Needed)

```python
from ai_qa import validate

# Validate JSON format - works immediately!
result = validate(
    output='{"name": "John", "age": 30}',
    checks=["json_format"]
)

print(f"Passed: {result.passed}")
for check in result.checks:
    print(f"  {check.name}: {check.score:.2f}")
```

### 2. LLM-Powered Checks (Configure Your Provider)

**Get an API key first:**
- **Anthropic (Claude):** https://console.anthropic.com → API Keys
- **OpenAI (GPT):** https://platform.openai.com/api-keys

**Then configure and use:**
```python
from ai_qa import configure, validate

# Configure your preferred provider
configure(provider="anthropic", api_key="sk-ant-YOUR-KEY-HERE")

# Or use OpenAI instead
# configure(provider="openai", api_key="sk-YOUR-KEY-HERE")

# Now you can use any LLM-powered check
result = validate(
    output="The capital of France is Paris",
    input="What is the capital of France?",
    checks=["factuality", "relevance", "hallucination"]
)

print(f"Passed: {result.passed}")
print(f"Cost: ${result.total_cost:.4f}")

for check in result.checks:
    print(f"  {check.name}: {check.score:.2f}")
```

## Available Checks

### ✓ No API Key Required
- `json_format` - Validate JSON syntax
- `regex` - Pattern matching
- `schema` - JSON schema validation
- `length` - Output length bounds

### 🔑 Requires API Key (Anthropic or OpenAI)
- `hallucination` - Detect made-up facts
- `factuality` - Check factual accuracy
- `relevance` - Relevance to input
- `toxicity` - Harmful content detection
- `bias` - Bias/fairness issues
- `pii` - Personal information detection
- `consistency` - Internal consistency
- `contradiction` - Logical contradictions
- `groundedness` - Grounded in provided context
- And more!

## Providers

ai-qa-sdk supports multiple providers out of the box:

| Provider | Setup | Cost |
|----------|-------|------|
| **Anthropic (Claude)** | `configure(provider="anthropic", api_key="...")` | ~$0.003-0.015 per 1K tokens |
| **OpenAI (GPT)** | `configure(provider="openai", api_key="...")` | ~$0.03-0.06 per 1K tokens |
| **Local (Ollama)** | `configure(provider="local", base_url="http://localhost:11434")` | Free (runs locally) |

**Get API keys:**
- Anthropic: https://console.anthropic.com/api-keys
- OpenAI: https://platform.openai.com/api-keys
