Metadata-Version: 2.4
Name: ai-qa-sdk
Version: 0.1.4
Summary: Universal AI validation SDK — hallucination, factuality, safety checks for any LLM output
Project-URL: Homepage, https://github.com/Nagam-Naidu/ai-qa-sdk
Project-URL: Documentation, https://nagam-naidu.github.io/ai-qa-sdk
Project-URL: Repository, https://github.com/Nagam-Naidu/ai-qa-sdk
Project-URL: Issues, https://github.com/Nagam-Naidu/ai-qa-sdk/issues
Project-URL: Changelog, https://github.com/Nagam-Naidu/ai-qa-sdk/blob/main/CHANGELOG.md
Author-email: Nagam Naidu <jdnaidu15@gmail.com>
License: Apache-2.0
Keywords: agents,ai,evaluation,hallucination,llm,qa,rag,testing,validation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: click<9.0,>=8.0
Requires-Dist: httpx<1.0,>=0.27
Requires-Dist: jsonschema<5.0,>=4.0
Requires-Dist: pydantic<3.0,>=2.0
Requires-Dist: pyyaml<7.0,>=6.0
Requires-Dist: rich<14.0,>=13.0
Provides-Extra: all
Requires-Dist: anthropic>=0.40; extra == 'all'
Requires-Dist: langsmith>=0.1; extra == 'all'
Requires-Dist: numpy>=1.26; extra == 'all'
Requires-Dist: openai>=1.50; extra == 'all'
Provides-Extra: all-providers
Requires-Dist: anthropic>=0.40; extra == 'all-providers'
Requires-Dist: openai>=1.50; extra == 'all-providers'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff>=0.8; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
Requires-Dist: mkdocstrings[python]>=0.25; extra == 'docs'
Provides-Extra: langsmith
Requires-Dist: langsmith>=0.1; extra == 'langsmith'
Provides-Extra: local
Provides-Extra: numpy
Requires-Dist: numpy>=1.26; extra == 'numpy'
Provides-Extra: openai
Requires-Dist: openai>=1.50; extra == 'openai'
Description-Content-Type: text/markdown

# ai-qa-sdk

Python SDK for validating AI outputs.

## Installation

```bash
pip install ai-qa-sdk
```

## Quick Start

### 1. Without API Key (Deterministic Checks)

```python
from ai_qa import validate

# Validate JSON format (no API key needed)
result = validate(
    output='{"name": "John", "age": 30}',
    checks=["json_format"]
)

print(f"Passed: {result.passed}")
for check in result.checks:
    print(f"  {check.name}: {check.score:.2f}")
```

### 2. With API Key (LLM-Powered Checks)

**Install with provider support:**
```bash
# For Anthropic (Claude)
pip install ai-qa-sdk[anthropic]

# For OpenAI (GPT)
pip install ai-qa-sdk[openai]

# For all providers
pip install ai-qa-sdk[all-providers]
```

**Configure and validate:**
```python
from ai_qa import configure, validate

# Configure Anthropic (Claude)
configure(provider="anthropic", api_key="sk-ant-...")

# Or configure OpenAI
# configure(provider="openai", api_key="sk-...")

# Now use LLM-powered checks
result = validate(
    output="The capital of France is Paris",
    input="What is the capital of France?",
    checks=["factuality", "relevance", "hallucination"]
)

print(f"Passed: {result.passed}")
print(f"Cost: ${result.total_cost:.4f}")

for check in result.checks:
    print(f"  {check.name}: {check.score:.2f}")
```

## Available Checks

### ✓ No API Key Required
- `json_format` - Validate JSON syntax
- `regex` - Pattern matching
- `schema` - JSON schema validation
- `length` - Output length bounds

### 🔑 Requires API Key (Anthropic or OpenAI)
- `hallucination` - Detect made-up facts
- `factuality` - Check factual accuracy
- `relevance` - Relevance to input
- `toxicity` - Harmful content detection
- `bias` - Bias/fairness issues
- `pii` - Personal information detection
- `consistency` - Internal consistency
- `contradiction` - Logical contradictions
- `groundedness` - Grounded in provided context
- And more!

## Getting API Keys

**Anthropic (Claude):**
- Visit: https://console.anthropic.com
- Create account → API Keys → Create new key
- Format: `sk-ant-...`

**OpenAI (GPT):**
- Visit: https://platform.openai.com/api-keys
- Create account → API Keys → Create new key
- Format: `sk-...`
