Metadata-Version: 2.4
Name: eval-case-curator
Version: 0.1.0
Summary: Local CLI and desktop app for turning AI failure notes into reusable eval cases
Author: Haider Sattar
License-Expression: MIT
Keywords: evals,llm,ai,testing,jsonl,quality
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Text Processing
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

![Terminal report preview](docs/terminal-report.svg)

# Eval Case Curator

Eval Case Curator turns messy AI failure notes, support escalations, prompt review comments, and QA findings into reusable eval cases.

It runs locally, redacts common contact/credential patterns, and exports Markdown, JSON, JSONL, or HTML reports that can be dropped into prompt regression suites or review notes.

## Install

```bash
pip install eval-case-curator
```

## CLI

```bash
eval-case-curator examples/failure-notes.txt
eval-case-curator examples/failure-notes.txt --format jsonl --output evals.jsonl
eval-case-curator examples/failure-notes.txt --fail-under 3
```

## Desktop App

```bash
eval-case-curator-gui
```

Paste notes, curate cases, copy JSONL, or save a Markdown report.

## What It Extracts

- Candidate eval input from `Input:`, `Prompt:`, `User:`, or fallback context.
- Expected behavior from `Expected:`, `Should:`, `Desired:`, or failure wording.
- Priority from safety, privacy, financial, legal, hallucination, format, and missing-answer clues.
- Tags such as `privacy`, `format`, `retrieval`, `instruction-following`, `tool-use`, and `tone`.
- Source line references and risk notes for review traceability.

## Example

```bash
eval-case-curator examples/failure-notes.txt --format jsonl
```

```json
{"id":"eval-...","input":"Summarize the renewal risk...","expected":"Mention renewal risk without exposing direct contact details.","metadata":{"priority":"high","tags":["privacy"]}}
```

## Development

```bash
PYTHONPATH=src python3 -m unittest discover -s tests
python3 -m build --no-isolation
python3 -m twine check dist/*
```

## License

MIT
