Metadata-Version: 2.4
Name: waifmark
Version: 2.0.0
Summary: Benchmarking suite for evaluating agentic and roleplay capabilities of small VLLMs
Author: Waifmark Contributors
License-Expression: MIT
Keywords: benchmark,llm,agentic,roleplay,evaluation
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: huggingface_hub>=0.24.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: fastapi>=0.110.0
Requires-Dist: uvicorn[standard]>=0.29.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: black>=23.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.0; extra == "dev"
Requires-Dist: httpx>=0.27.0; extra == "dev"
Provides-Extra: build
Requires-Dist: build>=1.0.0; extra == "build"
Requires-Dist: pyinstaller>=6.0; extra == "build"
Dynamic: license-file

# WAIFMARK© 2 Benchmarking Suite

This directory is the Waifmark 2 benchmark suite. See the [root README](../README.md) for quickstart, scoring, and security notes.

The exact test bank is proprietary and is not committed; run with `data/test_bank.example.json` or provide your own `data/test_bank.json`.

```bash
# One-command boot (new default)
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env  # add OPENROUTER_API_KEY / HF_TOKEN if you have them
waifmark              # or: python -m api.run
# → http://127.0.0.1:8001  +  http://127.0.0.1:8001/docs  +  http://127.0.0.1:8001/chart
```

Headless: `python main.py --config config.yaml --test-bank data/test_bank.json`

Judge v2: calibrated prompt with anchor 0-100, confidence-weighted aggregation, heuristic fallback, low-confidence triage (<0.55).
