Metadata-Version: 2.4
Name: ai-assurance-toolkit
Version: 0.1.0
Summary: A Python toolkit for evaluating AI model reliability, performance, and deployment readiness.
Author-email: Happy Iguare <haigu1@morgan.edu>
License: Apache-2.0
Project-URL: Repository, https://github.com/harpiking/AI-Assurance-Toolkit
Keywords: ai assurance,model evaluation,machine learning,responsible ai,model audit,nist ai rmf,public sector ai
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: pandas>=2.0
Requires-Dist: scikit-learn>=1.3
Requires-Dist: joblib>=1.3
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Dynamic: license-file

# AI Assurance Toolkit

AI Assurance Toolkit is a lightweight Python package for evaluating machine learning model reliability, performance, and deployment readiness.

## What it does

The toolkit evaluates a trained machine learning model against a labeled test dataset and generates a structured performance report.

## Metrics included

- Accuracy
- Precision
- Recall
- F1 score
- False positive rate
- False negative rate
- AUC-ROC
- Calibration / Brier score
- Per-class metrics
- Plain-English deployment signal

## Installation

```bash
pip install ai-assurance-toolkit
```

## Quick start

```bash
ai-assurance evaluate \
  --model model.pkl \
  --dataset test_data.csv \
  --target credit_risk \
  --model-name "German Credit Risk Classifier"
```

## Example

Generate a test model and sample dataset:

```bash
python examples/setup_test_model.py
```

Then run the evaluator:

```bash
ai-assurance evaluate \
  --model model.pkl \
  --dataset test_data.csv \
  --target credit_risk \
  --model-name "German Credit Risk Classifier"
```

## Output

The package creates:

```text
module_a_outputs/performance_report.json
```

## Python usage

```python
from ai_assurance_toolkit import run_performance_evaluation
```

## License

Apache License 2.0.
