Metadata-Version: 2.4
Name: modeldoctor
Version: 0.1.0
Summary: A production-grade ML model diagnostic framework — behaves like a senior ML engineer reviewing your model
Project-URL: Homepage, https://github.com/modeldoctor/modeldoctor
Project-URL: Documentation, https://modeldoctor.readthedocs.io
Project-URL: Repository, https://github.com/modeldoctor/modeldoctor
Project-URL: Issues, https://github.com/modeldoctor/modeldoctor/issues
Project-URL: Changelog, https://github.com/modeldoctor/modeldoctor/blob/main/CHANGELOG.md
Author: ModelDoctor Contributors
License: MIT
Keywords: diagnostics,explainability,machine-learning,mlops,model-evaluation,model-health
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Typing :: Typed
Requires-Python: >=3.9
Requires-Dist: jinja2>=3.0
Requires-Dist: numpy>=1.21
Requires-Dist: pandas>=1.3
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: scikit-learn>=1.1
Requires-Dist: scipy>=1.7
Requires-Dist: typing-extensions>=4.5; python_version < '3.11'
Provides-Extra: ai
Requires-Dist: anthropic>=0.20; extra == 'ai'
Requires-Dist: openai>=1.0; extra == 'ai'
Provides-Extra: all
Requires-Dist: anthropic>=0.20; extra == 'all'
Requires-Dist: lime>=0.2; extra == 'all'
Requires-Dist: openai>=1.0; extra == 'all'
Requires-Dist: plotly>=5.0; extra == 'all'
Requires-Dist: shap>=0.42; extra == 'all'
Provides-Extra: deep
Requires-Dist: torch>=2.0; extra == 'deep'
Provides-Extra: dev
Requires-Dist: hypothesis>=6.0; extra == 'dev'
Requires-Dist: lightgbm>=4.0; extra == 'dev'
Requires-Dist: mypy>=1.5; extra == 'dev'
Requires-Dist: pre-commit>=3.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest-mock>=3.11; extra == 'dev'
Requires-Dist: pytest>=7.4; extra == 'dev'
Requires-Dist: ruff>=0.1; extra == 'dev'
Requires-Dist: xgboost>=2.0; extra == 'dev'
Provides-Extra: explainability
Requires-Dist: lime>=0.2; extra == 'explainability'
Requires-Dist: shap>=0.42; extra == 'explainability'
Provides-Extra: report
Requires-Dist: plotly>=5.0; extra == 'report'
Description-Content-Type: text/markdown

<div align="center">

<img src="docs/images/logoy.png" alt="ModelDoctor Logo" width="400" />

**Diagnose your machine learning models like a senior ML engineer.**

[![PyPI](https://img.shields.io/pypi/v/modeldoctor?style=flat-square&color=4f46e5)](https://pypi.org/project/modeldoctor)
[![Python](https://img.shields.io/badge/python-3.9%20%7C%203.10%20%7C%203.11%20%7C%203.12-blue?style=flat-square)](https://www.python.org)
[![License: MIT](https://img.shields.io/badge/license-MIT-green?style=flat-square)](https://github.com/modeldoctor/modeldoctor/blob/main/LICENSE)
[![Downloads](https://img.shields.io/pypi/dm/modeldoctor?style=flat-square&color=0ea5e9)](https://pypi.org/project/modeldoctor)
[![Documentation](https://img.shields.io/badge/docs-available-brightgreen?style=flat-square)](https://modeldoctor.readthedocs.io)
[![CI](https://img.shields.io/github/actions/workflow/status/modeldoctor/modeldoctor/tests.yml?branch=main&label=tests&style=flat-square)](https://github.com/modeldoctor/modeldoctor/actions)

</div>

<br />

<div align="center">
  <img src="docs/images/dashboard.png" alt="ModelDoctor Dashboard" width="100%" />
</div>

## Why ModelDoctor?

Traditional machine learning evaluation focuses almost entirely on aggregate metrics like accuracy, precision, and recall. While these numbers indicate how well a model performs on a specific dataset, they rarely explain *why* it behaves that way or whether the model will actually survive in a production environment. 

ModelDoctor evaluates models holistically. It runs a comprehensive suite of diagnostic checks to identify hidden problems—from subtle data leakage and overfitting to calibration errors and inference latency. 

Instead of leaving you to interpret raw numbers, ModelDoctor explains what issues exist, why they are problematic, and exactly how to fix them before deployment.

## Quick Example

```python
import modeldoctor as md

report = md.diagnose(
    model,
    X_train,
    y_train,
    X_test,
    y_test,
)

report.show()
```

<img src="docs/images/dreportcard.png" alt="ModelDoctor Generated Report" width="100%" />

## Installation

Install the core library:

```bash
pip install modeldoctor
```

You can optionally install extensions for specific features:

```bash
pip install modeldoctor[dashboard]
pip install modeldoctor[shap]
pip install modeldoctor[all]
```

## Features

### Diagnostics
- Overfitting
- Data Leakage
- Calibration
- Prediction Quality
- Feature Analysis
- Data Quality
- Generalization
- Production Readiness

### Reporting
- HTML Dashboard
- Markdown
- JSON
- PDF
- CLI
- MLflow

### Explainability
- SHAP
- Feature Importance
- Evidence Engine
- Confidence Engine
- Prescription Engine

## Example Output

**Overall Health**: 82/100 (Needs Review)

**Diagnosis**: Potential Data Leakage Detected (Critical)
The top feature `customer_id` accounts for 98% of the total feature importance and has a nearly perfect correlation (0.99) with the target variable.

**Prescription**: Remove `customer_id` from the training dataset.

**Recommendation**: Retrain the model and ensure no future information or unique identifiers are included in the feature set.

## Architecture

<div align="center">
  <img src="docs/images/architecturediagram.png" alt="ModelDoctor Architecture Diagram" width="100%" />
</div>

The evaluation process begins by wrapping your model and dataset into an `EvaluationContext`, computing metrics lazily only as needed. Specialized `Doctors` then analyze this context across different dimensions, extracting concrete `Evidence`. The `Confidence` and `Risk` engines synthesize this evidence into prioritized `Prescriptions`, ultimately rendering them into a standardized `Report`.

## Validation

ModelDoctor is continuously verified against the Validation Laboratory, an independent framework designed to stress-test the diagnostic engine across real-world edge cases. 

| Metric | Result |
|---|---|
| Validation Scenarios | 54 |
| Diagnostic Accuracy | 98.1% |
| Supported Models | Scikit-learn |
| Validation Framework | Included |

## Dashboard

<div align="center">
  <img src="docs/images/dashboard.png" alt="ModelDoctor Interactive Dashboard" width="100%" />
</div>

The HTML dashboard provides an interactive interface to explore your model's health score, drill down into specific diagnostic charts, and review prioritized recommendations. The report is fully self-contained, searchable, and easy to export and share with stakeholders.

## Documentation

- [Installation](docs/getting-started/installation.md)
- [Quick Start](docs/getting-started/quickstart.md)
- [API Reference](docs/api/reference.md)
- [Dashboard Guide](docs/getting-started/dashboard.md)
- [Validation Laboratory](docs/validation/README.md)
- [Examples](docs/examples/basic.md)

## Examples

- [`classification.py`](examples/classification.py) — Diagnose a standard binary classifier.
- [`regression.py`](examples/regression.py) — Evaluate a regressor's prediction quality.
- [`dashboard.py`](examples/dashboard.py) — Generate and serve an interactive HTML dashboard.
- [`html_report.py`](examples/html_report.py) — Export static reports for CI/CD pipelines.
- [`custom_doctor.py`](examples/custom_doctor.py) — Build and register a custom diagnostic rule.

## Roadmap

- [x] Core diagnostics
- [x] Validation laboratory
- [x] Dashboard
- [ ] CLI
- [ ] MLflow integration
- [ ] Regression support enhancements

**Future**
- PyTorch support
- TensorFlow support
- CatBoost enhancements

## Contributing

Contributions are welcome. Please refer to our [CONTRIBUTING.md](docs/contributing.md) for details.

1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Open a Pull Request

## Citation

If you use ModelDoctor in academic research, please cite it:

```bibtex
@software{modeldoctor,
  title = {ModelDoctor: Clinical Diagnostics for Machine Learning Models},
  author = {ModelDoctor Contributors},
  year = {2026},
  url = {https://github.com/modeldoctor/modeldoctor}
}
```

## Security

Please report security vulnerabilities according to the guidelines in [SECURITY.md](SECURITY.md).

Do **not** report security vulnerabilities through public GitHub Issues.

## License

This project is licensed under the [MIT License](LICENSE).

---

Built by the ModelDoctor community.
