Metadata-Version: 2.5
Name: ml-inspector-mcp
Version: 0.1.0
Summary: Framework-agnostic ML model analysis MCP server — evaluate, explain, and report on any trained model via Claude
Project-URL: Homepage, https://github.com/YOUR_USERNAME/ml-inspector-mcp
Project-URL: Repository, https://github.com/YOUR_USERNAME/ml-inspector-mcp
License: MIT
Keywords: claude,explainability,machine-learning,mcp,mlops,model-evaluation,shap
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: anyio
Requires-Dist: fastapi
Requires-Dist: httpx
Requires-Dist: jinja2
Requires-Dist: joblib
Requires-Dist: matplotlib
Requires-Dist: mcp<2.0.0,>=1.0.0
Requires-Dist: numpy
Requires-Dist: onnx>=1.14.0
Requires-Dist: onnxruntime>=1.16.0
Requires-Dist: packaging
Requires-Dist: pandas
Requires-Dist: pillow
Requires-Dist: pydantic>=2.0
Requires-Dist: scikit-learn
Requires-Dist: scipy
Requires-Dist: seaborn
Requires-Dist: uvicorn
Provides-Extra: dev
Requires-Dist: mypy>=1.10.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Provides-Extra: drift
Requires-Dist: evidently; extra == 'drift'
Provides-Extra: explain
Requires-Dist: shap; extra == 'explain'
Provides-Extra: full
Requires-Dist: anthropic; extra == 'full'
Requires-Dist: evidently; extra == 'full'
Requires-Dist: plotly; extra == 'full'
Requires-Dist: shap; extra == 'full'
Requires-Dist: skl2onnx; extra == 'full'
Requires-Dist: tensorflow>=2.13; extra == 'full'
Requires-Dist: tf2onnx>=1.15; extra == 'full'
Requires-Dist: torch; extra == 'full'
Requires-Dist: torchvision; extra == 'full'
Requires-Dist: weasyprint; extra == 'full'
Provides-Extra: pytorch
Requires-Dist: torch; extra == 'pytorch'
Requires-Dist: torchvision; extra == 'pytorch'
Provides-Extra: reports
Requires-Dist: anthropic; extra == 'reports'
Requires-Dist: weasyprint; extra == 'reports'
Provides-Extra: sklearn-onnx
Requires-Dist: skl2onnx; extra == 'sklearn-onnx'
Provides-Extra: tensorflow
Requires-Dist: tensorflow>=2.13; extra == 'tensorflow'
Requires-Dist: tf2onnx>=1.15; extra == 'tensorflow'
Description-Content-Type: text/markdown

# ml-inspector-mcp

[![PyPI](https://img.shields.io/pypi/v/ml-inspector-mcp)](https://pypi.org/project/ml-inspector-mcp/)
[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

Framework-agnostic ML model analysis MCP server. Drop in any trained model
and test data — Claude evaluates it, explains predictions, detects drift,
and generates PDF reports via natural language.

## Installation

```bash
pip install ml-inspector-mcp                                          # minimal
pip install "ml-inspector-mcp[full]"                                  # everything
pip install "ml-inspector-mcp[sklearn-onnx,explain,reports]"          # common combo
```

## Claude Desktop setup

Add to `~/Library/Application Support/Claude/claude_desktop_config.json` (Mac):

```json
{
  "mcpServers": {
    "ml-inspector": {
      "command": "ml-inspector",
      "env": {
        "ANTHROPIC_API_KEY": "your-key-here",
        "MLFLOW_TRACKING_URI": "http://localhost:5000"
      }
    }
  }
}
```

## Quick start

```bash
python examples/train_demo_model.py
```

Then in Claude Desktop:
> "Load the demo model from examples/demo_model.onnx"
> "Load test data from examples/demo_test.csv"
> "Evaluate the model and tell me how it's performing"
> "Explain what drove the prediction for sample 5"
> "Generate a PDF evaluation report"

## Model compatibility

| Format | Framework | Install |
|--------|-----------|---------|
| `.onnx` | Any | Always works — recommended |
| `.pkl` / `.joblib` | scikit-learn | `pip install "ml-inspector-mcp[sklearn-onnx]"` |
| `.h5` / `.keras` | TensorFlow/Keras | `pip install "ml-inspector-mcp[tensorflow]"` |
| `.pt` / `.pth` | PyTorch (full model only) | `pip install "ml-inspector-mcp[pytorch]"` |

### Version mismatch fix

If you get version errors loading a `.pkl` or `.pt` file, export to ONNX first:

```bash
# scikit-learn — use the convert_to_onnx tool after loading, or:
python -c "
import joblib
from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType
model = joblib.load('model.pkl')
onnx_model = convert_sklearn(model, initial_types=[('input', FloatTensorType([None, N_FEATURES]))])
open('model.onnx', 'wb').write(onnx_model.SerializeToString())
"

# PyTorch
torch.onnx.export(model, dummy_input, "model.onnx", opset_version=17)

# TensorFlow / Keras
python -m tf2onnx.convert --keras model.h5 --output model.onnx
```

## All tools

| Tool | Description |
|------|-------------|
| `load_model` | Load any model file (`.pkl`, `.h5`, `.pt`, `.onnx`) — auto-detects framework |
| `get_model_info` | Info about the currently loaded model |
| `convert_to_onnx` | Convert loaded model to ONNX format |
| `list_supported_formats` | Show all supported formats and install instructions |
| `load_test_data` | Load a CSV as test dataset |
| `evaluate_model` | Full evaluation — accuracy, F1, AUC, confusion matrix, per-class metrics |
| `find_worst_predictions` | Find samples the model struggled most with |
| `evaluate_by_slice` | Evaluate on a data subset (e.g. by group or label) |
| `threshold_analysis` | Sweep decision threshold — precision/recall/F1/FPR trade-offs |
| `explain_prediction` | SHAP explanation for a single sample |
| `global_feature_importance` | Mean absolute SHAP values across all samples |
| `plot_shap_summary` | SHAP beeswarm summary plot saved as PNG |
| `data_quality_report` | Null counts, class imbalance, outliers, data type warnings |
| `detect_drift` | Statistical drift detection between two datasets (Evidently) |
| `plot_confusion_matrix` | Confusion matrix heatmap (raw + normalized) saved as PNG |
| `plot_roc_curve` | ROC curve with per-class AUC scores saved as PNG |
| `generate_report` | Full PDF / HTML / Markdown report with metrics, charts, and optional AI narrative |

## License

MIT
