Metadata-Version: 2.4
Name: mlevalu
Version: 2.0.0
Summary: Zero-code ML Evaluation, Debugging, and Explanation Toolkit
Home-page: https://github.com/your-org/smart_eval
Author: Smart Eval Team
Author-email: Smart Eval Team <contact@smarteval.dev>
License-Expression: MIT
Project-URL: Homepage, https://github.com/your-org/smart_eval
Project-URL: Documentation, https://github.com/your-org/smart_eval#readme
Project-URL: Repository, https://github.com/your-org/smart_eval.git
Project-URL: Issues, https://github.com/your-org/smart_eval/issues
Project-URL: Changelog, https://github.com/your-org/smart_eval/releases
Keywords: machine-learning,ml,evaluation,metrics,debugging,insights,analysis
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.19.0
Requires-Dist: scikit-learn>=0.24.0
Requires-Dist: pandas>=1.1.0
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=2.10.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.9.0; extra == "dev"
Requires-Dist: mypy>=0.900; extra == "dev"
Requires-Dist: twine>=3.4.0; extra == "dev"
Requires-Dist: build>=0.7.0; extra == "dev"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Smart Eval

**Zero-code ML Evaluation, Debugging, and Explanation Toolkit**

A production-ready Python library that makes ML model evaluation, debugging, and interpretation effortless.

Think beyond metrics → auto-evaluation + intelligence + actionable insights.

## 🎯 Features

### ✨ Automatic Task Detection
Detects whether your problem is classification or regression automatically.

### 📊 Comprehensive Metrics
- **Classification**: Accuracy, Precision, Recall, F1-Score, Confusion Matrix, Class Imbalance Detection
- **Regression**: MAE, MSE, RMSE, R² Score, MAPE

### 🧠 Intelligent Insights Engine
Generates human-readable insights that go beyond raw numbers:
- Detects overfitting / underfitting
- Identifies class imbalance issues
- Highlights false positive / false negative patterns
- Suggests actionable improvements

### 🔧 Model Debugging Engine
Comprehensive debugging to catch common issues:
- Missing value detection
- Feature scaling issues
- Overfitting detection (train vs test gap)
- Data quality assessment
- Feature analysis and statistics

### 🎨 Beautiful Terminal Output
Colored, formatted output with:
- Professional report layout
- Confusion matrices
- Color-coded warnings and suggestions
- Clear visual hierarchy

### 💻 CLI Support
Evaluate models directly from the command line:
```bash
smart-eval data.csv --true-col y_true --pred-col y_pred
```

## 🚀 Quick Start

### Installation

```bash
pip install smart-eval
```

Or from source:
```bash
git clone https://github.com/your-org/smart_eval
cd smart_eval
pip install -e .
```

### Basic Usage

```python
import smart_eval

# Classification example
y_true = [0, 1, 1, 0, 1, 1, 0, 0, 1, 0]
y_pred = [0, 1, 0, 0, 1, 1, 0, 0, 1, 0]

# One line evaluation with full insights
results = smart_eval.evaluate(y_true, y_pred)
```

### Output Example

```
======================================================================
  SMART EVALUATION REPORT
======================================================================

Task Information
──────────────
  Task Type........... classification
  Number of Classes... 2
  Is Binary........... True

Performance Metrics
──────────────
  Accuracy.............. 0.8000
  Precision............. 0.8000
  Recall................ 0.8000
  F1 score.............. 0.8000

Summary
──────────────
  Good model performance with room for improvement

Metrics Interpretation
──────────────
  f1_score: Good balance between precision and recall
```

### Regression Example

```python
import smart_eval
import numpy as np

# Regression example
y_true = np.array([3.0, -0.5, 2.0, 7.0])
y_pred = np.array([2.5, 0.0, 2.0, 8.0])

results = smart_eval.evaluate(y_true, y_pred)
# Outputs: MAE, RMSE, R², MAPE and intelligent insights
```

### Model Debugging

```python
import smart_eval
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

# Train model
X = np.random.randn(200, 10)
y = np.random.randint(0, 2, 200)
X_train, X_test, y_train, y_test = train_test_split(X, y)

model = RandomForestClassifier()
model.fit(X_train, y_train)

# Debug the model
debug_report = smart_eval.debug(model, X_train, y_train, X_test, y_test)
# Detects overfitting, scaling issues, missing values, etc.
```

### Getting Only Metrics (No Output)

```python
# Quick evaluation - returns metrics only
metrics = smart_eval.quick_eval(y_true, y_pred)
print(metrics["accuracy"])  # 0.8
```

## 📚 API Reference

### Main Functions

#### `evaluate(y_true, y_pred, verbose=True)`
Main evaluation function with automatic task detection.

**Parameters:**
- `y_true`: Ground truth labels/values
- `y_pred`: Model predictions
- `verbose`: Print colored output (default: True)

**Returns:** Dictionary with task_info, metrics, and insights

#### `debug(model=None, X_train=None, y_train=None, X_test=None, y_test=None, verbose=True)`
Debug model for common issues.

**Parameters:**
- `model`: Trained model (optional)
- `X_train`: Training features
- `y_train`: Training labels
- `X_test`: Test features
- `y_test`: Test labels
- `verbose`: Print output (default: True)

**Returns:** Debug report with warnings and recommendations

#### `quick_eval(y_true, y_pred)`
Quick evaluation returning only metrics without output.

**Returns:** Dictionary of metrics

### Core Classes (Advanced Usage)

```python
from smart_eval.core.detector import TaskDetector
from smart_eval.core.metrics import MetricsCalculator
from smart_eval.core.insights import InsightGenerator
from smart_eval.debugger import ModelDebugger

# Task detection
detector = TaskDetector()
task_info = detector.detect(y_true, y_pred)

# Calculate metrics
calculator = MetricsCalculator()
metrics = calculator.calculate_classification_metrics(y_true, y_pred)

# Generate insights
generator = InsightGenerator()
insights = generator.generate_classification_insights(y_true, y_pred, metrics, num_classes=2, is_binary=True)

# Debug model
debugger = ModelDebugger()
report = debugger.debug(model, X_train, y_train, X_test, y_test)
```

## 🎯 Classification Features

### Automatic Detection
- Binary classification
- Multi-class classification
- Class imbalance warnings

### Metrics
- **Accuracy**: Overall correctness
- **Precision**: False positive rate
- **Recall**: False negative rate
- **F1-Score**: Balance between precision and recall
- **Confusion Matrix**: Per-class performance

### Insights Generated
- ✓ Overfitting detection
- ✓ Class imbalance warnings with severity levels
- ✓ Precision vs Recall analysis
- ✓ Model constant prediction detection
- ✓ Per-class performance hints
- ✓ Dataset size adequacy analysis

## 📈 Regression Features

### Metrics
- **MAE**: Mean Absolute Error
- **MSE**: Mean Squared Error
- **RMSE**: Root Mean Squared Error
- **R² Score**: Coefficient of determination
- **MAPE**: Mean Absolute Percentage Error

### Insights Generated
- ✓ R² score interpretation with severity
- ✓ Negative R² detection (model worse than baseline)
- ✓ Outlier prediction detection
- ✓ Systematic bias detection
- ✓ Constant prediction detection
- ✓ Over/underfitting heuristics

## 🔍 Debugging Features

- ✓ Missing value detection
- ✓ Feature scaling analysis
- ✓ Overfitting/underfitting detection
- ✓ Feature statistics (mean, std, min, max)
- ✓ Constant feature detection
- ✓ High variance feature detection
- ✓ Train/test performance gap analysis

## 💡 Use Cases

1. **Quick Model Validation**: Evaluate models in notebooks instantly
2. **Model Comparison**: Compare different models side-by-side
3. **Debugging**: Identify why models underperform
4. **Production Monitoring**: Monitor model performance over time
5. **Data Quality**: Verify data preprocessing pipeline
6. **Teaching**: Teach ML concepts with immediate visual feedback

## 🛠️ CLI Usage

### Basic Evaluation
```bash
smart-eval predictions.csv --true-col target --pred-col prediction
```

### Save Results to JSON
```bash
smart-eval predictions.csv -o results.json
```

### Quiet Mode (no colors)
```bash
smart-eval predictions.csv --quiet
```

### Custom Column Names
```bash
smart-eval data.csv --true-col ground_truth --pred-col model_output
```

## 📋 CSV Format Example

Your CSV file should have columns for true labels and predictions:

```csv
y_true,y_pred
0,0
1,1
1,0
0,0
1,1
```

## 🔗 Integration Examples

### With scikit-learn
```python
import smart_eval
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y)
model = RandomForestClassifier()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
smart_eval.evaluate(y_test, y_pred)
```

### With XGBoost
```python
import smart_eval
import xgboost as xgb

model = xgb.XGBClassifier()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
smart_eval.evaluate(y_test, y_pred)
```

### With TensorFlow/Keras
```python
import smart_eval
from tensorflow import keras

model = keras.Sequential([...])
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
smart_eval.evaluate(y_test, y_pred)
```

### With PyTorch
```python
import smart_eval
import torch

# ... train model ...
y_pred = model(X_test).argmax(dim=1).numpy()
smart_eval.evaluate(y_test.numpy(), y_pred)
```

## 🎨 Customization

### Disable Colors
```python
from smart_eval.visualizer import ColorCodes
ColorCodes.disable()

smart_eval.evaluate(y_true, y_pred)
```

### Access Raw Data
```python
results = smart_eval.evaluate(y_true, y_pred, verbose=False)

# Access individual components
metrics = results["metrics"]
insights = results["insights"]
task_info = results["task_info"]
```

## 🐛 Troubleshooting

### ImportError: No module named 'smart_eval'
```bash
pip install smart-eval
# or from source:
pip install -e .
```

### "Column not found" in CLI
Check that your column names match your CSV:
```bash
smart-eval data.csv --true-col correct_name --pred-col prediction_name
```

### No insights generated
Ensure you have sufficient data (minimum 10 samples recommended)

## 📝 Requirements

- Python >= 3.7
- numpy >= 1.19.0
- scikit-learn >= 0.24.0
- pandas >= 1.1.0

## 📄 License

MIT License - see LICENSE file for details

## 🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md

## 🙏 Acknowledgments

Built with ❤️ for the ML community

## 📞 Support

- 📧 Email: contact@smarteval.dev
- 🐛 Issues: [GitHub Issues](https://github.com/your-org/smart_eval/issues)
- 💬 Discussions: [GitHub Discussions](https://github.com/your-org/smart_eval/discussions)

---

**Smart Eval** - Making ML evaluation intelligent and effortless ⚡
