scitex_ml.classification.reporters.reporter_utils.validation

Validation utilities for classification metrics.

Provides validation for completeness, consistency, and scientific requirements.

Functions

check_required_metrics(metrics_dict, required)	Check if all required metrics are present.
validate_completeness(output_dir, ...)	Validate completeness of saved metrics.

Classes

MetricValidator(required_metrics)

Validates classification metrics for completeness and consistency.

class scitex_ml.classification.reporters.reporter_utils.validation.MetricValidator(required_metrics)

Validates classification metrics for completeness and consistency.

This class checks that all required metrics are present across folds and validates metric values are within expected ranges.

METRIC_RANGES = {'accuracy': (0.0, 1.0), 'balanced_accuracy': (0.0, 1.0), 'f1': (0.0, 1.0), 'mcc': (-1.0, 1.0), 'pr_auc': (0.0, 1.0), 'precision': (0.0, 1.0), 'recall': (0.0, 1.0), 'roc_auc': (0.0, 1.0)}

__init__(required_metrics)

Initialize validator with required metrics.

Parameters:

required_metrics (List[str]) – List of metric names that must be present

validate_metric_value(metric_name, value)

Validate a single metric value is within expected range.

Parameters:

• metric_name (str) – Name of the metric

• value (Union[float, np.ndarray]) – Metric value to validate

Returns:

(is_valid, error_message)

Return type:

Tuple[bool, Optional[str]]

validate_fold(fold_data, fold)

Validate metrics for a single fold.

Parameters:

• fold_data (Dict[str, Any]) – Dictionary of metrics for the fold

• fold (int) – Fold index

Returns:

Validation results for the fold

Return type:

Dict[str, Any]

validate_all_folds(folds_data)

Validate metrics across all folds.

Parameters:

folds_data (List[Dict[str, Any]]) – List of metric dictionaries for each fold

Returns:

Complete validation report

Return type:

Dict[str, Any]

save_report(path)

Save validation report to file.

Parameters:

path (Union[str, Path]) – Path to save report

Return type:

None

print_summary()

Print validation summary to console.

Return type:

None

scitex_ml.classification.reporters.reporter_utils.validation.validate_completeness(output_dir, required_metrics, n_folds)

Validate completeness of saved metrics.

Parameters:

• output_dir (Union[str, Path]) – Directory containing saved metrics

• required_metrics (List[str]) – List of required metric names

• n_folds (int) – Expected number of folds

Returns:

Validation report

Return type:

Dict[str, Any]

Examples

>>> report = validate_completeness(
...     "./results",
...     ['balanced_accuracy', 'mcc', 'confusion_matrix'],
...     n_folds=5
... )
>>> if report['complete']:
...     print("All metrics present!")

scitex_ml.classification.reporters.reporter_utils.validation.check_required_metrics(metrics_dict, required)

Check if all required metrics are present.

Parameters:

• metrics_dict (Dict[str, Any]) – Dictionary of available metrics

• required (List[str]) – List of required metric names

Returns:

(all_present, missing_metrics)

Return type:

Tuple[bool, List[str]]

Examples

>>> metrics = {'balanced_accuracy': 0.85, 'mcc': 0.7}
>>> complete, missing = check_required_metrics(
...     metrics,
...     ['balanced_accuracy', 'mcc', 'roc_auc']
... )
>>> print(f"Missing: {missing}")  # ['roc_auc']