Metadata-Version: 2.4
Name: identifiability-diagnostic
Version: 1.0.1
Summary: Enterprise-grade pre-training identifiability diagnostics for regime-switching models
Home-page: https://github.com/yourorg/identifiability-diagnostic
Author: MLOps Team
Author-email: "Prakul S. Hiremath" <prakulhiremath03@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/prakulhiremath/identifiability-diagnostic-framework
Project-URL: Repository, https://github.com/prakulhiremath/identifiability-diagnostic-framework
Project-URL: Bug Tracker, https://github.com/prakulhiremath/identifiability-diagnostic-framework/issues
Keywords: identifiability,regime-switching,state-space,diagnostics,svd,machine-learning
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.22.0
Requires-Dist: scipy>=1.8.0
Requires-Dist: pandas>=1.4.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=3.0.0; extra == "dev"
Requires-Dist: pytest-xdist>=2.5.0; extra == "dev"
Requires-Dist: black>=22.0.0; extra == "dev"
Requires-Dist: flake8>=4.0.0; extra == "dev"
Requires-Dist: isort>=5.10.0; extra == "dev"
Requires-Dist: mypy>=0.950; extra == "dev"
Requires-Dist: pylint>=2.13.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=4.5.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Identifiability Diagnostic Framework

[![CI/CD Pipeline](https://github.com/prakulhiremath/identifiability-diagnostic/actions/workflows/ci.yml/badge.svg)](https://github.com/prakulhiremath/identifiability-diagnostic/actions/workflows/ci.yml)
[![Python Version](https://img.shields.io/badge/python-3.8%2B-blue)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Linting: flake8](https://img.shields.io/badge/linting-flake8-brightgreen.svg)](https://github.com/PyCQA/flake8)
[![PyPI](https://img.shields.io/pypi/v/identifiability-diagnostic)](https://pypi.org/project/identifiability-diagnostic/)

## Overview

An enterprise-grade, production-ready **unsupervised pre-training geometric diagnostic** system for identifying practical identifiability boundaries in **2-regime Switching State-Space Models (S-SSMs)**.

This framework implements a novel algorithm that constructs an **Observable Moment Matrix** from interventional data and applies **Singular Value Decomposition (SVD)** to determine whether regime parameters are identifiable from observations alone.

### Key Features

- 🎯 **SVD-based Identifiability Analysis**: Automatically detects parameter identifiability via spectral metrics
- 🔄 **Regime-Switching Support**: Full support for multi-regime autoregressive processes with exogenous interventions
- 📊 **Comprehensive Metrics**: Effective rank, condition numbers, singular values, deployment decisions
- 🛡️ **Production-Grade**: Full type hints, error handling, logging, and extensive test coverage
- 🚀 **CI/CD Ready**: GitHub Actions workflows, automated testing, code quality checks
- 📦 **Easy Installation**: PyPI-ready package with full dependency management
- 💻 **Cross-Platform**: Runs on Linux, macOS, and Windows

## Installation

### From PyPI
```bash
pip install identifiability-diagnostic
```

### From Source
```bash
git clone https://github.com/yourorg/identifiability-diagnostic.git
cd identifiability-diagnostic
pip install -e .
```

### Development Installation
```bash
pip install -e ".[dev]"
```

## Quick Start

### Basic Usage

```python
from identifiability_diagnostic import (
    RegimeSwitchingGenerator,
    PretrainingDiagnosticPipeline
)
from identifiability_diagnostic.utils.config import load_config

# Load configuration
config = load_config('config/parameters.yaml')

# Generate synthetic data
generator = RegimeSwitchingGenerator(config)
y, u = generator.generate(epsilon=1e-3, symmetric=False)

# Run diagnostic pipeline
pipeline = PretrainingDiagnosticPipeline(config)
metrics = pipeline.run(y, u)

# Interpret results
print(f"σ_2 = {metrics['sigma_2']:.4e}")
print(f"Decision: {metrics['deploy_decision']}")
print(f"Identifiable: {metrics['identifiable']}")
```

### Command-Line Interface

```bash
# Run parametric sweep
python main.py --config config/parameters.yaml --log-level INFO

# Run with custom epsilon values
python main.py --epsilon 1e-5 1e-4 1e-3 1e-2 1e-1

# Test mode (smaller dataset)
python main.py --test --verbose

# Symmetric control only
python main.py --symmetric-only
```

## Algorithm Overview

The framework implements a 7-step diagnostic pipeline:

1. **Compute First-Order Innovations**: dy_t = y_t - y_{t-1}
2. **Lift to Feature Space**: v_t = [dy_t, dy_{t-1}]^T
3. **Align Interventional Context**: Match intervention levels to innovation timeline
4. **Construct Moment Matrix**: M ∈ ℝ^{|U| × 4} with conditional covariances
5. **Execute SVD**: Singular Value Decomposition of M
6. **Compute Metrics**: Effective rank, singular values, condition numbers
7. **Deploy Gate**: σ_2 > τ determines identifiability

### Mathematical Foundation

For regime-switching model:
```
y_t = a_{s_t} y_{t-1} + b_{s_t} u_t + c_{s_t} u_t y_{t-1} + η_t
```

The Observable Moment Matrix captures covariance structure stratified by intervention level. SVD spectrum determines whether regimes can be distinguished from data.

**Key Decision Rule**:
- If σ_2 > threshold (τ = 5.12e-17): **Identifiable** ✓ Deploy estimator
- If σ_2 ≤ threshold: **Non-identifiable** ✗ Abort training

## Project Structure

```
identifiability-diagnostic/
├── .github/workflows/           # CI/CD configurations
├── src/
│   └── identifiability_diagnostic/
│       ├── __init__.py
│       ├── core/                # Core algorithms
│       │   ├── data_generator.py
│       │   ├── pipeline.py
│       │   └── metrics.py
│       ├── utils/               # Utilities
│       │   ├── config.py
│       │   └── logging.py
│       └── exceptions.py
├── tests/
│   ├── unit/                    # Unit tests (20+ tests)
│   └── integration/             # Integration tests (10+ tests)
├── examples/                    # Usage examples
├── config/
│   └── parameters.yaml          # Default configuration
├── main.py                      # CLI entry point
├── setup.py                     # Package setup
├── pyproject.toml              # Modern packaging config
├── requirements.txt            # Dependencies
└── README.md                   # This file
```

## Testing

### Run All Tests
```bash
pytest tests/ -v --cov=src/identifiability_diagnostic
```

### Unit Tests Only
```bash
pytest tests/unit -v
```

### Integration Tests Only
```bash
pytest tests/integration -v
```

### With Coverage Report
```bash
pytest tests/ --cov=src/identifiability_diagnostic --cov-report=html
```

Test suite includes:
- ✓ Data generation tests
- ✓ Pipeline execution tests
- ✓ Metrics computation tests
- ✓ Configuration validation tests
- ✓ End-to-end workflow tests
- ✓ Edge case handling
- ✓ Numerical stability tests

## Configuration

Default configuration in `config/parameters.yaml`:

```yaml
simulation:
  T: 100000              # Time series length
  seed: 42               # Random seed
  sigma_n: 0.5           # Process noise std dev
  regime_0:
    a: 0.90              # AR coefficient regime 0
    b: 1.0               # Intervention coupling
    c: 0.0               # Bilinear term
  regime_1:
    a: 0.90              # AR coefficient regime 1
    b: 1.0
    c: 0.2               # Asymmetric bilinear!

diagnostic:
  u_levels: [-2, -1, 0, 1, 2]     # Intervention grid
  threshold_tau: 5.12e-17         # Gating threshold
```

## Examples

See `examples/basic_usage.py` for detailed examples:

```bash
python examples/basic_usage.py
```

Examples include:
1. Basic workflow (single epsilon)
2. Parametric sweep (multiple epsilon values)
3. Symmetric control test
4. Custom configuration

## API Reference

### RegimeSwitchingGenerator
```python
gen = RegimeSwitchingGenerator(config)
y, u = gen.generate(epsilon=1e-3, symmetric=False, verbose=True)
results = gen.batch_generate([1e-5, 1e-4, 1e-3])
```

### PretrainingDiagnosticPipeline
```python
pipeline = PretrainingDiagnosticPipeline(config)
metrics = pipeline.run(y, u, verbose=True)
batch_results = pipeline.batch_run(data_dict)
```

### MetricsCalculator
```python
calc = MetricsCalculator(threshold_tau=5.12e-17)
metrics = calc.compute_metrics(moment_matrix)
eff_rank = calc.compute_effective_rank(singular_values)
```

## Performance

Benchmark results on Intel i7 @ 3.6GHz:

| Task | Data Size | Time |
|------|-----------|------|
| Data Generation (T=100k) | 100,000 obs | ~50 ms |
| Pipeline Execution | 100,000 obs | ~80 ms |
| Full Diagnostic Cycle | 100,000 obs | ~130 ms |
| Parametric Sweep (5 ε values) | 500,000 obs | ~650 ms |

## Code Quality

- **Coverage**: >85% test coverage
- **Type Hints**: Full type annotations throughout
- **Linting**: flake8, black, isort, pylint
- **Static Analysis**: mypy type checking
- **Documentation**: Comprehensive docstrings and comments

## CI/CD Pipeline

Automated workflows for:
- ✓ Unit & integration tests (Python 3.8-3.11)
- ✓ Multi-platform testing (Ubuntu, macOS, Windows)
- ✓ Code quality checks (flake8, black, mypy, pylint)
- ✓ Security scanning (bandit, safety)
- ✓ Coverage reporting (Codecov)
- ✓ Package building and verification

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## License

MIT License - see [LICENSE](LICENSE) file for details.

## Citation

If you use this framework in research, please cite:

```bibtex
@software{identifiability_diagnostic_2024,
  title={Identifiability Diagnostic Framework for Regime-Switching Models},
  author={Prakul S. Hiremath, Aliens on Earth},
  year={2026},
  url={https://github.com/prakulhiremath/identifiability-diagnostic}
}
```

## Support

- 📚 [Documentation](https://identifiability-diagnostic.readthedocs.io)
- 🐛 [Issue Tracker](https://github.com/prakulhiremath/identifiability-diagnostic/issues)
- 💬 [Discussions](https://github.com/prakulhiremath/identifiability-diagnostic/discussions)

## Team

**[Aliens on Earth](https://aliensonearth.in)**

## Acknowledgments

This framework implements novel identifiability diagnostics for regime-switching state-space models using Observable Moment Matrix analysis and SVD-based metrics.

---
