Metadata-Version: 2.4
Name: theorion-ai
Version: 0.1.0
Summary: Navigate large model training and inference with confidence
Author: Orion-ML Contributors
License: Apache-2.0
Project-URL: Homepage, https://github.com/orion-ml/orion
Project-URL: Documentation, https://github.com/orion-ml/orion#readme
Project-URL: Repository, https://github.com/orion-ml/orion
Keywords: ml,llm,vlm,training,inference,observability,distributed,pytorch
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.21
Requires-Dist: rich>=13.0
Requires-Dist: psutil>=5.9
Provides-Extra: torch
Requires-Dist: torch>=2.2; extra == "torch"
Provides-Extra: gpu
Requires-Dist: nvidia-ml-py>=12.535.133; extra == "gpu"
Provides-Extra: trackers
Requires-Dist: wandb>=0.15; extra == "trackers"
Requires-Dist: mlflow>=2.0; extra == "trackers"
Provides-Extra: hf
Requires-Dist: transformers>=4.30; extra == "hf"
Requires-Dist: accelerate>=0.20; extra == "hf"
Provides-Extra: lightning
Requires-Dist: lightning>=2.0; extra == "lightning"
Provides-Extra: all
Requires-Dist: torch>=2.2; extra == "all"
Requires-Dist: nvidia-ml-py>=12.535.133; extra == "all"
Requires-Dist: wandb>=0.15; extra == "all"
Requires-Dist: mlflow>=2.0; extra == "all"
Requires-Dist: transformers>=4.30; extra == "all"
Requires-Dist: accelerate>=0.20; extra == "all"
Requires-Dist: lightning>=2.0; extra == "all"
Dynamic: license-file

# Orion-ML

**Navigate large model training and inference with confidence.**

Diagnose bottlenecks. Get ranked optimization strategies. Understand cost-accuracy trade-offs. Works with PyTorch-centric stacks and common trainers. No infrastructure changes required.

Just as navigators used Orion to find their position and chart their course across open ocean, Orion-ML tells you where training or serving is bottlenecked and charts ranked strategies to improve throughput, memory, and cost.

- **PyPI:** `orion-ml`
- **Tagline:** Navigate large model training and inference with confidence
- **Users:** ML engineers and platform teams training LLMs and VLMs on multi-GPU and multi-node clusters

## Components

| Component | Import | Purpose |
|-----------|--------|---------|
| **orion-train** | `orion.train` | Training pipeline metrics, bottleneck classification, parallelism advice, checkpointing, cost reasoning |
| **orion-serve** | `orion.serve` | Inference profiling, quantization guidance, serving and KV-cache analysis |
| **orion-sight** | `orion.sight` | Unified metrics, anomaly detection, rich/HTML/JSON/Markdown reporting, dashboards |

## Install

```bash
pip install orion-ml
# Optional: PyTorch, GPU telemetry, experiment trackers, HF Trainer, Lightning
pip install "orion-ml[torch,gpu,trackers,hf,lightning]"
```

## Quick start: three lines in your training loop

```python
from orion.train import TrainingTracker

tracker = TrainingTracker(job_name="my-run", output_dir="./orion_output")
with tracker.step():
    loss = model(inputs)
    loss.backward()
    optimizer.step()
```

Metrics are written to local SQLite first; optional W&B / MLflow export runs asynchronously and never blocks the step.

## Hugging Face Trainer

```python
from transformers import Trainer
from orion.train.callbacks import OrionTrainCallback

trainer = Trainer(..., callbacks=[OrionTrainCallback(job_name="hf-run", output_dir="./orion_output")])
```

## Parallelism recommendation

```python
from orion.train import ParallelismAdvisor, ParallelismConfig

advisor = ParallelismAdvisor()
rec = advisor.recommend(
    ParallelismConfig(
        model_params=7_000_000_000,
        gpu_memory_gb=80,
        num_gpus=8,
        num_nodes=1,
        interconnect="nvlink",
        precision="bf16",
        target_batch_size=64,
    )
)
print(rec.strategy, rec.reasoning)
```

## Inference profiling

```python
from orion.serve import InferenceProfiler

profiler = InferenceProfiler(model=model, tokenizer=tokenizer)
results = profiler.benchmark(prompts=test_prompts, max_new_tokens=256, batch_sizes=[1, 8, 32])
profiler.report()
```

## Reports

```python
from orion.sight import Reporter

reporter = Reporter(tracker)
reporter.report()  # terminal (rich)
reporter.report(format="html", output_path="report.html")
reporter.report(format="json", output_path="report.json")
```

## Repository layout

```
orion/
├── orion/           # Python package
├── docs/            # Guides and API reference
├── examples/        # Runnable demos (may require torch/GPU)
└── tests/           # unit / integration / benchmarks
```

Output:
sheetal@Sheetal-MBP project-orion % python gate3_real_bottleneck.py

==================================================
GATE 3: REAL BOTTLENECK DETECTION
==================================================

[1/2] Creating real data loader bottleneck...
[transformers] Disabling PyTorch because PyTorch >= 2.4 is required but found 2.2.2
[transformers] PyTorch was not found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
[Orion-ML] Warning: PyTorch < 2.4 detected; some packages recommend torch>=2.4. Core TrainingTracker metrics are unchanged.
╭────── Orion-ML Training summary ──────╮
│ Primary bottleneck: DATA_LOADER_BOUND │
│ Efficiency score: 25/100              │
│ Anomalies flagged: 0                  │
╰───────────────────────────────────────╯
                Top recommendations                 
┏━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Rank ┃ Strategy                  ┃ Accuracy risk ┃
┡━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ 1    │ increase_num_workers      │ NONE          │
│ 2    │ pin_memory                │ NONE          │
│ 3    │ prefetch_factor           │ NONE          │
│ 4    │ streaming_dataset_or_dali │ LOW           │
└──────┴───────────────────────────┴───────────────┘
  Data loader overhead: 92.0%
  ✅ Real data loader bottleneck correctly detected

[2/2] Creating real gradient explosion...
[Orion-ML] Warning: PyTorch < 2.4 detected; some packages recommend torch>=2.4. Core TrainingTracker metrics are unchanged.
╭────── Orion-ML Training summary ──────╮
│ Primary bottleneck: DATA_LOADER_BOUND │
│ Efficiency score: 25/100              │
│ Anomalies flagged: 1                  │
╰───────────────────────────────────────╯
                Top recommendations                 
┏━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Rank ┃ Strategy                  ┃ Accuracy risk ┃
┡━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ 1    │ increase_num_workers      │ NONE          │
│ 2    │ pin_memory                │ NONE          │
│ 3    │ prefetch_factor           │ NONE          │
│ 4    │ streaming_dataset_or_dali │ LOW           │
└──────┴───────────────────────────┴───────────────┘
                                   Anomalies                                    
┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Type            ┃ Severity ┃ Message                                         ┃
┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ LOSS_DIVERGENCE │ CRITICAL │ Loss increased for 3+ consecutive steps — check │
│                 │          │ LR, batch, or bad data.                         │
└─────────────────┴──────────┴─────────────────────────────────────────────────┘
  Anomalies detected: ['LOSS_DIVERGENCE']
  ✅ Loss divergence detected (same root cause — acceptable)

==================================================
GATE 3: ✅ ALL PASSED — Ready for Lambda GPU demo


## License

Apache 2.0 — see [LICENSE](LICENSE).

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md).
