Metadata-Version: 2.4
Name: optimus-dl
Version: 0.0.2
Summary: A clean, modular framework for training large language models with modern PyTorch features
Author-email: Alex Dremov <dremov.me@gmail.com>
Maintainer-email: Alex Dremov <dremov.me@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/alexdremov/optimus-dl
Project-URL: Repository, https://github.com/alexdremov/optimus-dl
Project-URL: Documentation, https://github.com/alexdremov/optimus-dl#readme
Project-URL: Bug Tracker, https://github.com/alexdremov/optimus-dl.git/issues
Keywords: machine-learning,deep-learning,pytorch,llm,transformer,training,nlp
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.14,>=3.11
Description-Content-Type: text/markdown
Requires-Dist: click>=8.2.1
Requires-Dist: colorlog>=6.9.0
Requires-Dist: datasets>=4.0.0
Requires-Dist: hydra-core>=1.3.2
Requires-Dist: lm-eval>=0.4.9
Requires-Dist: omegaconf>=2.3.0
Requires-Dist: pydantic>=2.12.5
Requires-Dist: rich>=13.7.0
Requires-Dist: tiktoken>=0.9.0
Requires-Dist: torch>=2.7.0
Requires-Dist: torchdata>=0.11.0
Requires-Dist: torcheval>=0.0.7
Requires-Dist: tqdm>=4.67.1
Requires-Dist: transformers>=4.53.2
Requires-Dist: wandb>=0.21.0
Requires-Dist: zstandard>=0.23.0
Provides-Extra: dev
Requires-Dist: black>=25.1.0; extra == "dev"
Requires-Dist: isort>=6.0.1; extra == "dev"
Requires-Dist: pytest>=8.4.1; extra == "dev"
Requires-Dist: pytest-cov>=6.0.0; extra == "dev"
Requires-Dist: pytest-xdist>=3.8.0; extra == "dev"
Requires-Dist: mypy>=1.13.0; extra == "dev"
Requires-Dist: pre-commit>=4.0.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: sphinx>=8.1.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=3.0.0; extra == "docs"
Requires-Dist: myst-parser>=4.0.0; extra == "docs"
Provides-Extra: notebook
Requires-Dist: ipython>=8.37.0; extra == "notebook"
Requires-Dist: jupyter>=1.1.0; extra == "notebook"
Requires-Dist: matplotlib>=3.9.0; extra == "notebook"
Requires-Dist: seaborn>=0.13.0; extra == "notebook"

# Optimus-DL

**Optimus-DL** is a modular, high-performance research framework for training Large Language Models (LLMs) and other deep learning models. It leverages modern PyTorch features (AMP, DDP, Compile) and a flexible, composition-based architecture.

## Key Features

*   **Modular "Recipe" Architecture**: Clean separation between model definitions, data pipelines, and training logic.
*   **Hydra-based Configuration**: Hierarchical, type-safe, and easily conveniently override-able configurations.
*   **Universal Metrics System**: Lazy evaluation and automatic distributed aggregation of metrics.
*   **Modern PyTorch**: Built-in support for Mixed Precision (AMP), Distributed Data Parallel (DDP), and `torch.compile`.
*   **Registry System**: easy dependency injection and component swapping via a centralized registry.

## Quick Start

### Installation

```bash
# Clone the repository
git clone <repository-url>
cd optimus-dl

# Install in editable mode with dependencies
pip install -e .
```

### Training

Training is orchestrated via `scripts/train.py` using Hydra configs.

```bash
# Run with default configuration (Llama2 on TinyShakespeare)
python scripts/train.py

# Override specific parameters
python scripts/train.py model=gpt2 optimization.batch_size=64 common.use_gpu=true
```

## Project Structure

*   `optimus_dl/`: Main package source code.
    *   `core/`: Fundamental utilities (logging, registry, device management).
    *   `modules/`: Pluggable components (models, optimizers, data loaders).
    *   `recipe/`: Orchestration logic (training loops, evaluation).
*   `configs/`: Hierarchical Hydra configuration files.
*   `scripts/`: Entry points for training and evaluation.

## Development

The project enforces strict code quality standards.

```bash
# Run tests
pytest

# Format code
black .
isort .

# Type check
mypy .
```

## License

MIT License.
