Metadata-Version: 2.4
Name: auto-lora
Version: 0.1.0
Summary: Automated Hyperparameter Optimization Platform for Efficient LLM Fine-Tuning
Author: shrey1720
License: MIT
Project-URL: Homepage, https://github.com/shrey1720/alora
Project-URL: Repository, https://github.com/shrey1720/alora
Keywords: llm,lora,fine-tuning,hyperparameter-optimization,peft
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1.0
Requires-Dist: transformers>=4.38.0
Requires-Dist: peft>=0.8.0
Requires-Dist: datasets>=2.16.0
Requires-Dist: accelerate>=0.26.0
Requires-Dist: bitsandbytes>=0.42.0
Requires-Dist: trl>=0.7.0
Requires-Dist: sentencepiece>=0.1.99
Requires-Dist: protobuf>=3.20.0
Requires-Dist: optuna>=3.5.0
Requires-Dist: typer[all]>=0.9.0
Requires-Dist: rich>=13.0.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: matplotlib>=3.8.0
Requires-Dist: jinja2>=3.1.0
Requires-Dist: psutil>=5.9.0
Requires-Dist: pynvml>=11.5.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: unsloth
Requires-Dist: rouge-score
Requires-Dist: nltk
Requires-Dist: py-cpuinfo
Requires-Dist: tabulate
Requires-Dist: sentence-transformers
Requires-Dist: sacrebleu
Requires-Dist: scikit-learn
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Requires-Dist: ruff>=0.2.0; extra == "dev"
Provides-Extra: wandb
Requires-Dist: wandb>=0.16.0; extra == "wandb"
Dynamic: license-file

# Auto-LoRA

> 🚀 Automated Hyperparameter Optimization Platform for Efficient LLM Fine-Tuning

Auto-LoRA automates parameter-efficient fine-tuning of Large Language Models using LoRA/QLoRA. It reduces trial-and-error by automatically selecting optimal training configurations based on dataset characteristics and available hardware.

## Features

- **⚡ Unsloth Powered** — 2x–5x faster training and 70% less VRAM (automatic detection)
- **📊 Scientific Benchmarking** — Journal-grade metrics (ROUGE-L, TPS, Latency, Peak VRAM)
- **🎨 Dynamic Feedback** — Real-time row-by-row dataset loading and analysis progress bars
- **🎯 Automatic Tuning** — Bayesian optimization via Optuna finds the best LoRA configuration
- **🔍 Hardware-Aware** — Auto-detects GPU/VRAM and adjusts strategy (LoRA vs QLoRA)
- **📈 Rich Reports** — HTML dashboards, publication-quality charts, CSV exports
- **🔧 Presets** — Optimized defaults for chatbot, coding, QA, summarization, domain tasks
- **💾 Run History** — SQLite tracking of all runs, trials, and artifacts

## Quick Start

### Installation

```bash
# Clone the repository
git clone https://github.com/shrey1720/alora.git
cd alora

# Install in development mode
pip install -e ".[dev]"

# Optional: Install Unsloth for 2-5x speedup
# (Recommended for NVIDIA GPUs)
pip install unsloth xformers
```

### Usage

```bash
# 1. Check system health
auto-lora doctor

# 2. Initialize project
auto-lora init

# 3. Train with automatic tuning
auto-lora train --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --data datasets/sample_faq.json --max-trials 5

# 4. View results
auto-lora runs list
auto-lora runs show <run_id>

# 5. Generate report
auto-lora report --run <run_id>

# 6. Scientific Benchmark
# Generates a journal-ready technical profile (TPS, ROUGE-L, VRAM)
auto-lora benchmark --run <run_id> --references gold_answers.txt

# 7. Export adapter
auto-lora export --run <run_id>
```

### Using Presets

```bash
# Chatbot preset (conversational focus)
auto-lora train --model mistralai/Mistral-7B-v0.1 --data chat_data.json --preset chatbot

# Coding preset (lower LR, code tokenization)
auto-lora train --model codellama/CodeLlama-7b-hf --data code_data.json --preset coding

# QA preset (fact-oriented)
auto-lora train --model meta-llama/Llama-2-7b-hf --data qa_data.json --preset qa
```

## CLI Commands

| Command | Description |
|---------|-------------|
| `auto-lora init` | Initialize project structure |
| `auto-lora doctor` | System health check |
| `auto-lora train` | Train with automatic tuning |
| `auto-lora runs list` | List all training runs |
| `auto-lora runs show <id>` | Show run details |
| `auto-lora report --run <id>` | Generate HTML report |
| `auto-lora benchmark --run <id>` | Benchmark trained adapter |
| `auto-lora export --run <id>` | Export adapter for deployment |

## Architecture

```
auto_lora/
├── cli/          # Typer CLI commands
├── core/         # Config, presets, constants
├── hardware/     # GPU/CPU detection
├── dataset/      # Loading, validation, formatting
├── tuner/        # Optuna hyperparameter search
├── trainer/      # LoRA/QLoRA training engine
├── metrics/      # Scoring, benchmarking
├── reports/      # HTML reports, charts, exports
├── db/           # SQLite run/trial storage
└── utils/        # Logging, display, filesystem
```

## Requirements

- Python >= 3.10
- PyTorch >= 2.1.0
- NVIDIA GPU with CUDA (recommended, CPU supported)
- 8GB+ VRAM for QLoRA, 16GB+ for LoRA

## License

MIT
