Metadata-Version: 2.4
Name: tuneprompt
Version: 1.0.2
Summary: Open-source prompt quality scoring and optimization for LLMs
Author: PromptForge Contributors
License: MIT
Project-URL: Homepage, https://github.com/arjun988/promptModel
Project-URL: Issues, https://github.com/arjun988/promptModel/issues
Project-URL: Documentation, https://github.com/arjun988/promptModel#readme
Project-URL: Hugging Face (Quality), https://huggingface.co/ArjunShukla/PromptForge-Quality
Project-URL: Hugging Face (Optimizer), https://huggingface.co/ArjunShukla/PromptForge-Optimizer
Keywords: prompt-engineering,llm,modernbert,huggingface,prompt-quality,promptforge,ai,agent,finetune
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1.0
Requires-Dist: transformers>=4.48.0
Requires-Dist: datasets>=2.19.0
Requires-Dist: accelerate>=0.30.0
Requires-Dist: peft>=0.11.0
Requires-Dist: scikit-learn>=1.4.0
Requires-Dist: pandas>=2.1.0
Requires-Dist: numpy>=1.26.0
Requires-Dist: scipy>=1.11.0
Requires-Dist: evaluate>=0.4.0
Requires-Dist: huggingface_hub>=0.23.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: tqdm>=4.66.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Provides-Extra: demo
Requires-Dist: gradio>=4.0.0; extra == "demo"
Dynamic: license-file

# PromptForge

**Local-first prompt quality scoring and optimization.**

Published on PyPI as **[`tuneprompt`](https://pypi.org/project/tuneprompt/)**.

PromptForge scores LLM prompts across seven quality dimensions, then rewrites weak prompts into clear, intent-preserving instructions — runnable on your machine via Python API, CLI, or Gradio.

```text
"Make an app about social media like facebook"
                    │
                    ▼
         ┌─────────────────────┐
         │  Quality Scorer     │  ModernBERT · ~150M
         │  41.5 → issues…     │
         └──────────┬──────────┘
                    ▼
         ┌─────────────────────┐
         │  Prompt Optimizer   │  Qwen2.5-1.5B + LoRA
         └──────────┬──────────┘
                    ▼
   Build a social media app similar to Facebook…
   profiles · feed · likes · constraints · output format
                    │
                    ▼
              41.5 → 94.0
```

[PyPI](https://pypi.org/project/tuneprompt/) · [Docs](docs/LOCAL.md) · [Product plan](docs/PRD.md) · [Contributing](CONTRIBUTING.md) · [License](LICENSE)

---

## Why PromptForge

Most prompt tools either **judge** quality or **rewrite** text. PromptForge does both in one local pipeline:

| Capability | What you get |
|------------|--------------|
| **Multi-dimension scoring** | Clarity, specificity, context, goals, constraints, completeness, actionability |
| **Intent-preserving rewrite** | Optimizes the *same* topic — not a generic template |
| **Validation & fallback** | Rejects repetitive / off-topic generations |
| **Runs locally** | ~1.65B total params; trains on 8 GB GPUs |
| **Dev-ready surface** | `pip` package, CLI, Gradio demo, Colab notebooks |

No API key required for inference once models are on disk.

---

## Example

**Input**

```text
Make an app about social media like facebook and stuff
```

**Output (optimizer)**

```text
Build a social media app similar to Facebook for product managers.
This is for a portfolio demo.

Core features:
- User profiles and friend connections
- News feed with posts, likes, and comments
- Basic notifications

Requirements:
- Use Python and Flask.
- Keep the first version simple and usable
- Include error handling and clear project structure

Include short examples.
```

**Quality:** `41.5 → 94.0` (Δ +52.5) · topic preserved · no fallback

---

## Models

| Component | Base | Size | Training |
|-----------|------|------|----------|
| **Scorer** | [`ModernBERT-base`](https://huggingface.co/answerdotai/ModernBERT-base) | ~150M | Full fine-tune |
| **Optimizer** | [`Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) | 1.5B | LoRA (base frozen) |

Weights are **not** stored in git. Train locally or download from Hugging Face:

```bash
pip install tuneprompt

python -m promptforge download \
  --quality-repo ArjunShukla/PromptForge-Quality \
  --optimizer-repo ArjunShukla/PromptForge-Optimizer
```

- Quality: https://huggingface.co/ArjunShukla/PromptForge-Quality  
- Optimizer: https://huggingface.co/ArjunShukla/PromptForge-Optimizer

---

## Results

Quality scorer (held-out):

| Split | MAE | Pearson |
|-------|----:|--------:|
| Validation | **2.73** | **0.993** |
| Test (overall) | **0.96** | **0.999** |

---

## Quickstart

### Install

```bash
pip install tuneprompt
```

Package: [`tuneprompt` on PyPI](https://pypi.org/project/tuneprompt/)  
Import module: `promptforge` · CLI: `tuneprompt` or `promptforge`

### Download models & run

```bash
python -m promptforge download \
  --quality-repo ArjunShukla/PromptForge-Quality \
  --optimizer-repo ArjunShukla/PromptForge-Optimizer

python -m promptforge init
python -m promptforge doctor
python -m promptforge run "Build me a website for a startup"
python -m promptforge analyze "Make an app." --json
```

Same via CLI entrypoints:

```bash
tuneprompt run "Build me a website for a startup"
# or
promptforge run "Build me a website for a startup"
```

> On some Windows setups, Application Control blocks `.venv\Scripts\*.exe`. Prefer `python -m promptforge …`.

### Python API

```python
from promptforge import PromptForge

# After download + init, or pass Hub / local paths:
pf = PromptForge(
    quality_model_path="ArjunShukla/PromptForge-Quality",
    optimizer_model_path="ArjunShukla/PromptForge-Optimizer",
)

print(pf.analyze("Make an app."))
result = pf.run("Make an app about social media like facebook and stuff")
print(result["optimized_prompt"])
print(result["delta"]["quality_score"])
```

### Install from source (optional)

```bash
git clone https://github.com/arjun988/promptModel.git
cd promptModel

python -m venv .venv
# Windows:  .venv\Scripts\activate
# Unix:     source .venv/bin/activate

pip install -U pip
pip install -e ".[demo,dev]"
```

**GPU tip:** default `pip install torch` is often CPU-only. For NVIDIA (incl. RTX 50-series):

```bash
pip uninstall -y torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
```

Full local guide: [docs/LOCAL.md](docs/LOCAL.md)

---

## Train your own

Anyone can improve the models with their own data:

```bash
# Phase 1 — quality scorer
python scripts/train_quality.py --require-gpu --regenerate

# Phase 2 — optimizer (recommended on 8GB GPUs)
python scripts/train_optimizer.py --require-gpu --fast --regenerate
```

| Flag / config | Purpose |
|---------------|---------|
| `--fast` | Loads `configs/optimizer_fast_8gb.yaml` |
| `--regenerate` | Rebuild curated optimizer dataset |
| `load_in_4bit: true` | Use if 1.5B LoRA OOMs |

---

## CLI

| Command | Description |
|---------|-------------|
| `init` | Create `~/.promptforge` and register model paths |
| `doctor` | GPU / config / model health check |
| `download` | Pull models from Hugging Face |
| `analyze` | Score a prompt |
| `optimize` | Rewrite a prompt |
| `run` | Score → optimize → compare |
| `eval` | Pipeline evaluation reports |
| `space` | Launch Gradio demo |
| `train-quality` / `train-optimizer` | Training entrypoints |

---

## Project layout

```text
promptModel/
├── src/promptforge/     # Package: scorer, optimizer, pipeline, CLI
├── configs/             # Training + local defaults
├── scripts/             # Train / eval / Hub export
├── demo/                # Gradio app
├── notebooks/
│   ├── colab/           # Self-contained experiments
│   └── package/         # Thin package drivers
├── docs/                # PRD + local setup
├── tests/
└── pyproject.toml
```

Notebooks: [notebooks/README.md](notebooks/README.md)

---

## Roadmap

| Phase | Deliverable | Status |
|------:|-------------|--------|
| 1 | Multi-dimension quality scorer | Done |
| 2 | Intent-preserving prompt optimizer (LoRA) | Done |
| 3 | Combined pipeline + eval + Gradio | Done |
| 4 | Local package + CLI (`tuneprompt`) | Done |
| 5 | VS Code / Cursor extension | Planned |

---

## Contributing

Issues and PRs welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, style, and PR expectations.

```bash
pip install -e ".[dev]"
pytest -q
```

---

## License

[MIT](LICENSE) © PromptForge contributors
