Metadata-Version: 2.4
Name: queryclf
Version: 0.1.0
Summary: Ultra-fast, lightweight ONNX query difficulty classifier for pre-routing LLM cascades
Author: Praveen Ramesh
License-Expression: MIT
Project-URL: Homepage, https://github.com/prvn-ramesh/QueryClassifier
Project-URL: Repository, https://github.com/prvn-ramesh/QueryClassifier
Project-URL: Model Hub, https://huggingface.co/prvn-ramesh/query-classifier-onnx
Project-URL: Bug Tracker, https://github.com/prvn-ramesh/QueryClassifier/issues
Keywords: llm,routing,query-complexity,onnx,fastembed,modernbert
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: onnxruntime>=1.17.0
Requires-Dist: tokenizers>=0.19.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: huggingface-hub>=0.20.0
Requires-Dist: python-dotenv>=1.0.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: build>=1.0.0; extra == "dev"
Requires-Dist: twine>=5.0.0; extra == "dev"

# QueryClassifier

**Goal:** A pip-installable Python library that classifies a text query into `low` / `medium` / `hard` complexity, for use as a pre-routing step in an LLM cascade (cheap model → medium model → frontier model).

**Approach:** Fine-tuned `answerdotai/ModernBERT-large` quantized into int8 ONNX format (`query-classifier-onnx`). Uses a FastEmbed-inspired architecture powered by **pure ONNXRuntime + Rust `tokenizers`** — zero PyTorch, zero `transformers` library dependencies, and zero heavy GPU requirements.

---

## ⚡ Quickstart

### Installation

```bash
pip install queryclf
```

Dependencies installed are lightweight (~38MB total): `onnxruntime`, `tokenizers`, `numpy`, `huggingface-hub`, and `python-dotenv`.

---

## ⚙️ Configuration (`.env`)

Configure model paths dynamically in `.env` (refer to `.env.example`):

```env
# Hugging Face Repository ID (Default: prvn-ramesh/query-classifier-onnx)
QUERY_CLASSIFIER_HF_REPO=prvn-ramesh/query-classifier-onnx

# Optional local model directory override
QUERY_CLASSIFIER_MODEL_PATH=
```

**Resolution Priority:**
1. Explicit Python Argument (`hf_repo="..."` or `model_path="..."`)
2. Environment Variable `QUERY_CLASSIFIER_HF_REPO`
3. Environment Variable `QUERY_CLASSIFIER_MODEL_PATH`
4. Default Hugging Face Repository (`prvn-ramesh/query-classifier-onnx`)

---

## 🐍 Python API Usage

```python
import asyncio
from query_classifier import QueryClassifier

# Initializes classifier with built-in LRU cache and CPU thread tuning
classifier = QueryClassifier(
    cache_size=1000,             # Max items in LRU query cache (default: 1000)
    intra_op_num_threads=4,      # Intra-node CPU parallelism
)

# 1. Single query prediction
result = classifier.predict("Can you summarize this document?")

print(result.label)        # Output: "low"
print(result.confidence)   # Output: 0.5779
print(result.latency_ms)   # Output: 29.8 ms
print(result.scores)       # Output: {'low': 0.5779, 'medium': 0.4201, 'hard': 0.0020}

# 2. Batch query prediction
queries = [
    "What is the capital of France?",
    "Calculate the integral of x^2 * sin(x) dx",
    "Write a lock-free multi-threaded queue in C++ using atomics"
]

batch_results = classifier.predict_batch(queries)
for q, res in zip(queries, batch_results):
    print(f"[{res.label.upper()}] ({res.latency_ms:.1f}ms) -> {q}")

# 3. Async prediction (for FastAPI / AsyncIO frameworks)
async def main():
    async_res = await classifier.predict_async("Explain quantum computing")
    print(f"Async prediction: {async_res.label} ({async_res.confidence:.2f})")

asyncio.run(main())

# 4. Cache statistics & management
print(classifier.cache_info())  # e.g., {'hits': 1, 'misses': 4, 'maxsize': 1000, 'currsize': 4}
classifier.clear_cache()
```

---

## 💻 CLI Command Usage

```bash
# Single query classification
queryclf "What is 2 + 2?"

# JSON output mode
queryclf "Write a Rust lock-free SPMC queue" --json

# Batch mode from file
queryclf --file queries.txt
```

---

## 📊 Benchmark & Accuracy Results

The underlying model [`prvn-ramesh/query-classifier-onnx`](https://huggingface.co/prvn-ramesh/query-classifier-onnx) is evaluated on an independent **held-out test set of 301 unseen queries** (balanced across `low`, `medium`, and `hard` buckets). For full details, see the [Hugging Face Model README](https://huggingface.co/prvn-ramesh/query-classifier-onnx).

### Accuracy & F1-Score Breakdown

- **Overall Accuracy**: **88.04%**
- **Macro F1-Score**: **0.8797**

| Class | Precision | Recall | F1-Score | Support |
| :--- | :--- | :--- | :--- | :--- |
| **`low`** | **100.00%** | 81.00% | **0.8950** | 100 |
| **`medium`** | **82.18%** | 83.00% | **0.8259** | 100 |
| **`hard`** | **84.87%** | **100.00%** | **0.9182** | 101 |
| **Overall** | **88.04% Acc** | — | **0.8797 Macro-F1** | **301 total** |

### Latency & Throughput Profile (CPU Execution)

| Metric | Measurement |
| :--- | :--- |
| **Median Latency (p50)** | **67.68 ms** |
| **Mean Latency** | **81.44 ms** |
| **p90 Latency** | **134.69 ms** |
| **p99 Latency** | **161.74 ms** |
| **Throughput** | **12.3 queries/sec** |

---

## 🛠️ Project Roadmap Status

- [x] **Phase 0 — Environment & Scaffold**: Package directory layout (`src/query_classifier`, `tests/`, `pyproject.toml`).
- [x] **Phase 2 — Labeled Dataset**: `dataset.csv` with query samples.
- [x] **Phase 4 & 5 — Fine-Tuned ModernBERT & ONNX int8 Quantization**: Quantized model published on Hugging Face at [`prvn-ramesh/query-classifier-onnx`](https://huggingface.co/prvn-ramesh/query-classifier-onnx).
- [x] **Phase 6 — Inference Engine**: Pure ONNXRuntime + Tokenizers inference engine (`onnx_engine.py`, `classifier.py`, `cli.py`).
