Metadata-Version: 2.4
Name: pythonai-grounded-generator
Version: 0.1.1
Summary: Lightweight grounded text generation from retrieved context
Keywords: llm,rag,grounded-generation,transformers,qwen
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: torch>=2.2
Requires-Dist: transformers>=4.51.0

# PythonAI Grounded Generator

A compact Python helper for generating answers from retrieved context with an already loaded Hugging Face causal language model.

It provides:

- Custom system prompts and questions
- Token-budget-aware context truncation
- Qwen3 non-thinking chat formatting
- CPU or CUDA inference (device controlled by the caller)
- A plain-text answer suitable for wrapping in application-specific Pydantic models

Retrieval, context assembly, and structured JSON serialization remain the responsibility of the calling application. This package does **not** guarantee factual correctness or replace source validation.

## Installation

After publication:

```bash
pip install pythonai-grounded-generator
```

Or install the local project:

```bash
pip install .
```

## Example

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from pythonai_grounded import generate_answer

name = "Qwen/Qwen3-0.6B"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name).to("cpu")
model.eval()

answer = generate_answer(
    model=model,
    tokenizer=tokenizer,
    question="Which endpoint loads an adapter?",
    context="Use POST /v1/load_lora_adapter to load an adapter.",
    system_prompt="Answer only from the context. Copy technical names exactly.",
    max_new_tokens=64,
    device="cpu",
)
print(answer)
```

## API

```python
generate_answer(
    model,
    tokenizer,
    question: str,
    context: str,
    system_prompt: str,
    max_new_tokens: int = 64,
    device: str = "cpu",
) -> str
```

The supplied `model` and `tokenizer` must be compatible. The model must already be on the requested device and in evaluation mode. Model context limits are read from `model.config.max_position_embeddings`, falling back to 32768 if absent. Check actual model capabilities when switching models.

## Build

```bash
uv build
```

The distributions are created in `dist/`.
