Metadata-Version: 2.4
Name: kerdosai
Version: 0.2.1
Summary: Universal LLM Training & RAG Agent for HuggingFace
Author: KerdosAI Team
Author-email: contact@kerdosai.com
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.30.0
Requires-Dist: datasets>=2.12.0
Requires-Dist: accelerate>=0.20.0
Requires-Dist: sentencepiece>=0.1.99
Requires-Dist: protobuf>=4.23.0
Requires-Dist: numpy>=1.24.0
Requires-Dist: pandas>=2.0.0
Requires-Dist: scikit-learn>=1.2.0
Requires-Dist: tqdm>=4.65.0
Requires-Dist: faiss-cpu>=1.7.4
Requires-Dist: sentence-transformers>=2.2.2
Requires-Dist: PyMuPDF>=1.22.5
Requires-Dist: python-docx>=0.8.11
Requires-Dist: gradio>=4.0.0
Requires-Dist: huggingface-hub>=0.28.0
Requires-Dist: tenacity>=8.2.0
Requires-Dist: wandb>=0.15.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: fastapi>=0.100.0
Requires-Dist: uvicorn>=0.22.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: docker>=6.1.0
Requires-Dist: pyyaml>=6.0.0
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: license
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

---
license: apache-2.0
---

# KerdosAI — Universal LLM Training & RAG Agent

[![PyPI version](https://badge.fury.io/py/kerdosai.svg)](https://pypi.org/project/kerdosai/)
[![Python](https://img.shields.io/pypi/pyversions/kerdosai)](https://pypi.org/project/kerdosai/)
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
[![HuggingFace Space](https://img.shields.io/badge/🤗%20HuggingFace-Demo-yellow)](https://huggingface.co/spaces/kerdosdotio/Custom-LLM-Chat)
[![GitHub](https://img.shields.io/badge/GitHub-kerdosai-black?logo=github)](https://github.com/bhaskarvilles/kerdosai)

> **Enterprise-grade LLM Training + Retrieval-Augmented Generation (RAG) toolkit.**  
> Fine-tune any HuggingFace model on your data, then deploy it with a full document Q&A chat interface — locally or on cloud.

---

## What's New in v0.2.0

- 🆕 **`kerdosai.rag` submodule** — full RAG pipeline (document loading → FAISS indexing → LLM answering)
- 🆕 **`KnowledgeBase`** — high-level API to index PDF/DOCX/TXT/MD/CSV files
- 🆕 **`RAGAgent`** — streaming and blocking chat with conversation history
- 🆕 **`deployment_type="gradio-rag"`** — launch the [HuggingFace Space UI](https://huggingface.co/spaces/kerdosdotio/Custom-LLM-Chat) locally with one line
- 🆕 **`kerdosai rag-chat` CLI command** — on-premise RAG UI in one command
- Updated dependencies: `faiss-cpu`, `sentence-transformers`, `PyMuPDF`, `python-docx`, `gradio`, `huggingface-hub`, `tenacity`

---

## Installation

```bash
# Standard install
pip install kerdosai

# With all optional extras
pip install "kerdosai[all]"
```

**Requirements:** Python 3.8+ · PyTorch 2.0+ · CUDA-compatible GPU (recommended for training)

---

## Quick Start

### 1. RAG — Document Q&A (no GPU needed)

```python
from kerdosai.rag import KnowledgeBase, RAGAgent

# Index your documents (PDF, DOCX, TXT, MD, CSV)
kb = KnowledgeBase()
kb.index_documents(["handbook.pdf", "faq.docx", "policy.txt"])
print(f"Indexed {kb.chunk_count} chunks from {kb.indexed_sources}")

# Chat with your documents (HF Inference API — no GPU required)
agent = RAGAgent(hf_token="hf_...", knowledge_base=kb)

# Blocking answer
print(agent.chat("What is the leave policy?"))

# Streaming answer
for partial in agent.chat_stream("Summarise the refund section."):
    print(partial, end="\r")
```

### 2. Launch the Enterprise Chat UI Locally

```python
from kerdosai.deployer import Deployer
from kerdosai.rag import KnowledgeBase

kb = KnowledgeBase()
kb.index_documents(["report.pdf"])

deployer = Deployer(model=None, tokenizer=None)
deployer.deploy(
    deployment_type="gradio-rag",
    host="0.0.0.0",
    port=7860,
    hf_token="hf_...",       # or set HF_TOKEN env var
    knowledge_base=kb,
)
# → Open http://localhost:7860
```

### 3. CLI — One-Command RAG Chat Server

```bash
# Pre-index files and open the Gradio UI
kerdosai rag-chat \
  --data handbook.pdf faq.docx policy.txt \
  --hf-token hf_... \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --port 7860

# Use HF_TOKEN env var instead of --hf-token
export HF_TOKEN=hf_...
kerdosai rag-chat --data ./company_docs/ --port 7860
```

### 4. Fine-Tune a Model (Training Pipeline)

```python
from kerdosai import KerdosAgent

agent = KerdosAgent(
    base_model="meta-llama/Llama-3.1-8B",
    training_data="data/training.csv",
)

metrics = agent.train(epochs=3, batch_size=4, learning_rate=2e-5)
agent.save("./my-finetuned-model")
print(metrics)
```

### 5. Deploy Fine-Tuned Model as REST API

```python
from kerdosai import KerdosAgent

agent = KerdosAgent.load("./my-finetuned-model")
agent.deploy(deployment_type="rest", host="0.0.0.0", port=8000)
# POST /generate → {"text": "...", "max_length": 200}
```

---

## RAG Module Reference

### `KnowledgeBase`

```python
from kerdosai.rag import KnowledgeBase

kb = KnowledgeBase(
    embedding_model="BAAI/bge-small-en-v1.5",  # SentenceTransformer model
    chunk_size=512,                              # Max chars per chunk
    chunk_overlap=64,                            # Overlap between chunks
    min_score=0.30,                              # Min cosine similarity threshold
)

kb.index_documents(["doc1.pdf", "doc2.docx"])   # Add documents (duplicate-safe)
kb.index_documents(["doc3.txt"])                 # Incrementally add more

results = kb.search("What is the refund policy?", top_k=5)
# → [{"source": "doc1.pdf", "text": "...", "score": 0.87}, ...]

print(kb.chunk_count)       # Total indexed chunks
print(kb.indexed_sources)   # Set of indexed filenames
kb.clear()                  # Reset index
```

### `RAGAgent`

```python
from kerdosai.rag import RAGAgent

agent = RAGAgent(
    hf_token="hf_...",                              # HF API token
    model="meta-llama/Llama-3.1-8B-Instruct",       # LLM model ID
    top_k=5,                                         # Chunks retrieved per query
    embedding_model="BAAI/bge-small-en-v1.5",        # Embedding model
)

agent.index(["report.pdf", "handbook.docx"])         # Index documents

# Blocking
reply = agent.chat("What are the payment terms?")

# Streaming
for partial in agent.chat_stream("Summarise section 3."):
    print(partial, end="\r")

agent.reset_history()                                # Clear conversation
```

### Low-Level RAG API

```python
from kerdosai.rag import (
    load_documents,    # Parse files → [{"source", "text"}]
    build_index,       # List[dict] → VectorIndex
    add_to_index,      # Incrementally extend a VectorIndex
    retrieve,          # VectorIndex + query → top-K chunks
    answer_stream,     # Chunks + HF token → streaming generator
    answer,            # Chunks + HF token → full string
    VectorIndex,       # Dataclass: chunks + faiss index + embedder
)

docs     = load_documents(["report.pdf", "policy.docx"])
index    = build_index(docs, embedding_model="BAAI/bge-small-en-v1.5")
chunks   = retrieve("What is the refund policy?", index, top_k=5)
response = answer("What is the refund policy?", chunks, hf_token="hf_...")
```

**Supported file types:** `.pdf` (PyMuPDF) · `.docx` (python-docx, incl. tables) · `.txt` · `.md` · `.csv`

---

## Training API Reference

### `KerdosAgent`

| Method                                                | Description                                                      |
| ----------------------------------------------------- | ---------------------------------------------------------------- |
| `KerdosAgent(base_model, training_data, device=None)` | Initialize with a HuggingFace model ID and path to training data |
| `.train(epochs, batch_size, learning_rate, **kwargs)` | Fine-tune the model; returns metrics dict                        |
| `.deploy(deployment_type, host, port, **kwargs)`      | Deploy as REST / Docker / Kubernetes / Gradio-RAG                |
| `.save(output_dir)`                                   | Save model + tokenizer to disk                                   |
| `.load(model_dir)`                                    | Class method — load a saved model                                |

### `Trainer`

| Method                                                    | Description                     |
| --------------------------------------------------------- | ------------------------------- |
| `Trainer(model, tokenizer, device, use_wandb=True)`       | Initialize trainer              |
| `.train(dataset, epochs, batch_size, learning_rate, ...)` | Run HuggingFace training loop   |
| `.evaluate(dataset, batch_size)`                          | Evaluate and return `eval_loss` |

### `DataProcessor`

| Method                                                         | Description                                   |
| -------------------------------------------------------------- | --------------------------------------------- |
| `DataProcessor(data_path, max_length=512, text_column="text")` | Initialize                                    |
| `.prepare_dataset(tokenizer=None)`                             | Load, clean, tokenize → HuggingFace `Dataset` |
| `.validate_data()`                                             | Check data quality and print warnings         |

**Supported training data formats:** `.csv` (with `text` column) · `.json` (list of objects with `text` key)

---

## CLI Reference

```
kerdosai <command> [options]

Commands:
  train        Fine-tune a model on custom data
  deploy       Deploy a trained model
  rag-chat     Launch a RAG document Q&A UI (no local GPU needed)
```

### `kerdosai train`

```bash
kerdosai train \
  --model meta-llama/Llama-3.1-8B \
  --data ./data/training.csv \
  --output ./my-model \
  --epochs 3 \
  --batch-size 4 \
  --learning-rate 2e-5
```

### `kerdosai deploy`

```bash
# REST API
kerdosai deploy --model-dir ./my-model --type rest --port 8000

# Gradio RAG UI (with a fine-tuned local model)
kerdosai deploy --model-dir ./my-model --type gradio-rag --hf-token hf_... --port 7860

# Docker container
kerdosai deploy --model-dir ./my-model --type docker
```

### `kerdosai rag-chat`

```bash
kerdosai rag-chat \
  --data company.pdf handbook.docx faq.txt \
  --hf-token hf_... \
  --model meta-llama/Llama-3.1-8B-Instruct \
  --embedding-model BAAI/bge-small-en-v1.5 \
  --host 0.0.0.0 \
  --port 7860
```

---

## Deployment Options

| Type         | Command                                | Description                          |
| ------------ | -------------------------------------- | ------------------------------------ |
| `rest`       | `deploy(deployment_type="rest")`       | FastAPI REST API on `POST /generate` |
| `gradio-rag` | `deploy(deployment_type="gradio-rag")` | Full Kerdos RAG Chat UI (Gradio)     |
| `docker`     | `deploy(deployment_type="docker")`     | Build & run Docker container         |
| `kubernetes` | `deploy(deployment_type="kubernetes")` | Generate K8s YAML manifests          |

---

## Architecture

```
kerdosai/
├── __init__.py          # KerdosAgent, Trainer, Deployer, DataProcessor, KnowledgeBase, RAGAgent
├── agent.py             # KerdosAgent — orchestrates training + deployment
├── trainer.py           # Trainer — HuggingFace training loop + W&B logging
├── deployer.py          # Deployer — REST / Gradio-RAG / Docker / Kubernetes
├── data_processor.py    # DataProcessor — CSV/JSON loading, tokenization
├── cli.py               # CLI — train / deploy / rag-chat commands
└── rag/
    ├── __init__.py          # Public RAG API surface
    ├── document_loader.py   # PDF / DOCX / TXT / MD / CSV parser
    ├── embedder.py          # FAISS + SentenceTransformer index builder
    ├── retriever.py         # Top-K cosine-similarity retrieval
    ├── chain.py             # HF Inference API streaming + blocking answer
    ├── knowledge_base.py    # KnowledgeBase high-level class
    └── rag_agent.py         # RAGAgent — chat + history management
```

---

## HuggingFace Space Demo

Try the live RAG demo at 👉 **[huggingface.co/spaces/kerdosdotio/Custom-LLM-Chat](https://huggingface.co/spaces/kerdosdotio/Custom-LLM-Chat)**

Upload any PDF, DOCX, or TXT file and ask questions. The AI answers **only from your documents** — never from outside knowledge.

> The full `kerdosai` package lets you run this same UI privately on your own servers with `kerdosai rag-chat`.

---

## Environment Variables

| Variable    | Description                         | Default                            |
| ----------- | ----------------------------------- | ---------------------------------- |
| `HF_TOKEN`  | HuggingFace API token for inference | —                                  |
| `LLM_MODEL` | LLM model ID for generation         | `meta-llama/Llama-3.1-8B-Instruct` |

---

## Real-World Applications

### Healthcare

- Clinical documentation automation
- Patient Q&A from medical records
- HIPAA-compliant private deployment

### Financial Services

- Policy & compliance Q&A
- Risk report summarisation
- Private on-premise data security

### Legal

- Contract review and clause extraction
- Case research from uploaded precedents
- Confidential document handling

### Enterprise Internal Tools

- HR handbook chatbot
- IT knowledge base
- Onboarding document Q&A

---

## Requirements

```
Python >= 3.8
torch >= 2.0.0
transformers >= 4.30.0
faiss-cpu >= 1.7.4
sentence-transformers >= 2.2.2
PyMuPDF >= 1.22.5
python-docx >= 0.8.11
gradio >= 4.0.0
huggingface-hub >= 0.28.0
tenacity >= 8.2.0
```

Full list in [requirements.txt](requirements.txt).

---

## Contributing

We welcome contributions! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

---

## License

MIT License — see [LICENSE](LICENSE) for details.

---

## About Kerdos Infrasoft

**Kerdos Infrasoft Private Limited** · CIN: U62099KA2023PTC182869  
🌐 [kerdos.in](https://kerdos.in) · 📬 [partnership@kerdos.in](mailto:partnership@kerdos.in)

> We are actively seeking **investment & partnerships** to build the fully customisable enterprise edition — including private LLM hosting, custom model fine-tuning, data privacy guarantees, and white-label deployments.

---

## Citation

```bibtex
@software{kerdosai2024,
  title  = {KerdosAI: Universal LLM Training & RAG Agent},
  author = {Kerdos Infrasoft Private Limited},
  year   = {2024},
  url    = {https://github.com/bhaskarvilles/kerdosai},
  note   = {v0.2.1}
}
```
