Metadata-Version: 2.4
Name: adaptive-intelligence
Version: 4.0.5
Summary: Self-improving retrieval framework with RL routing, context engineering, MCP integration, agentic workflow, and persistent memory.
Author-email: Venkatkumar Rajan <venkatkumarr.vk99@gmail.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/VK-Ant/adaptive-intelligence
Project-URL: Documentation, https://github.com/VK-Ant/adaptive-intelligence#readme
Project-URL: Repository, https://github.com/VK-Ant/adaptive-intelligence
Project-URL: Issues, https://github.com/VK-Ant/adaptive-intelligence/issues
Project-URL: PyPI, https://pypi.org/project/adaptive-intelligence/
Project-URL: Paper, https://www.researchgate.net/publication/405076088
Project-URL: Portfolio, https://vk-ant.github.io/Venkatkumar/
Project-URL: Changelog, https://github.com/VK-Ant/adaptive-intelligence/blob/main/CHANGELOG.md
Keywords: rag,retrieval,reinforcement-learning,knowledge-graph,document-intelligence,adaptive,self-improving,llm,ai,nlp,evaluation,self-learning,context-engineering,mcp,agentic,memory,tool-use,multi-agent
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: vector
Requires-Dist: chromadb>=0.4.0; extra == "vector"
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == "openai"
Provides-Extra: huggingface
Requires-Dist: transformers>=4.35.0; extra == "huggingface"
Requires-Dist: torch>=2.0.0; extra == "huggingface"
Provides-Extra: pdf
Requires-Dist: PyMuPDF>=1.23.0; extra == "pdf"
Provides-Extra: docx
Requires-Dist: python-docx>=1.0.0; extra == "docx"
Provides-Extra: xlsx
Requires-Dist: openpyxl>=3.1.0; extra == "xlsx"
Provides-Extra: pptx
Requires-Dist: python-pptx>=0.6.21; extra == "pptx"
Provides-Extra: sql
Requires-Dist: sqlalchemy>=2.0.0; extra == "sql"
Provides-Extra: rerank
Requires-Dist: sentence-transformers>=2.2.0; extra == "rerank"
Provides-Extra: ocr
Requires-Dist: pytesseract>=0.3.10; extra == "ocr"
Requires-Dist: Pillow>=10.0; extra == "ocr"
Provides-Extra: all
Requires-Dist: chromadb>=0.4.0; extra == "all"
Requires-Dist: openai>=1.0.0; extra == "all"
Requires-Dist: transformers>=4.35.0; extra == "all"
Requires-Dist: torch>=2.0.0; extra == "all"
Requires-Dist: PyMuPDF>=1.23.0; extra == "all"
Requires-Dist: python-docx>=1.0.0; extra == "all"
Requires-Dist: openpyxl>=3.1.0; extra == "all"
Requires-Dist: python-pptx>=0.6.21; extra == "all"
Requires-Dist: pytesseract>=0.3.10; extra == "all"
Requires-Dist: Pillow>=10.0; extra == "all"
Requires-Dist: sentence-transformers>=2.2.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: mypy>=1.5; extra == "dev"
Dynamic: license-file

<div align="center">

<img src="https://raw.githubusercontent.com/VK-Ant/adaptive-intelligence/main/docs/images/image_ai.png" alt="adaptive-intelligence" width="100%">

# adaptive-intelligence

**Self-improving retrieval framework that learns, remembers, and connects tools.**

[![PyPI](https://img.shields.io/pypi/v/adaptive-intelligence)](https://pypi.org/project/adaptive-intelligence/)
[![Python](https://img.shields.io/pypi/pyversions/adaptive-intelligence)](https://pypi.org/project/adaptive-intelligence/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)
[![Paper](https://img.shields.io/badge/Paper-ResearchGate-00CCBB)](https://www.researchgate.net/publication/405076088)

[PyPI](https://pypi.org/project/adaptive-intelligence/) · [Paper](https://www.researchgate.net/publication/405076088) · [Portfolio](https://vk-ant.github.io/Venkatkumar/) · [llmevalkit](https://pypi.org/project/llmevalkit/)

</div>

---

## Why adaptive-intelligence?

Every RAG system uses the same retrieval strategy for every query. A revenue lookup gets vector search. A multi-document relationship chain gets the same vector search.

adaptive-intelligence fixes this. It uses reinforcement learning to select the best retrieval strategy per query type. The system evaluates every response, uses the score as a reward signal, and improves with every query answered.

```python
from adaptive_intelligence import AdaptiveAI

engine = AdaptiveAI()
engine.ingest("./documents")
response = engine.ask("What are the key risks?")
```

## Install

```bash
pip install adaptive-intelligence                # Zero deps (Ollama, no-LLM mode)
pip install adaptive-intelligence[vector]         # + ChromaDB vector search
pip install adaptive-intelligence[openai]         # + Any OpenAI-compatible API
pip install adaptive-intelligence[huggingface]    # + Local HuggingFace models
pip install adaptive-intelligence[all]            # Everything
```

## Features

### Context Engineering
Optimizes the entire context window — not just which chunks to retrieve, but what memory to include, how much history to keep, which tool results to add, and how to structure the prompt.

```python
engine = AdaptiveAI(context_engineering=True)
```

### MCP Integration
Connect external tools. Register MCP servers, REST APIs, or Python functions. The RL policy learns which tools to call per query type.

```python
engine.add_tool("financial", server="http://localhost:8081")
engine.add_tool("calculator", function=my_calc_function)
engine.add_tool("search", api_endpoint="https://api.example.com/search")

engine.list_tools()
engine.remove_tool("search")

# Serve your retrieval as an MCP server
engine.serve_mcp(port=8080)
```

### Agentic Workflow
Multi-round retrieval. The system retrieves, evaluates confidence, refines the query, calls tools, and retrieves again until the answer is sufficient.

```python
response = engine.ask("Analyze supply chain risks and mitigation", mode="agentic")
```

### Persistent Memory
Remembers across sessions. Routing patterns, user preferences, and facts persist to disk.

```python
engine.remember("focus_area", "supply chain risk")
engine.recall("focus_area")
engine.search_memory("supply chain")
```

### Incremental Learning
Add new documents anytime. The RL policy, knowledge graph, and memory continue from their current state — no restart needed.

```python
engine.ingest("./initial_docs")
engine.ingest("./quarterly_update.pdf")   # System continues, no restart
```

### RL-Based Retrieval Routing
Thompson Sampling or PPO learns which retrieval strategy works best per query type.

```python
engine = AdaptiveAI(rl_algorithm="ppo")
engine = AdaptiveAI(pretrained_policy=True, domain="financial")
engine.export_policy("learned.json")
engine.import_policy("learned.json")
```

### Conditional Graph Activation
Knowledge graph auto-built during ingestion. A 5-signal gate activates graph traversal only when the query needs relational reasoning. Saves compute on 70% of queries.

### Vectorless Mode
No ChromaDB, no embeddings, zero dependencies. Uses page-level BM25 with page citations.

```python
engine = AdaptiveAI(vectorless=True)
```

### Structured Output

```python
response = engine.ask("Extract metrics", output_format="json")
response = engine.ask("List items", output_format="csv")
response = engine.ask("Summarize", output_format="yaml")
```

### User Feedback

```python
response = engine.ask("What are the risks?")
engine.feedback(response.query_id, "good")   # +0.2 RL reward
engine.feedback(response.query_id, "bad")    # -0.3 RL reward + prompt evolution
```

## LLM Providers

The library works with any LLM. Zero required dependencies for basic usage.

### Free (no credit card needed)

```python
# Ollama — local, no extras needed
engine = AdaptiveAI()

# NVIDIA NIM
engine = AdaptiveAI(api_key="nvapi-...", base_url="https://integrate.api.nvidia.com/v1",
                    llm_model="meta/llama-3.1-70b-instruct")

# Groq
engine = AdaptiveAI(api_key="gsk_...", base_url="https://api.groq.com/openai/v1",
                    llm_model="llama-3.3-70b-versatile")

# Google Gemini
engine = AdaptiveAI(api_key="...", base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
                    llm_model="gemini-2.0-flash")

# Together AI
engine = AdaptiveAI(api_key="...", base_url="https://api.together.xyz/v1",
                    llm_model="meta-llama/Llama-3-70b-chat-hf")

# Fireworks AI
engine = AdaptiveAI(api_key="...", base_url="https://api.fireworks.ai/inference/v1",
                    llm_model="accounts/fireworks/models/llama-v3p1-70b-instruct")

# HuggingFace local — needs [huggingface] extras
engine = AdaptiveAI(llm_backend="huggingface", llm_model="Qwen/Qwen2.5-1.5B-Instruct")

# No LLM — retrieval only
engine = AdaptiveAI(llm_backend="none")
```

### Paid

```python
# OpenAI
engine = AdaptiveAI(api_key="sk-...", llm_model="gpt-4o")

# Grok (xAI)
engine = AdaptiveAI(api_key="xai-...", base_url="https://api.x.ai/v1")

# Azure OpenAI
engine = AdaptiveAI(azure_endpoint="https://your.openai.azure.com/", api_key="...")
```

### Self-hosted

```python
# vLLM server
engine = AdaptiveAI(base_url="http://localhost:8000/v1")

# Any OpenAI-compatible server
engine = AdaptiveAI(base_url="http://your-server:8000/v1")
```

## How It Works

1. **Understand** — Trigger interpreter classifies query type, complexity, domain, entities (no LLM call)
2. **Decide** — RL policy selects retrieval route, depth, graph activation, tools to call
3. **Retrieve** — Executes via vector, BM25, or page index with RRF fusion. Graph activates conditionally.
4. **Generate** — Cross-encoder reranks chunks. Context engineer assembles full context window. LLM generates.
5. **Learn** — 6 metrics evaluate response. Composite score = RL reward. Policy updates. Next query is better.

## Demos

### Colab (no setup needed)

Run on free T4 GPU with no API key: [`notebooks/adaptive_intelligence_v4_demo.ipynb`](notebooks/adaptive_intelligence_v4_demo.ipynb)

### Local demos

```bash
cd demo_mcp_agenticai
pip install -r requirements.txt
python demo_basic.py      # Basic usage + incremental learning
python demo_tools.py      # Tool registry + cost optimization
python demo_agentic.py    # Agentic multi-round retrieval
python demo_mcp_server.py # Serve as MCP server (terminal 1)
python demo_mcp_client.py # Connect to MCP server (terminal 2)
```

## FAQ

**1. Does it work without an LLM?**

Yes. Set `llm_backend="none"` and the system returns relevant document excerpts. RL routing, graph activation, memory, and evaluation all work without an LLM.

**2. Does it work without a vector database?**

Yes. Set `vectorless=True` for page-level BM25 search with zero dependencies. Same RL routing, same graph, same learning.

**3. How many queries before it starts learning?**

Default warmup is 15 queries. During warmup, the system uses smart heuristic defaults while collecting RL statistics. After 15 queries, the learned policy takes over. Use `pretrained_policy=True` to skip warmup entirely.

**4. Which retrieval strategies does the RL choose from?**

Six routes: keyword only, vector only, hybrid (keyword + vector with RRF), table first, graph first, and graph hybrid. The RL also selects retrieval depth and whether to activate the knowledge graph.

**5. How is the knowledge graph built?**

Automatically during document ingestion from entity co-occurrences. No manual setup. A 5-signal gate decides per-query whether to activate graph traversal.

**6. What is context engineering?**

Instead of just stuffing retrieved chunks into a prompt, context engineering optimizes the entire context window — system prompt, memory entries, conversation history, tool results, and chunks — with token budget allocation per component.

**7. How does agentic mode work?**

The system retrieves, evaluates confidence, and if it's below threshold, refines the query and retrieves again. It can also call registered tools between rounds. Maximum 3 rounds by default.

**8. Can I add documents after initial ingestion?**

Yes. Call `engine.ingest()` again with new documents. The RL policy, knowledge graph, and memory all continue from their current state. No restart needed.

**9. What about latency?**

The system adds approximately 100-150ms overhead per query (classification, RL decision, evaluation, policy update). The LLM call typically takes 500-3000ms. Overhead is roughly 5-10% of total response time.

**10. How does it save cost?**

The RL learns optimal retrieval depth per query type. Factual queries get depth 2 (2 chunks), complex queries get depth 8. Fewer chunks = fewer tokens = lower LLM cost. The RL also learns which tools to skip, reducing unnecessary API calls.

**11. Is it production-ready?**

The library has 99 tests, crash recovery with auto-checkpoint, and graceful shutdown. BM25 is in-memory which works for hundreds of documents. For 50K+ documents, a disk-backed index is planned.

**12. How is this different from LangChain or LlamaIndex?**

Those are orchestration frameworks with static pipelines. You configure the retrieval strategy once. adaptive-intelligence learns the optimal strategy per query type through reinforcement learning. The system improves with every query answered.

## Version History

| Version | Focus | Features |
|---------|-------|----------|
| v1 | Core Innovation | RL routing, conditional graph, 6-metric evaluation, 15 features |
| v2 | Production Ready | Vectorless mode, crash recovery, SQL connector, 10+ providers, 28 features |
| v3 | Intelligence | PPO, reranking, multi-query, pre-trained policies, transfer learning, 44 features |
| v4 | Context + Agentic | Context engineering, MCP, agentic workflow, persistent memory, incremental learning |

## Project Structure

```
adaptive_intelligence/
    core/           # Engine, config, response objects
    ingestion/      # Parser (10+ formats), chunker, ingestion engine
    indexes/        # Vector (ChromaDB), keyword (BM25), page index
    query/          # Trigger interpreter, query classification
    rl/             # Thompson Sampling, PPO, reranker, multi-query, pretrained
    graph/          # Knowledge graph, conditional activation
    prompts/        # Adaptive prompt engine, template evolution
    evaluation/     # 6-metric evaluation engine
    llm/            # LLM providers (Ollama, OpenAI, HuggingFace)
    memory/         # Learning memory, persistent memory
    context/        # Context engineering, token budget
    mcp/            # MCP server, tool registry
    agentic/        # Agentic workflow, multi-round retrieval
    security/       # Audit trail, PII scanning
    utils/          # Logging, timing, ID generation
```

## Citation

If you use adaptive-intelligence in your research, please cite:

```bibtex
@article{rajan2026adaptive,
  title={Adaptive Retrieval Orchestration for Self-Learning Knowledge Systems},
  author={Rajan, Venkatkumar},
  year={2026},
  publisher={ResearchGate},
  url={https://www.researchgate.net/publication/405076088}
}
```

## Contributing

Contributions are welcome.

1. Fork the repository
2. Create a feature branch (`git checkout -b feature/your-feature`)
3. Add tests for your changes
4. Run tests (`pytest tests/ -v`)
5. Submit a pull request

## Also by me

[**llmevalkit**](https://pypi.org/project/llmevalkit/) — 61 metrics for LLM evaluation with HIPAA, GDPR, DPDP, and EU AI Act compliance modules.

adaptive-intelligence was born from llmevalkit. If you can measure LLM quality (llmevalkit), you can use those measurements as a reward signal to improve retrieval (adaptive-intelligence).

## Author

**Venkatkumar Rajan** · [@VK_Venkatkumar](https://linkedin.com/in/venkatkumarvk) · [Portfolio](https://vk-ant.github.io/Venkatkumar/)

## License

[Apache License 2.0](LICENSE)
