Metadata-Version: 2.4
Name: ragify-docs-mcp
Version: 0.2.0
Summary: Recursively scrapes websites and builds a localized memory context to return specific reference text chunks.
Requires-Python: >=3.12
Description-Content-Type: text/markdown
Requires-Dist: bs4>=0.0.2
Requires-Dist: fastmcp>=3.4.5
Requires-Dist: langchain>=1.3.14
Requires-Dist: langchain-classic>=1.0.8
Requires-Dist: langchain-community>=0.4.2
Requires-Dist: langchain-groq>=1.1.3
Requires-Dist: langchain-huggingface>=1.2.2
Requires-Dist: langchain-mcp-adapters>=0.3.1
Requires-Dist: langchain-ollama>=1.1.0
Requires-Dist: langchain-text-splitters>=1.1.2
Requires-Dist: langgraph>=1.2.10
Requires-Dist: lxml>=6.1.1
Requires-Dist: python-dotenv>=1.2.2
Requires-Dist: sentence-transformers>=5.6.1
Requires-Dist: tqdm>=4.70.0

# 🔌 RAGify Docs: Recursive Documentation Scraper MCP Server

A production-ready Model Context Protocol (MCP) server that empowers AI agents to recursively scrape entire software documentation sites, compile them into an in-memory vector store, and provide accurate, context-grounded answers to developer questions.

Built using the **FastMCP Framework**, **LangChain**, and **Ollama**, this server lets tools like Claude Desktop, Cursor, or Zed read documentation pages in real-time to resolve coding problems without leaving the chat interface.

---

## 🔥 Key Features

- **Recursive Deep Scraping:** Crawls documentation sites up to two levels deep out of the box using custom beautifulsoup extraction.
- **Dynamic Context Assembly:** Automatically splits raw website texts into clean, code-aware semantic blocks.
- **High-Diversity MMR Search:** Uses Maximal Marginal Relevance to fetch contrasting context points rather than duplicating search matches from single sections.
- **Local-First Architecture:** Leverages a lightweight in-memory vector index alongside local `llama3.2` models for privacy and cost efficiency.
- **Safe Data Pipeline:** Explicitly channels all background terminal metrics into `stderr` to avoid protocol communication corruption over standard I/O channels.

---

## 🛠️ Prerequisites

Before installing the server, ensure your local environment contains the following applications:

- **Python:** version 3.10 or higher.
- **Ollama:** Installed and running locally with the target model pulled:
  ```bash
  ollama pull llama3.2
  ```

---

## 📦 Installation & Setup

Follow these steps to download and configure the project directory on your local machine.

### 1. Clone and Navigate to the Project

```bash
git clone https://github.com
cd ragify_docs_2.0
```

### 2. Set Up a Virtual Environment

Create and boot up an isolated Python runtime container to avoid package conflicts:

```powershell
# Windows PowerShell
python -m venv .venv
.venv\Scripts\Activate.ps1

# macOS / Linux
python3 -m venv .venv
source .venv/bin/activate
```

### 3. Install Dependencies

Install the required framing libraries directly via `pip`:

```bash
pip install fastmcp langchain-community langchain-text-splitters langchain-huggingface langchain-core langchain-ollama tqdm bs4 lxml sentence-transformers python-dotenv
```

### 4. Create Environment Configuration

Create a configuration file named `.env` in the root of your workspace:

```text
# .env Configuration File
# Note: You can optionally add service platform API credentials here if swapping components
```

---

## 🚀 Local Testing & Debugging

The platform provides simple verification mechanisms to make sure everything links up fine before giving it over to real apps.

### Running via FastMCP

Execute the framework inspection validator to ensure the internal tool parsing definitions register correctly:

```bash
fastmcp inspect .\src\ragify_docs\tools.py
```

To boot up the live server instance locally via your custom python orchestration file, run:

```bash
python .\src\ragify_docs\main.py
```

_(You will see informational logging indicate the server is successfully listening for external connections over the standard I/O communication channel)._

---

## 💻 Client Integration (Claude Desktop)

To expose this capability directly to **Claude Desktop**, add its execution routing coordinates to your application parameters profile.

1. Press `Win + R` (Windows) and paste the following directory location to open your config file:
   ```text
   %APPDATA%\Claude\claude_desktop_config.json
   ```
2. Inject the following block into the structure (Ensure you escape all absolute disk location paths with double-backslashes `\\`):

```json
{
  "mcpServers": {
    "ragify-docs-server": {
      "command": "C:\\Users\\yasho\\OneDrive\\Desktop\\ragify_docs_2.0\\.venv\\Scripts\\python.exe",
      "args": [
        "C:\\Users\\yasho\\OneDrive\\Desktop\\ragify_docs_2.0\\src\\ragify_docs\\main.py"
      ],
      "env": {}
    }
  }
}
```

3. Completely **Restart Claude Desktop**. If correctly parsed, a 🔌 **Hammer Icon** will materialize inside your message submission console tray!

---

## 📖 Usage Guide & Example Prompts

Once integrated, you do not need to format your query manually. Simply prompt the AI agent with a URL and a specific engineering problem:

- _"Scrape `https://yashodeep.me` and tell me what the Anuvadak project is."_
- _"Look up the framework guidelines at `https://example.com` and show me how to initialize its basic auth route."_

---
