Metadata-Version: 2.4
Name: graphmind-rj
Version: 0.1.1
Summary: Knowledge graph builder for code and product assets with token optimization for LLMs
Author-email: Ramesh J <ramesh@example.com>
Maintainer: Ramesh J
License-Expression: MIT
Project-URL: Homepage, https://github.com/rameshj/graphmind-rj
Project-URL: Repository, https://github.com/rameshj/graphmind-rj.git
Project-URL: Bug Tracker, https://github.com/rameshj/graphmind-rj/issues
Project-URL: Documentation, https://github.com/rameshj/graphmind-rj#readme
Keywords: knowledge-graph,code-analysis,llm,ast,context,token-optimization
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Build Tools
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: networkx>=3.2
Requires-Dist: fastapi>=0.111.0
Requires-Dist: uvicorn>=0.30.0
Requires-Dist: pydantic>=2.7.0
Requires-Dist: sqlalchemy>=2.0.30
Requires-Dist: python-docx>=1.1.2
Requires-Dist: python-multipart>=0.0.9
Provides-Extra: dev
Requires-Dist: pytest>=8.2.0; extra == "dev"
Requires-Dist: httpx>=0.27.0; extra == "dev"
Dynamic: requires-python

# graphmind-rj

> **Knowledge graph builder for code and product assets — with smart token optimization for LLMs.**
>
> Built by **Ramesh J** · [PyPI](https://pypi.org/project/graphmind-rj/) · Python 3.10+

[![PyPI version](https://img.shields.io/pypi/v/graphmind-rj)](https://pypi.org/project/graphmind-rj/)
[![Python](https://img.shields.io/pypi/pyversions/graphmind-rj)](https://pypi.org/project/graphmind-rj/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

---

## What is graphmind-rj?

When you use AI assistants like Claude, ChatGPT, or Copilot to work on your codebase, they need **context** — which files exist, what functions do, how modules relate to each other. The problem:

- Sending full files = **too many tokens = expensive and slow**
- Sending random snippets = **AI gives wrong answers**
- Reloading everything every query = **wasteful and repetitive**

**graphmind-rj solves this.** It:

1. Reads your codebase once and builds a **knowledge graph** — a map of every function, class, table, and document and how they connect
2. Stores that graph in a local SQL database
3. On every LLM query, returns **only the context that matters** for that specific task — 60–80% fewer tokens, same quality answers

---

## Features

| Feature | Description |
|---|---|
| 🔍 Multi-language extraction | Python (AST), SQL (schemas), DOCX, Markdown |
| 📊 Knowledge graph | NetworkX graph with community detection |
| 🚀 REST API | FastAPI backend with 6 endpoints |
| 💾 SQL persistence | SQLAlchemy + SQLite — full run history |
| 🧠 Token optimization | 3-tier smart context (graph → snippets → full files) |
| 📋 Prompt templates | Task-specific formats for bug fix, feature, refactor, review |
| 🖥️ React dashboard | Frontend for viewing runs, uploading files, browsing history |
| ⚡ Incremental runs | SHA-256 hashing skips unchanged files |

---

## Installation

```bash
pip install graphmind-rj
```

That's it. No Docker. No database server. Works on Windows, macOS, Linux.

### Verify Installation

```bash
graphmind --help
```

---

## End-to-End Tutorial

This walkthrough covers a real scenario: **you have a codebase and want to use AI to debug and add features without blowing your token budget.**

---

### Step 1 — Start the API Server

```bash
graphmind-api
```

Expected output:

```
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
```

Leave this terminal open. Open a **new terminal** for the next steps.

Health check:

```bash
curl http://localhost:8000/health
# → {"status": "ok"}
```

---

### Step 2 — Analyse Your Project

```bash
graphmind run /path/to/your/project
```

Or from the current directory:

```bash
cd /path/to/your/project
graphmind run .
```

Output:

```
Scanning files...
  Found 32 Python files, 4 SQL files, 6 Markdown files
Extracting nodes and edges...
  Python AST:   89 functions, 22 classes
  SQL schemas:  11 tables, 34 columns
  Markdown:     18 headings, 45 concept links
Building graph...  Nodes: 142  Edges: 198
Detecting communities...  Communities: 7
Exporting...
  graphmind-out/graph.json
  graphmind-out/graph.html
  graphmind-out/GRAPH_REPORT.md
  graphmind-out/graphmind.db
Done in 3.2s
```

Creates `graphmind-out/` with:

| File | What it is |
|---|---|
| `graph.json` | Full knowledge graph as JSON |
| `graph.html` | Human-readable HTML table view |
| `GRAPH_REPORT.md` | Markdown summary |
| `graphmind.db` | SQLite database with all run data |
| `cache/file_hashes.json` | File hash cache for incremental runs |

---

### Step 3 — Run via REST API (Alternative to CLI)

```bash
curl -X POST http://localhost:8000/api/run \
  -H "Content-Type: application/json" \
  -d '{"path": "/path/to/your/project", "full": false}'
```

- `"full": false` — incremental (skip unchanged files)
- `"full": true` — force full rescan

Response:

```json
{
  "run_id": 1,
  "files": 42,
  "words": 15600,
  "nodes": 142,
  "edges": 198,
  "communities": 7,
  "out_dir": "graphmind-out"
}
```

**Save the `run_id`** — you need it in later steps.

---

### Step 4 — Upload Standalone Files

For design docs, migrations, specs — upload them directly:

```bash
curl -F "files=@architecture.md" \
     -F "files=@schema.sql" \
     -F "files=@requirements.docx" \
     http://localhost:8000/api/upload
```

Response:

```json
{
  "folder": "graphmind-out/uploads/20260408_143021",
  "files_saved": [
    "graphmind-out/uploads/20260408_143021/architecture.md",
    "graphmind-out/uploads/20260408_143021/schema.sql"
  ]
}
```

Then run the pipeline on that folder:

```bash
curl -X POST http://localhost:8000/api/run \
  -d '{"path": "graphmind-out/uploads/20260408_143021", "full": true}'
```

---

### Step 5 — View Run History

```bash
curl http://localhost:8000/api/runs
```

Response:

```json
[
  {
    "id": 2,
    "target_path": "/path/to/project",
    "files": 42,
    "nodes": 142,
    "edges": 198,
    "communities": 7,
    "created_at": "2026-04-08T14:30:21"
  }
]
```

---

### Step 6 — Inspect the Graph

```bash
curl http://localhost:8000/api/runs/1/graph
```

Returns all nodes and edges:

```json
{
  "nodes": [
    {"id": "authenticate_user", "kind": "function", "source_file": "auth.py"},
    {"id": "users", "kind": "table", "source_file": "schema.sql"}
  ],
  "edges": [
    {"source": "authenticate_user", "target": "check_password", "relation": "calls"}
  ]
}
```

---

### Step 7 — Get Optimized Context for an LLM (The Core Feature)

```bash
curl -X POST http://localhost:8000/api/context-pack \
  -H "Content-Type: application/json" \
  -d '{
    "run_id": 1,
    "task_type": "bug_fix",
    "query": "Why is user authentication failing after the latest deployment?",
    "token_budget": 6000,
    "include_artifacts": true
  }'
```

| Field | Type | Description |
|---|---|---|
| `run_id` | int | Which run to use as source |
| `task_type` | string | `bug_fix`, `feature`, `refactor`, `test`, `review`, `optimize` |
| `query` | string | Your specific question |
| `token_budget` | int | Max tokens (default 8000) |
| `include_artifacts` | bool | Include cached summaries |

Response:

```json
{
  "tier": "snippets",
  "graph_summary": {
    "nodes_count": 142,
    "edges_count": 198,
    "communities": 7,
    "top_hubs": ["UserService", "AuthModule", "DatabaseLayer"],
    "summary": "Graph with 142 nodes, 7 communities"
  },
  "code_snippets": [
    {
      "file_path": "auth.py",
      "start_line": 45,
      "end_line": 65,
      "content": "def authenticate_user(username, password):\n    ...",
      "reason": "function_definition"
    }
  ],
  "prompt_template": {
    "task_type": "bug_fix",
    "sections": ["## Problem", "## Current Behavior", "## Expected Behavior", "## Relevant Code", "## Proposed Fix"]
  },
  "total_tokens": 2380,
  "recommended_next_steps": [
    "Run the failing test to reproduce",
    "Add breakpoints in the shown code",
    "Check git blame for recent changes"
  ]
}
```

**How to use this response:**

1. Paste `graph_summary` as the codebase overview to your AI
2. Paste `code_snippets` as the relevant code
3. Use `prompt_template.sections` to structure your question
4. Follow `recommended_next_steps` for guided debugging

**Result: 2,380 tokens instead of 15,000** — same quality answer.

---

### Step 8 — Task Types Reference

| Task | Use for | Recommended budget |
|---|---|---|
| `bug_fix` | Crashes, wrong output, failing tests | 6,000 |
| `feature` | New endpoints, modules, integrations | 8,000–10,000 |
| `refactor` | Reducing complexity, improving patterns | 8,000 |
| `test` | Writing unit/integration tests | 4,000 |
| `review` | PR reviews, security audits | 12,000 |
| `optimize` | Slow queries, memory, bottlenecks | 6,000 |

---

### Step 9 — Use the React Dashboard

```bash
cd frontend
npm install
npm run dev
```

Open: http://localhost:5173

The dashboard provides:
- **Run panel** — enter a path and click Run
- **Upload panel** — drag and drop `.md`, `.docx`, `.sql` files
- **History panel** — browse all past runs with metrics

---

## How Token Optimization Works

graphmind-rj uses a **3-tier retrieval system**:

```
Query arrives
     │
     ▼
Tier 1: Graph Summary (~500 tokens)
     → node count, communities, top hubs
     → enough for architecture questions
     │
     ▼  (query contains: debug, bug, fix, error, refactor...)
Tier 2: Code Snippets (~1,500–3,000 tokens)
     → function definitions from key files
     → enough for most bug fixes and feature planning
     │
     ▼  (complex tasks only: full refactor, deep review)
Tier 3: Full Files (~3,000+ tokens)
     → complete file content
```

**Token Budget Allocation:**

| Priority | Files | Budget |
|---|---|---|
| CRITICAL | Recently changed files | 50% |
| HIGH | Direct dependencies | 30% |
| MEDIUM | Tests, related modules | 15% |
| LOW | Docs, configs | 5% |

---

## File Support

| Extension | What gets extracted |
|---|---|
| `.py` | Functions, classes, methods, imports |
| `.sql` | Tables, columns (CREATE TABLE) |
| `.md` | Headings, paragraphs, concept links |
| `.docx` | Headings, paragraphs (sensitive info redacted) |

---

## Token Savings — Real Numbers

| Scenario | Without | With graphmind-rj | Saved |
|---|---|---|---|
| Bug fix in 40-file project | ~12,000 tokens | ~2,400 tokens | **80%** |
| Feature in 80-file project | ~25,000 tokens | ~5,000 tokens | **80%** |
| Code review of 10 files | ~8,000 tokens | ~3,200 tokens | **60%** |
| Architecture question | ~10,000 tokens | ~500 tokens | **95%** |

---

## Use with VS Code / Cursor

1. Run `graphmind-api` in a background terminal
2. Run `graphmind run .` on your project once per session
3. Before asking AI anything, call `POST /api/context-pack` with your question
4. Paste the response into Copilot Chat, Claude, or Cursor
5. Ask your question — AI now has exactly the right context

---

## API Endpoints Quick Reference

| Method | Endpoint | Purpose |
|---|---|---|
| GET | `/health` | Health check |
| POST | `/api/run` | Run pipeline on a directory |
| POST | `/api/upload` | Upload `.md`, `.docx`, `.sql` files |
| GET | `/api/runs` | List run history |
| GET | `/api/runs/{run_id}/graph` | Get graph for a run |
| POST | `/api/context-pack` | Get token-optimized context for LLM |

---

## Project Structure

```
graphmind-rj/
├── graphmind/
│   ├── api.py                  ← FastAPI app (6 endpoints)
│   ├── cli.py                  ← CLI entry point
│   ├── pipeline.py             ← detect → extract → build → export
│   ├── models.py               ← Node, Edge, ContextPack data models
│   ├── db.py                   ← SQLAlchemy ORM
│   ├── cache.py                ← File hashing + ArtifactCache
│   ├── context_budget.py       ← Token allocation engine
│   ├── retrieval_planner.py    ← 3-tier context retrieval
│   ├── prompt_templates.py     ← Task-specific LLM formats
│   ├── extractors/
│   │   ├── python_ast.py       ← Python AST extraction
│   │   ├── sql_schema.py       ← SQL schema extraction
│   │   ├── text_semantic.py    ← Markdown extraction
│   │   └── docx_semantic.py    ← DOCX extraction
│   ├── graph/
│   │   ├── builder.py          ← NetworkX graph
│   │   └── analytics.py        ← Communities, hubs
│   └── exporters/
│       ├── json_exporter.py
│       └── html_exporter.py
├── frontend/                   ← React + Vite dashboard
└── tests/
```

---

## Development

```bash
git clone https://github.com/rameshj/graphmind-rj.git
cd graphmind-rj
pip install -e ".[dev]"
pytest tests/ -v
```

---

## License

MIT — See [LICENSE](LICENSE)

## Author

**Ramesh J**

## Links

- **Install**: `pip install graphmind-rj`
- **PyPI**: https://pypi.org/project/graphmind-rj/
- **Issues**: https://github.com/rameshj/graphmind-rj/issues

## Quick Start

### Start Backend API

```bash
graphmind-api
# API running on http://localhost:8000
```

### Run Pipeline on Directory

```bash
graphmind run .
```

### Upload Files

```bash
curl -F "files=@file.md" -F "files=@schema.sql" http://localhost:8000/api/upload
```

## Architecture

### Backend Modules

- **api.py**: FastAPI service with 6+ endpoints (run, upload, history, graph, context-pack)
- **cli.py**: Command-line interface
- **pipeline.py**: Full orchestration (detect → extract → build → cluster → export)
- **extractors/**: Multi-language extraction registry
  - `python_ast.py`: Functions, classes, methods (Python)
  - `sql_schema.py`: Tables, columns, constraints (SQL)
  - `docx_semantic.py`: Structure and concepts (DOCX)
  - `text_semantic.py`: Headings and paragraphs (Markdown)
- **graph/**: Graph building and analysis
  - `builder.py`: NetworkX graph construction
  - `analytics.py`: Community detection, hub ranking
- **db.py**: SQLAlchemy ORM (RunRecord, NodeRecord, EdgeRecord)
- **context_budget.py**: Token allocation by priority tier
- **retrieval_planner.py**: Tiered context retrieval (graph → snippets → full files)
- **prompt_templates.py**: LLM-optimized templates for bug_fix, feature, refactor, test, review

### Frontend

- React + Vite dashboard for run management and visualization
- Located in `frontend/` directory

## API Endpoints

| Method | Endpoint | Purpose |
|--------|----------|---------|
| GET | `/health` | Health check |
| POST | `/api/run` | Run pipeline on directory |
| POST | `/api/upload` | Upload .md, .docx, .sql files |
| GET | `/api/runs` | List run history |
| GET | `/api/runs/{run_id}/graph` | Get graph for specific run |
| POST | `/api/context-pack` | Get optimized context for LLM |

## Example: Get Optimized Context

```bash
curl -X POST http://localhost:8000/api/context-pack \
  -H "Content-Type: application/json" \
  -d '{
    "run_id": 1,
    "task_type": "bug_fix",
    "query": "Why is authentication failing?",
    "token_budget": 6000
  }'
```

Response includes graph summary, code snippets, and prompt template.

## Token Savings

- **Graph Summary Tier**: ~500 tokens - graph structure only
- **Snippets Tier**: ~1500-3000 tokens - changed files + dependencies
- **Full Files Tier**: ~3000+ tokens - complete context (fallback)

Smart expansion prevents unnecessary token usage. Most queries stay in Tier 1-2.

## Development

### Install for Development

```bash
git clone https://github.com/rameshj/graphmind.git
cd graphmind
pip install -e ".[dev]"
```

### Run Tests

```bash
pytest tests/
```

### Build Documentation

See [PUBLISH.md](PUBLISH.md) for PyPI publishing guide.

## Configuration

Create `graphmind.toml` or `graphmind.yaml` to customize:

```yaml
# Output directory
out_dir: graphmind-out

# Security settings
redact_secrets: true
redact_emails: true

# Extraction policy
extract_comments: true
extract_docstrings: true

# Graph building
min_edge_confidence: 0.5
```

## License

MIT - See LICENSE file

## Author

Ramesh J

## Support

- 📖 [GitHub](https://github.com/rameshj/graphmind)
- 🐛 [Issue Tracker](https://github.com/rameshj/graphmind/issues)
- 💬 [Discussions](https://github.com/rameshj/graphmind/discussions)
```

Backend runs on http://localhost:8000.

### 2) Start frontend

```bash
cd ramesh-graphmind/frontend
npm install
npm run dev
```

Frontend runs on http://localhost:5173.

## SQL data model

The backend stores:

1. runs: each pipeline execution summary
2. nodes: graph nodes linked to a run
3. edges: graph edges linked to a run

Database file is created at graphmind-out/graphmind.db.

## Ingestion behavior

- .md and .txt style docs: heading-based concept extraction
- .docx: paragraph and heading extraction using python-docx
- .sql: schema extraction from CREATE TABLE statements

## What to build next

1. Add auth and role-based access
2. Add migration tooling (Alembic)
3. Add richer SQL parsing for foreign keys and joins
4. Add graph diff views between runs
