Production-grade RAG ingestion, knowledge graphs, AI observability, automated compliance, and multi-modal extraction — all in one elegant, open-source platform. No cloud lock-in.
From raw documents to AI-powered retrieval with observability, compliance, and knowledge graphs built in.
PDFs, DOCX, HTML, images, audio, video. Unstructured.io + VLM + Whisper for multi-modal parsing.
Header-aware splitter preserves document hierarchy. Configurable token limits with overlap.
LLM summaries + entity extraction (NER) + PII scanning (GuardRail). Context prepended to every chunk.
BGE-m3 dense vectors + JinaCLIP images + CLAP audio. HNSW index in OpenSearch k-NN.
Entities extracted → Neo4j knowledge graph. Multi-hop graph traversal + hybrid search fusion.
┌────────────────────────────────────────────────────────────────────────────┐ │ SemantixRAG Platform v2.0 │ ├────────────────────────────────────────────────────────────────────────────┤ │ API Gateway (FastAPI) ─── OPA Policy Engine ─── AuthZ & Rate Limiting │ ├────────────────────────────────────────────────────────────────────────────┤ │ Core Pipeline: │ │ Extraction → Chunking → Enrichment → Embedding → Indexing │ │ │ │ │ │ │ │ │ │ VLM/Whisper│ Header │ LLM Summ. │ BGE-m3 │ OpenSearch │ │ │ Multi-modal│ Splitter │ Entity │ JinaCLIP│ Neo4j Graph │ │ │ │ │ PII Scan │ CLAP │ (GraphRAG) │ ├────────────────────────────────────────────────────────────────────────────┤ │ Obsidian │ GuardRail │ GraphRAG │ CostSentinel │ AdminCopilot │ │ Tracing │ PII Detect │ Entity │ Cost Track │ NL Admin │ │ Metrics │ Masking │ KG Write │ Optimize │ Auto Config │ │ Eval │ DSAR │ Graph │ Budget │ Reports │ │ Alerting │ Audit │ Search │ Forecast │ Incident │ └────────────────────────────────────────────────────────────────────────────┘
Every feature designed for production RAG workloads with enterprise governance.
End-to-end distributed tracing for every pipeline stage. RAG quality metrics (faithfulness, MRR, Recall@k), latency histograms, cost attribution, and drift detection.
Automatic entity extraction (spaCy + LLM) → Neo4j knowledge graph. Multi-hop graph traversal fused with vector search via RRF for entity-aware retrieval.
Real-time PII detection via Presidio + regex fallback. Dynamic masking, GDPR DSAR automation (find/delete/export), OPA policy engine, and immutable audit trails.
Extract text from images (VLM), transcribe audio (Whisper), sample video frames. Unified embedding space via JinaCLIP + CLAP for cross-modal retrieval.
Production-grade API with 5 route modules: ingestion, retrieval, compliance, observability, and admin. OpenAPI docs, CORS, async handlers, file upload.
Per-pipeline cost tracking, anomaly detection, optimization recommendations, model routing intelligence, and budget guardrails with hard-stop enforcement.
Natural-language platform administration. "Why is ingestion failing?", "Show PII findings", "Schema status" — managed via conversational interface.
Documents parsed with full structural fidelity. Headers, tables, lists, figures extracted with relationships intact — not flat text.
Dense vector + keyword search fused via Reciprocal Rank Fusion. HNSW ANN with cosine similarity. Configurable top-k and reranking pipeline.
Python Watchdog monitors directories. File changes trigger automatic re-indexing with old chunk deletion. Batch processing with progress reporting.
GitHub Actions workflows for lint, unit tests (41+), integration tests, security scanning (Trivy), Docker build. OpenSearch + Neo4j service containers.
Zero cloud dependencies. Everything runs on your hardware with Docker. Embedding models and LLMs run locally via Ollama or HuggingFace. MIT licensed.
Battle-tested tools for production AI workloads.
Core orchestration & logic
Containerized infrastructure
Vector store + BM25 search
Knowledge graph + GraphRAG
NEWDocument parsing engine
Dense embeddings (1024 dim)
REST API server
NEWMulti-modal VLM + audio
NEWPII detection + policy engine
NEWDashboard & alerting
Data validation & settings
41+ unit & integration tests
NEWGet the full platform running on your machine in minutes.
Join the community building production-grade retrieval systems with open-source tools. No cloud lock-in.