Metadata-Version: 2.4
Name: mosaicx
Version: 1.1.0
Summary: Medical cOmputational Suite for Advanced Intelligent eXtraction
Project-URL: Homepage, https://github.com/LalithShiyam/MOSAICX
Project-URL: Repository, https://github.com/LalithShiyam/MOSAICX
Project-URL: Documentation, https://github.com/LalithShiyam/MOSAICX#readme
Project-URL: Bug Tracker, https://github.com/LalithShiyam/MOSAICX/issues
Author-email: Lalith Kumar Shiyam Sundar <lalith.shiyam@med.uni-muenchen.de>
License: DUAL LICENSING NOTICE
        ====================
        
        MOSAICX is dual-licensed under the terms of both the GNU Affero General Public License v3.0 (AGPL-3.0) and a Commercial License.
        
        OPEN SOURCE LICENSE
        ===================
        
        This software is available under the GNU Affero General Public License v3.0 (AGPL-3.0).
        
        Under this license, you are free to use, modify, and distribute this software, provided that:
        - Any derivative work or application that uses this software must also be open-sourced under AGPL-3.0
        - If you run this software on a server and provide it as a service, you must make the complete source code of your application (including modifications) available to your users
        - You must include this license notice and copyright information in all copies
        
        For the complete AGPL-3.0 license terms, see LICENSE-AGPL-3.0.txt
        
        COMMERCIAL LICENSE
        ==================
        
        If you wish to use this software in a commercial product or service without the open-source requirements of AGPL-3.0, you must obtain a commercial license.
        
        Commercial licenses are available from:
        
            Zenta GmbH
            
            For commercial licensing inquiries, please contact:
            Email: info@zenta.solutions
            Subject: MOSAICX Commercial License Request
        
        Commercial licensing allows you to:
        - Use this software in proprietary applications
        - Distribute applications containing this software without open-source obligations
        - Customize and modify the software without sharing changes
        - Receive commercial support and maintenance
        
        COPYRIGHT AND ATTRIBUTION
        ==========================
        
        Copyright (c) 2024 DIGITX Lab, Department of Radiology, LMU Munich University Hospital
        Developed by Lalith Kumar Shiyam Sundar, PhD
        
        Commercial licensing managed by Zenta GmbH
        
        IMPORTANT NOTICE
        ================
        
        By using this software, you agree to comply with the terms of one of the above licenses.
        If you are unsure which license applies to your use case, please contact Zenta GmbH for clarification.
License-File: LICENSE
Keywords: extraction,llm,medical,nlp,pdf,radiology
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Healthcare Industry
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.11
Requires-Dist: click>=8.1.0
Requires-Dist: docling>=2.0.0
Requires-Dist: dspy-ai>=2.4.0
Requires-Dist: httpx>=0.25.0
Requires-Dist: instructor>=1.0.0
Requires-Dist: ollama>=0.3.0
Requires-Dist: openai>=1.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: python-cfonts>=1.0.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: reportlab>=4.4.4
Requires-Dist: requests>=2.31.0
Requires-Dist: rich-click>=1.8.0
Requires-Dist: rich>=13.0.0
Requires-Dist: typing-extensions>=4.8.0
Provides-Extra: dev
Requires-Dist: black>=23.7.0; extra == 'dev'
Requires-Dist: isort>=5.12.0; extra == 'dev'
Requires-Dist: mypy>=1.5.0; extra == 'dev'
Requires-Dist: pre-commit>=3.3.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest>=7.4.0; extra == 'dev'
Requires-Dist: ruff>=0.0.280; extra == 'dev'
Description-Content-Type: text/markdown

<div align="center">
  <img src="assets/mosaicx_logo.png" alt="MOSAICX Logo" width="800"/>
</div>

<p align="center">
  <a href="https://pypi.org/project/mosaicx/"><img alt="PyPI" src="https://img.shields.io/pypi/v/mosaicx.svg?label=PyPI&style=flat-square&logo=python&logoColor=white&color=bd93f9"></a>
  <a href="https://www.python.org/downloads/"><img alt="Python" src="https://img.shields.io/badge/Python-3.11%2B-50fa7b?style=flat-square&logo=python&logoColor=white"></a>
  <a href="https://www.gnu.org/licenses/agpl-3.0"><img alt="License" src="https://img.shields.io/badge/License-AGPL--3.0-ff79c6?style=flat-square&logo=gnu&logoColor=white"></a>
  <a href="https://pepy.tech/project/mosaicx"><img alt="Downloads" src="https://img.shields.io/pepy/dt/mosaicx?style=flat-square&color=8be9fd&label=Downloads"></a>
  <a href="https://pydantic.dev"><img alt="Pydantic v2" src="https://img.shields.io/badge/Pydantic-v2-ffb86c?style=flat-square&logo=pydantic&logoColor=white"></a>
  <a href="https://ollama.ai"><img alt="Ollama Compatible" src="https://img.shields.io/badge/Ollama-Compatible-6272a4?style=flat-square&logo=ghost&logoColor=white"></a>
  <a href="mailto:lalith@zenta.solutions"><img alt="Commercial License" src="https://img.shields.io/badge/Commercial%20Use-Contact%20Zenta-orange?style=flat-square&logo=data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iOTIiIGhlaWdodD0iOTIiIHZpZXdCb3g9IjAgMCA5MiA5MiIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPGNpcmNsZSBjeD0iNDUuODk2NiIgY3k9IjcuOTEyOTYiIHI9IjYuMjkxNzEiIHRyYW5zZm9ybT0icm90YXRlKDkwIDQ1Ljg5NjYgNy45MTI5NikiIGZpbGw9IiNFNkU0REUiIHN0cm9rZT0iI0U2RTRERSIgc3Ryb2tlLXdpZHRoPSIzLjI0MjUxIi8+CjxjaXJjbGUgY3g9IjgzLjg3ODEiIGN5PSIyNi45MDQyIiByPSI2LjI5MTcxIiB0cmFuc2Zvcm09InJvdGF0ZSg5MCA4My44NzgxIDI2LjkwNDIpIiBmaWxsPSIjRTZFNERFIiBzdHJva2U9IiNFNkU0REUiIHN0cm9rZS13aWR0aD0iMy4yNDI1MSIvPgo8Y2lyY2xlIGN4PSI3LjkxMzIxIiBjeT0iMjYuOTA0MiIgcj0iNi4yOTE3MSIgdHJhbnNmb3JtPSJyb3RhdGUoOTAgNy45MTMyMSAyNi45MDQyKSIgZmlsbD0iI0U2RTRERSIgc3Ryb2tlPSIjRTZFNERFIiBzdHJva2Utd2lkdGg9IjMuMjQyNTEiLz4KPGNpcmNsZSBjeD0iNy45MTMyMSIgY3k9IjQ1Ljg5NjQiIHI9IjYuMjkxNzEiIHRyYW5zZm9ybT0icm90YXRlKDkwIDcuOTEzMjEgNDUuODk2NCkiIGZpbGw9IiNFNkU0REUiIHN0cm9rZT0iI0U2RTRERSIgc3Ryb2tlLXdpZHRoPSIzLjI0MjUxIi8+CjxjaXJjbGUgY3g9IjY0Ljg4NjgiIGN5PSI4My44Nzg4IiByPSI2LjI5MTcxIiB0cmFuc2Zvcm09InJvdGF0ZSg5MCA2NC44ODY4IDgzLjg3ODgpIiBmaWxsPSIjRTZFNERFIiBzdHJva2U9IiNFNkU0REUiIHN0cm9rZS13aWR0aD0iMy4yNDI1MSIvPgo8Y2lyY2xlIGN4PSIyNi45MDQ0IiBjeT0iODMuODc4OCIgcj0iNi4yOTE3MSIgdHJhbnNmb3JtPSJyb3RhdGUoOTAgMjYuOTA0NCA4My44Nzg4KSIgZmlsbD0iI0U2RTRERSIgc3Ryb2tlPSIjRTZFNERFIiBzdHJva2Utd2lkdGg9IjMuMjQyNTEiLz4KPGNpcmNsZSBjeD0iNy45MTMyMSIgY3k9IjY0Ljg4NzYiIHI9IjYuMjkxNzEiIHRyYW5zZm9ybT0icm90YXRlKDkwIDcuOTEzMjEgNjQuODg3NikiIGZpbGw9IiNFNkU0REUiIHN0cm9rZT0iI0U2RTRERSIgc3Ryb2tlLXdpZHRoPSIzLjI0MjUxIi8+CjxyZWN0IHg9IjkwLjE2OTgiIHk9IjM5LjYwNDciIHdpZHRoPSIzMS41NzQ1IiBoZWlnaHQ9IjEyLjU4MzQiIHJ4PSI2LjI5MTcxIiB0cmFuc2Zvcm09InJvdGF0ZSg5MCA5MC4xNjk4IDM5LjYwNDcpIiBmaWxsPSIjRTZFNERFIiBzdHJva2U9IiNFNkU0REUiIHN0cm9rZS13aWR0aD0iMy4yNDI1MSIvPgo8cmVjdCB4PSI3MS4xNzg1IiB5PSIxLjYyMTI2IiB3aWR0aD0iMzEuNTc0NSIgaGVpZ2h0PSIxMi41ODM0IiByeD0iNi4yOTE3MSIgdHJhbnNmb3JtPSJyb3RhdGUoOTAgNzEuMTc4NSAxLjYyMTI2KSIgZmlsbD0iI0U2RTRERSIgc3Ryb2tlPSIjRTZFNERFIiBzdHJva2Utd2lkdGg9IjMuMjQyNTEiLz4KPHJlY3QgeD0iNTIuMTg4MyIgeT0iNTguNTk1OSIgd2lkdGg9IjMxLjU3NDUiIGhlaWdodD0iMTIuNTgzNCIgcng9IjYuMjkxNzEiIHRyYW5zZm9ybT0icm90YXRlKDkwIDUyLjE4ODMgNTguNTk1OSkiIGZpbGw9IiNFNkU0REUiIHN0cm9rZT0iI0U2RTRERSIgc3Ryb2tlLXdpZHRoPSIzLjI0MjUxIi8+CjxyZWN0IHg9IjMzLjE5NjEiIHk9IjIyLjE5NTUiIHdpZHRoPSIzMS41NzQ1IiBoZWlnaHQ9IjEyLjU4MzQiIHJ4PSI2LjI5MTcxIiB0cmFuc2Zvcm09InJvdGF0ZSg5MCAzMy4xOTYxIDIyLjE5NTUpIiBmaWxsPSIjRTZFNERFIiBzdHJva2U9IiNFNkU0REUiIHN0cm9rZS13aWR0aD0iMy4yNDI1MSIvPgo8cmVjdCB4PSIzMy4xOTYxIiB5PSIxLjYyMTI2IiB3aWR0aD0iMzEuNTc0NSIgaGVpZ2h0PSIxMi41ODM0IiByeD0iNi4yOTE3MSIgdHJhbnNmb3JtPSJyb3RhdGUoOTAgMzMuMTk2MSAxLjYyMTI2KSIgZmlsbD0iI0U2RTRERSIgc3Ryb2tlPSIjRTZFNERFIiBzdHJva2Utd2lkdGg9IjMuMjQyNTEiLz4KPHJlY3QgeD0iMTQuMjA0OSIgeT0iMzkuNjA0NyIgd2lkdGg9IjMxLjU3NDUiIGhlaWdodD0iMTIuNTgzNCIgcng9IjYuMjkxNzEiIHRyYW5zZm9ybT0icm90YXRlKDkwIDE0LjIwNDkgMzkuNjA0NykiIGZpbGw9IiNFNkU0REUiIHN0cm9rZT0iI0U2RTRERSIgc3Ryb2tlLXdpZHRoPSIzLjI0MjUxIi8+Cjwvc3ZnPgo="></a>
</p>

<p align="center">
  <strong>Developed by the <a href="https://www.linkedin.com/company/digitx-lmu/">DIGIT-X Lab</a> at LMU Munich University</strong><br>
  <em>Quietly ambitious about the hard things</em>
</p>

---

## 🧬 **Structure first. Insight follows.**

Medical data is inherently complex, unstructured, and heterogeneous. Before we can unlock meaningful patterns, predict outcomes, or enable clinical decision support, we must first impose order on chaos. **MOSAICX** embodies this fundamental principle: **structured data is the prerequisite for knowledge discovery**.

In healthcare, unstructured documents—radiology reports, clinical notes, pathology summaries—contain critical information locked in narrative text. MOSAICX transforms this chaos into validated, machine-readable structures using AI-driven schema generation and extraction pipelines. Only when data is properly structured can we apply advanced analytics, machine learning, and knowledge graphs to generate actionable insights.

**Core Capabilities:**
- 🔬 **Schema Generation**: Transform natural language descriptions into validated Pydantic models
- 📄 **Document Extraction**: Convert PDFs and clinical documents to structured JSON using generated schemas  
- 📊 **Clinical Summarization**: Generate timeline-based summaries of radiology reports with standardized outputs
- 🏥 **Privacy-First**: Process sensitive medical data locally using Ollama-compatible LLMs
- 🎯 **Production-Ready**: Robust error handling, validation, and reproducible outputs

> *Powered by local LLMs via **Ollama**, PDF processing via **Docling**, and strict validation via **Pydantic v2***

---

## 🚀 **Installation & Setup**

### System Requirements
- **Python**: 3.11+ (3.12 recommended)
- **Operating System**: macOS, Linux, Windows (with WSL2)
- **Memory**: 16GB RAM minimum, 32GB recommended
- **Storage**: 10GB free space for models

### Step 1: Install Ollama
```bash
# macOS/Linux (automatic installation)
curl -fsSL https://ollama.com/install.sh | sh

# Windows: Download from https://ollama.com/download/windows

# Start Ollama service
ollama serve
```

### Step 2: Install MOSAICX
```bash
# Using pip (standard)
pip install mosaicx

# Using uv (faster dependency resolution)
uv add mosaicx

# Using pipx (isolated installation)
pipx install mosaicx

# Development installation
git clone https://github.com/LalithShiyam/MOSAICX.git
cd MOSAICX
pip install -e .
```

### Step 3: Download Required Models
```bash
# Default model (recommended for most use cases)
ollama pull gpt-oss:120b

# Alternative models
ollama pull llama3.1:8b-instruct    # Smaller, faster
ollama pull qwen2.5:7b-instruct     # Good balance
ollama pull deepseek-r1:7b          # Reasoning model

# Verify installation
mosaicx --version
```



### Step 4: Quick Test
```bash
# Test connection to Ollama
mosaicx generate --desc "Simple patient record with name and age" --class-name TestModel
```

---

## 🔬 **Usage Guide**

### Command Overview
MOSAICX provides three main commands with extensive options:

```bash
mosaicx --help                    # Show all commands
mosaicx generate --help           # Schema generation options
mosaicx extract --help            # Document extraction options  
mosaicx summarize --help          # Report summarization options
mosaicx schemas --help            # Schema management options
```

### Default Settings
- **Model**: `gpt-oss:120b` (configurable via `--model`)
- **Temperature**: 
  - Schema generation: `0.2` (balanced creativity)
  - Data extraction: `0.0` (deterministic)
  - Summarization: `0.2` (slight creativity for readability)
- **Base URL**: `http://localhost:11434/v1` (Ollama default)
- **API Key**: `ollama` (Ollama default)


### 1. Schema Generation from Natural Language

Transform clinical requirements into validated Pydantic models:

```bash
# Basic usage (uses defaults)
mosaicx generate \
  --desc "Echocardiography report with patient demographics, LVEF, valve grades, impression"

# Generated Pydantic Model:
```python
from pydantic import BaseModel, Field
from datetime import datetime
from typing import Literal, Optional

class EchocardiographyReport(BaseModel):
    """Echocardiography report with patient demographics, LVEF, valve grades, impression"""
    
    patient_id: str = Field(..., description="Unique patient identifier")
    patient_name: str = Field(..., description="Patient full name")
    date_of_birth: datetime = Field(..., description="Patient date of birth")
    exam_date: datetime = Field(..., description="Date of echocardiogram examination")
    lvef_percent: float = Field(..., ge=0, le=100, description="Left ventricular ejection fraction (%)")
    mitral_valve_grade: Literal["Normal", "Mild", "Moderate", "Severe"] = Field(
        ..., description="Mitral valve regurgitation severity"
    )
    aortic_valve_grade: Literal["Normal", "Mild", "Moderate", "Severe"] = Field(
        ..., description="Aortic valve stenosis/regurgitation severity"
    )
    tricuspid_valve_grade: Literal["Normal", "Mild", "Moderate", "Severe"] = Field(
        ..., description="Tricuspid valve regurgitation severity"
    )
    clinical_impression: str = Field(..., min_length=10, description="Cardiologist's clinical impression")
```

**Advanced usage with custom settings:**
```bash
mosaicx generate \
  --desc "Complete blood count with patient ID, test date, hemoglobin, hematocrit, WBC count, differential counts, and reference ranges" \
  --class-name CBCReport \
  --model llama3.1:8b-instruct \
  --temperature 0.1 \
  --schema-path schemas/cbc_report.py

# Generated Pydantic Model:
```python
from pydantic import BaseModel, Field
from datetime import datetime
from typing import Optional

class CBCReport(BaseModel):
    """Complete blood count with patient ID, test date, hemoglobin, hematocrit, WBC count, differential counts, and reference ranges"""
    
    patient_id: str = Field(..., description="Unique patient identifier")
    test_date: datetime = Field(..., description="Date when CBC test was performed")
    hemoglobin: float = Field(..., ge=0, le=25, description="Hemoglobin level in g/dL")
    hematocrit: float = Field(..., ge=0, le=70, description="Hematocrit percentage")
    wbc_count: float = Field(..., ge=0, description="White blood cell count (thousands/μL)")
    neutrophils_percent: float = Field(..., ge=0, le=100, description="Neutrophils percentage")
    lymphocytes_percent: float = Field(..., ge=0, le=100, description="Lymphocytes percentage")
    monocytes_percent: float = Field(..., ge=0, le=100, description="Monocytes percentage")
    eosinophils_percent: float = Field(..., ge=0, le=100, description="Eosinophils percentage")
    basophils_percent: float = Field(..., ge=0, le=100, description="Basophils percentage")
    hemoglobin_ref_range: str = Field(..., description="Reference range for hemoglobin")
    hematocrit_ref_range: str = Field(..., description="Reference range for hematocrit")
    wbc_ref_range: str = Field(..., description="Reference range for WBC count")
```

**Available Options:**

- `--desc` (required): Natural language description
- `--class-name`: Pydantic class name (default: "GeneratedModel")
- `--model`: LLM model to use (default: "gpt-oss:120b")
- `--temperature`: Sampling temperature 0.0-2.0 (default: 0.2)
- `--schema-path`: Write the generated schema to this file
- `--base-url`: Custom API endpoint
- `--api-key`: Custom API key
- `--debug`: Enable verbose logging

### 2. Document Extraction to Structured Data

Extract structured information from clinical documents:

```bash
# Basic extraction
mosaicx extract \
  --pdf patient_reports/echo_001.pdf \
  --schema EchoReport

# Advanced extraction with custom model
mosaicx extract \
  --pdf "case studies/complex_cardiology_report.pdf" \
  --schema CBCReport_20250925_143022 \
  --model qwen2.5:7b-instruct \
  --save results/structured_data.json
```

Example CLI output (abridged – actual Rich formatting includes colors and panels):

```
📋 Extraction results based on schema: EchoReport

Field                  Extracted Value
patient_id             ECG-001-2025
exam_date              2025-09-15T00:00:00
lvef_percent           55.0
mitral_valve_grade     Mild
aortic_valve_grade     Normal
tricuspid_valve_grade  Normal
clinical_impression    Normal left ventricular systolic function...

📁 Extraction saved
JSON: results/structured_data.json
```

### 3. Clinical Report Summarization

Generate timeline-based summaries from radiology reports:

```bash
# Single patient, multiple reports
mosaicx summarize \
  --report patient_001/ct_baseline.pdf \
  --report patient_001/ct_3month.pdf \
  --report patient_001/ct_6month.pdf \
  --patient P001 \
  --json-out summaries/P001_longitudinal.json

# Process entire directory
mosaicx summarize \
  --dir ./radiology_reports/patient_P001/ \
  --patient P001 \
  --model llama3.1:8b-instruct \
  --temperature 0.1 \
  --json-out P001_summary.json
```

Example CLI output (abridged – actual Rich formatting includes colors and panels):

```
Patient: P001
DOB: —        Sex: —        Updated: 2025-09-25T14:30:22Z

Timeline
Date         Source                    Critical Note
2025-08-01   CT Chest/Abdomen/Pelvis   Baseline study: Multiple pulmonary nodules...
2025-09-15   CT Chest Follow-up        Interval growth: RUL nodule now 12mm...

Overall Summary
Progressive pulmonary nodular disease with interval growth of the RUL lesion and new LLL nodule. [Source: CT Chest Follow-up]
```

---

## 🐍 **Using MOSAICX as a Python Library**

The CLI features are also exposed as pure Python helpers so you can script or integrate them into other services.

```python
from pathlib import Path

from mosaicx import (
    extract_pdf,
    generate_schema,
    summarize_reports,
)

# 1) Generate a Pydantic schema from a plain-language description
schema = generate_schema(
    "Patient vitals with name, heart rate, systolic_bp, diastolic_bp",
    class_name="PatientVitals",
    model="gpt-oss:120b",
)
schema_path = schema.write(Path("schemas/patient_vitals.py"))

# 2) Extract structured data from a PDF using that schema
extraction = extract_pdf(
    pdf_path="tests/datasets/sample_patient_vitals.pdf",
    schema_path=schema_path,
)
payload = extraction.to_dict()

# 3) Summarize one or more clinical reports
summary = summarize_reports(
    paths=["tests/datasets/sample_patient_vitals.pdf"],
    patient_id="demo-patient",
)
```

Example Python output (illustrative values):

```python
payload
{
    "patient_name": "John Doe",
    "heart_rate": 72,
    "systolic_bp": 118,
    "diastolic_bp": 76,
}

summary.overall
'Stable vital signs with normal heart rate and blood pressure. [Source: sample_patient_vitals.pdf]'

summary.timeline[0].model_dump()
{
    "date": None,
    "source": "sample_patient_vitals.pdf",
    "note": "Vitals within normal limits; no acute concerns.",
}
```

All helpers accept optional `model`, `base_url`, and `api_key` arguments; when omitted the defaults mirror the CLI (environment variables first, then local Ollama).

---

## 🎯 **Why Structure Matters in Medical AI**

At the DIGIT-X Lab, we believe that **structure precedes insight**. The proliferation of unstructured medical data—radiology reports, clinical notes, pathology summaries—represents both an opportunity and a challenge. While this data contains rich clinical knowledge, its unstructured nature makes it largely inaccessible to computational analysis. 

Modern healthcare generates exabytes of unstructured text annually, yet most clinical decision support systems can only leverage structured fields from electronic health records. This fundamental disconnect limits our ability to develop robust clinical AI, conduct large-scale outcomes research, or enable personalized medicine approaches.

**MOSAICX addresses this gap by:**
- **Democratizing Data Structuring**: Transforming natural language descriptions into production-ready data schemas without requiring deep technical expertise
- **Enabling Reproducible Extraction**: Converting documents to validated JSON structures that can be reliably processed by downstream ML pipelines
- **Preserving Clinical Context**: Maintaining semantic meaning while imposing computational structure through intelligent schema design
- **Supporting Privacy Requirements**: Processing sensitive medical data locally without external API dependencies

The structured data produced by MOSAICX becomes the foundation for knowledge graphs, longitudinal analysis, cohort studies, and clinical prediction models. **Structure first. Insight follows.**

---

## 🔧 **Advanced Features**

### Schema Registry Management
The schema registry tracks all generated Pydantic models for easy reuse:

```bash
# List all generated schemas with details
mosaicx schemas

# Filter by clinical domain or keywords
mosaicx schemas --description "cardiology"
mosaicx schemas --class-name "Echo"

# Clean up orphaned registry entries (files deleted outside MOSAICX)
mosaicx schemas --cleanup

# Scan and register existing schema files not tracked by registry
mosaicx schemas --scan
```

**Available Schema Registry:**
- **EchoReport_20250925_143022**: Echocardiography report with LVEF and valve assessments
- **CBCReport_20250925_101530**: Complete blood count with differential and references  
- **PathologyReport_20250924_152045**: Surgical pathology with tumor staging and margins

💡 **Tip**: Use schema ID, filename, or file path in extract commands

### Batch Processing
Process multiple documents or directories efficiently:

```bash
# Batch summarization for multiple patients
for patient_dir in ./patients/*/; do
  patient_id=$(basename "$patient_dir")
  mosaicx summarize \
    --dir "$patient_dir" \
    --patient "$patient_id" \
    --json-out "summaries/${patient_id}_summary.json"
done

# Batch extraction using same schema
find ./reports -name "*.pdf" -exec mosaicx extract \
  --pdf {} \
  --schema UniversalLabReport \
  --save "structured_data/{}.json" \;
```

### Custom Model Endpoints
Use alternative LLM providers or local deployments:

```bash
# OpenAI API
mosaicx generate \
  --desc "Pathology report with tumor staging" \
  --base-url https://api.openai.com/v1 \
  --api-key sk-your-openai-key \
  --model gpt-4-turbo

# Local LM Studio
mosaicx extract \
  --pdf report.pdf \
  --schema PathologyReport \
  --base-url http://localhost:1234/v1 \
  --api-key lm-studio \
  --model local-medical-llm

# Custom medical LLM deployment
mosaicx summarize \
  --dir ./radiology_reports/ \
  --base-url https://your-medical-llm.hospital.com/v1 \
  --api-key your-internal-key \
  --model hospital-radiology-model
```

### Environment Variables
Set default values to avoid repetitive command-line options:

```bash
# Set default model and endpoint
export OPENAI_BASE_URL="http://localhost:11434/v1"
export OPENAI_API_KEY="ollama"
export MOSAICX_DEFAULT_MODEL="gpt-oss:120b"

# Now use simplified commands
mosaicx generate --desc "Simple patient record"
mosaicx extract --pdf report.pdf --schema PatientRecord
```

---

## 📋 **Best Practices & Model Selection**

### Recommended Models by Use Case

| Model | Size | Use Case | Memory | Speed | Accuracy |
|-------|------|----------|---------|-------|----------|
| `gpt-oss:120b` | ~120B | Complex schemas, high accuracy | 64GB+ | Slow | ★★★★★ |
| `llama3.1:8b-instruct` | ~8B | Balanced performance | 16GB+ | Fast | ★★★★☆ |
| `qwen2.5:7b-instruct` | ~7B | Batch processing | 12GB+ | Fastest | ★★★☆☆ |
| `deepseek-r1:7b` | ~7B | Reasoning tasks | 16GB+ | Medium | ★★★★☆ |

**Default Model**: `gpt-oss:120b` provides the best accuracy for medical schema generation and extraction tasks.

### Schema Design Guidelines

**✅ Good Schema Design:**
```python
# Descriptive field names with medical terminology
class EchocardiographyReport(BaseModel):
    patient_id: str = Field(..., description="Unique patient identifier")
    exam_date: datetime = Field(..., description="Date of echocardiogram")
    lvef_percent: float = Field(..., ge=0, le=100, description="Left ventricular ejection fraction (%)")
    mitral_valve_grade: Literal["Normal", "Mild", "Moderate", "Severe"] = Field(
        ..., description="Mitral valve regurgitation severity"
    )
    clinical_impression: str = Field(..., min_length=10, description="Cardiologist's interpretation")
```

**❌ Poor Schema Design:**
```python
# Vague field names, no validation, poor descriptions
class Report(BaseModel):
    data: str
    values: list
    result: float
```

### Extraction Optimization

**Document Preparation:**
- Ensure PDFs have searchable text layers (not just scanned images)
- Use OCR preprocessing for scanned documents: `tesseract input.pdf output.pdf`
- Remove password protection from PDFs before processing

**Parameter Tuning:**
- **Temperature 0.0**: Deterministic extraction for consistent results
- **Temperature 0.1-0.2**: Slight variation for creative schema generation
- **Higher models**: Use for complex medical terminology and relationships

**Validation Best Practices:**
- Always review extracted data for clinical accuracy
- Implement post-processing validation against medical standards
- Use enum constraints for standardized medical values
- Set appropriate ranges for numeric clinical measurements

### Production Deployment

**Performance Optimization:**
```bash
# Use quantized models for faster inference
ollama pull llama3.1:8b-instruct-q4_0    # 4-bit quantization

# Process in batches to maximize GPU utilization
# Use parallel processing for independent documents
```

**Error Handling:**
```bash
# Enable debug mode for troubleshooting
mosaicx extract --pdf document.pdf --schema MySchema --debug

# Implement retry logic for production systems
# Validate outputs against clinical standards
# Log failed extractions for manual review
```

---

## 🏥 **Clinical Applications**

### Research & Analytics
- **Cohort Studies**: Structure clinical notes for population-level analysis
- **Outcomes Research**: Extract standardized endpoints from heterogeneous reports
- **Quality Metrics**: Automate clinical quality measure extraction
- **Biomarker Discovery**: Structure pathology and lab reports for analysis

### Clinical Decision Support
- **Risk Stratification**: Extract risk factors into computable formats
- **Care Pathway Optimization**: Structure clinical workflows and outcomes
- **Longitudinal Tracking**: Generate patient timelines from multiple reports
- **Adverse Event Detection**: Structure safety data from clinical narratives

### Operational Excellence
- **Revenue Cycle**: Extract billable procedures and diagnoses
- **Compliance Reporting**: Structure regulatory reporting requirements
- **Care Coordination**: Generate structured handoff summaries
- **Quality Assurance**: Standardize report review workflows

---

## ⚡ **Performance & Scalability**

### Local Processing Benefits
- **Privacy Compliance**: No PHI transmitted to external services
- **Cost Efficiency**: Eliminate per-token API costs for large-scale processing
- **Latency Optimization**: Sub-second processing for typical clinical documents
- **Offline Capability**: Process data in air-gapped environments

### Hardware Recommendations
- **Minimum**: 16GB RAM, modern CPU (M1/M2 Mac, Intel i7/AMD Ryzen 7)
- **Recommended**: 32GB RAM, GPU acceleration (RTX 4080/4090, M2 Max/Ultra)
- **High-throughput**: 64GB+ RAM, multiple GPUs for batch processing

---

## 🔍 **Troubleshooting Guide**

### Installation Issues

**Python Version Compatibility:**
```bash
# Check Python version (requires 3.11+)
python --version

# Install specific Python version if needed
pyenv install 3.12.0
pyenv global 3.12.0
```

**Dependency Conflicts:**
```bash
# Use virtual environment to isolate dependencies
python -m venv mosaicx-env
source mosaicx-env/bin/activate  # macOS/Linux
# mosaicx-env\Scripts\activate    # Windows

pip install mosaicx
```

### Runtime Issues

| Issue | Cause | Solution |
|-------|-------|----------|
| `Connection refused` | Ollama not running | `ollama serve` |
| `Model not found` | Model not downloaded | `ollama pull model-name` |
| `Empty extraction` | Poor model/temperature | Try `gpt-oss:120b` with `--temperature 0.0` |
| `PDF processing error` | Scanned PDF without text | Use OCR: `tesseract input.pdf output.pdf` |
| `Memory error` | Model too large | Use quantized model: `llama3.1:8b-instruct-q4_0` |
| `JSON validation error` | Malformed output | Enable `--debug` and check model output |
| `Schema not found` | Registry out of sync | Run `mosaicx schemas --scan` |

### Debug Mode
Enable verbose logging to diagnose issues:

```bash
# Enable debug for all commands
mosaicx --debug generate --desc "Test schema"
mosaicx extract --pdf document.pdf --schema MySchema --debug
mosaicx summarize --dir ./reports --debug

# Check Ollama status
ollama list              # Show downloaded models
ollama ps               # Show running models
curl http://localhost:11434/api/tags  # API health check
```

### Common Error Messages

**"Schema class 'MySchema' not found"**
```bash
# Check available schemas
mosaicx schemas

# Regenerate if missing
mosaicx generate --desc "Your schema description" --class-name MySchema
```

**"No text extracted from PDF"**
```bash
# Test PDF text extraction
python -c "
from docling.document_converter import DocumentConverter
converter = DocumentConverter()
result = converter.convert('your_file.pdf')
print(result.document.text)
"
```

**"Temperature must be between 0.0 and 2.0"**
```bash
# Fix temperature value
mosaicx generate --desc "Test" --temperature 0.2  # Valid range: 0.0-2.0
```

### Performance Issues

**Slow Processing:**
- Use smaller models: `llama3.1:8b-instruct` instead of `gpt-oss:120b`
- Increase available RAM or use quantized models (`q4_0` suffix)
- Process documents in smaller batches

**High Memory Usage:**
- Close other applications
- Use quantized models
- Process one document at a time

**Inaccurate Results:**
- Use larger, more capable models
- Lower temperature for more deterministic output
- Improve schema descriptions with more specific field definitions
- Review and refine extracted data manually

### Getting Help

**Log Analysis:**
```bash
# Enable maximum verbosity
export MOSAICX_LOG_LEVEL=DEBUG
mosaicx extract --pdf document.pdf --schema MySchema --debug > debug.log 2>&1
```

**System Information:**
```bash
# Gather system info for bug reports
mosaicx --version
python --version
ollama --version
pip show mosaicx
```

For additional support:
- **GitHub Issues**: [Report bugs and feature requests](https://github.com/LalithShiyam/MOSAICX/issues)
- **Research Inquiries**: lalith.shiyam@med.uni-muenchen.de
- **Commercial Support**: lalith@zenta.solutions

---

## 🎓 **From DIGIT-X Lab**

**MOSAICX** is developed by the [DIGIT-X Lab](https://www.linkedin.com/company/digitx-lmu/) at LMU Munich University, a research group focused on digital transformation in radiology and medical imaging. Our mission is to bridge the gap between clinical practice and computational methods through practical, privacy-preserving tools.

**Research Focus Areas:**
- Medical Image Analysis & AI
- Clinical Natural Language Processing  
- Healthcare Data Standardization
- Privacy-Preserving Medical AI
- Radiomics & Quantitative Imaging

**Team**: Led by researchers and clinicians who understand both the technical challenges and clinical requirements of modern healthcare data processing.

*We are quietly ambitious about the hard things.*

---

## 📝 **License & Citation**

MOSAICX is released under the AGPL-3.0 license for academic and open-source use. For commercial applications in healthcare organizations, please contact us for licensing options.

### Citation
```bibtex
@software{mosaicx2025,
  title={MOSAICX: Medical cOmputational Suite for Advanced Intelligent eXtraction},
  author={Sundar, Lalith Kumar Shiyam and DIGIT-X Lab},
  year={2025},
  institution={LMU Munich University},
  url={https://github.com/LalithShiyam/MOSAICX},
  note={Developed at DIGIT-X Lab, Department of Radiology}
}
```

---

## 🤝 **Contributing & Support**

We welcome contributions from the medical informatics and clinical AI communities:

- **Bug Reports**: Submit issues with minimal reproducible examples
- **Feature Requests**: Propose new clinical use cases and requirements  
- **Documentation**: Improve clinical examples and best practices
- **Code Contributions**: Follow our development guidelines and testing requirements

**Contact**: 
- Research Inquiries: [lalith.shiyam@med.uni-muenchen.de](mailto:lalith.shiyam@med.uni-muenchen.de)
- Commercial Licensing: [lalith@zenta.solutions](mailto:lalith@zenta.solutions)
- DIGIT-X Lab: [https://www.linkedin.com/company/digitx-lmu/](https://www.linkedin.com/company/digitx-lmu/)

---

*Built with ❤️ for the medical community by researchers who understand that great clinical AI starts with great data structure.*

**MOSAICX is infrastructure for clinical data**: schema-driven, validated, local, and reproducible. Structure reports once, then reuse the same schemas and summarizers across departments and time—enabling longitudinal analysis, cross-modal integration, and downstream intelligence without sending data to the cloud.
