Metadata-Version: 2.4
Name: bibliodash
Version: 4.0.5
Summary: BiblioDash - An AI-powered Python package for bibliometric analysis, scientometrics, literature intelligence, and interactive research visualization.
Author-email: Robin Tomar <itsrobintomar@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/imrobintomar/BiblioDash
Project-URL: Repository, https://github.com/imrobintomar/BiblioDash
Project-URL: Issues, https://github.com/imrobintomar/BiblioDash/issues
Project-URL: Documentation, https://github.com/imrobintomar/BiblioDash/blob/main/docs/USER_GUIDE.md
Keywords: bibliometrics,scientometrics,research analytics,research intelligence,citation analysis,co-citation,bibliographic coupling,network analysis,knowledge graph,systematic review,literature review,AI,RAG,semantic search,dashboard,plotly
Classifier: Development Status :: 4 - Beta
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Scientific/Engineering :: Visualization
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Education
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: dash>=2.14
Requires-Dist: dash-bootstrap-components>=1.5
Requires-Dist: dash-ag-grid>=31
Requires-Dist: plotly>=5.18
Requires-Dist: networkx>=3.0
Requires-Dist: numpy>=1.24
Requires-Dist: rapidfuzz>=3.0
Requires-Dist: pydantic>=2.0
Requires-Dist: requests>=2.28
Requires-Dist: openpyxl>=3.1
Requires-Dist: pycountry>=23
Requires-Dist: geonamescache>=1.6
Requires-Dist: psutil>=5.9
Requires-Dist: rich>=13.0
Requires-Dist: python-dotenv>=1.0
Requires-Dist: waitress>=3.0
Provides-Extra: pdf
Requires-Dist: pymupdf>=1.23; extra == "pdf"
Requires-Dist: pdfplumber>=0.10; extra == "pdf"
Requires-Dist: docling>=2.0; extra == "pdf"
Provides-Extra: semantic
Requires-Dist: sentence-transformers>=2.6; extra == "semantic"
Requires-Dist: faiss-cpu>=1.7; extra == "semantic"
Requires-Dist: scikit-learn>=1.4; extra == "semantic"
Requires-Dist: rank-bm25>=0.2; extra == "semantic"
Provides-Extra: ai
Requires-Dist: ollama>=0.2; extra == "ai"
Provides-Extra: all
Requires-Dist: bibliodash[ai,pdf,semantic]; extra == "all"
Dynamic: license-file

<div align="center">
  <img src="https://raw.githubusercontent.com/imrobintomar/BiblioDash/main/src/Public/BiblioDash-logo-color.png" alt="BiblioDash" width="420" />

  <h3>Research Intelligence. Flowing Insights.</h3>

  <p>
    A local-first platform that transforms scientific literature collections into an<br/>
    interactive bibliometric knowledge base powered by AI, citation analytics, and metadata enrichment.
  </p>

  <p>
    <img src="https://img.shields.io/pypi/v/bibliodash?style=flat-square&color=3B82F6" alt="PyPI version"/>
    <a href="https://pepy.tech/project/bibliodash"><img src="https://static.pepy.tech/badge/bibliodash" alt="Total downloads"/></a>
    <img src="https://img.shields.io/pypi/pyversions/bibliodash?style=flat-square" alt="Python"/>
    <img src="https://img.shields.io/badge/Dash-4.x-008DE4?style=flat-square&logo=plotly&logoColor=white" alt="Dash"/>
    <img src="https://img.shields.io/badge/NetworkX-3.x-orange?style=flat-square" alt="NetworkX"/>
    <img src="https://img.shields.io/badge/SQLite-WAL%20mode-003B57?style=flat-square&logo=sqlite&logoColor=white" alt="SQLite"/>
    <img src="https://img.shields.io/badge/Ollama-local%20AI-black?style=flat-square" alt="Ollama"/>
    <img src="https://img.shields.io/badge/Docker-ready-2496ED?style=flat-square&logo=docker&logoColor=white" alt="Docker"/>
    <img src="https://img.shields.io/badge/status-active-brightgreen?style=flat-square" alt="Status"/>
  </p>
</div>

---

## What is BiblioDash?

**BiblioDash** is a local-first research intelligence platform that transforms collections of scientific literature into an interactive bibliometric knowledge base. It supports multiple data sources, runs entirely on your own hardware, and ships with 15+ analysis modules, AI-powered research intelligence via a local LLM, and multi-user support for team deployments.

**The core workflow:**

```
Import data  →  BiblioDash enriches metadata  →  Explore 15+ analysis modules
```

Supported import formats:

| Format | Source |
|---|---|
| **PDF folder** | Any research PDFs  DOI extracted, CrossRef verified, enriched via 6 open APIs |
| **Scopus CSV** | Direct export from Scopus  all fields imported |
| **BibTeX (.bib)** | Mendeley, Zotero, CrossRef, any reference manager |
| **RIS (.ris)** | Web of Science, PubMed, Mendeley, EndNote |
| **PubMed XML** | Direct PubMed export |

Everything runs locally  no data is sent to any external service.

---

## Architecture

<div align="center">
  <img src="https://raw.githubusercontent.com/imrobintomar/BiblioDash/main/src/Public/BiblioDash_workflow.png" alt="BiblioDash Architecture" width="100%" />
</div>

---

## Why BiblioDash?

- **Local-first:** no data leaves your machine; AI runs via Ollama
- **Multi-user ready:** authentication, per-user project isolation, WAL-mode SQLite
- **One-command install:** `pip install bibliodash && bibliodash`
- **Multiple import formats:** PDF, Scopus CSV, BibTeX, RIS, PubMed XML
- **15+ analysis modules:** publication trends to co-citation networks
- **AI research intelligence:** corpus summary, RAG chat, claim extraction
- **Semantic search:** FAISS vector embeddings over all abstracts
- **Research gap detection:** KMeans clustering + TF-IDF gap scoring
- **Production-ready:** Gunicorn + Docker + docker-compose included

---

## Installation

### Option 1 - PyPI (recommended)

```bash
pip install bibliodash

# First launch:
bibliodash
```

The wizard detects your hardware, creates a workspace at `~/BiblioDash/`, and optionally installs an AI model via Ollama.

### Option 2  With optional features

```bash
# Semantic search, novelty scores, gap detection
pip install "bibliodash[semantic]"

# Local AI features (also install Ollama: https://ollama.com)
pip install "bibliodash[ai]"

# PDF extraction
pip install "bibliodash[pdf]"

# Everything
pip install "bibliodash[all]"
```

## Usage

### Launch

```bash
bibliodash                              # Launch dashboard (setup wizard on first run)
bibliodash --port 9000                  # Custom port
bibliodash --no-browser                 # Skip auto-open
bibliodash --workspace ~/MyResearch     # Custom workspace directory
```

### Import data

```bash
bibliodash import-pdf ~/Papers/         # Import all PDFs from a folder
bibliodash import-scopus export.csv     # Import a Scopus CSV export
```

Or use the **Import Data** page in the dashboard  drag in any path for PDF folder, or upload Scopus CSV, BibTeX, RIS, or PubMed XML files directly.

### Manage projects

```bash
bibliodash new-project "Hypospadias Review 2026"
```

### AI model setup

```bash
bibliodash setup                        # Re-run the setup wizard
bibliodash install-ai                   # Install or change AI model
```


## Analysis Modules

### Overview
Annual, monthly, quarterly, and decade-level publication production. Growth rates (CAGR), cumulative output, moving averages, publication heatmaps, and forecasting  broken down by document type, country, institution, and journal.

### Sources
Identifies the most productive journals and publishers. Applies Bradford's Law to reveal the core zone of journals. Shows journal H-index, citation impact, open-access distribution, and publication timelines.

### Authors
Ranks authors by productivity and citation impact. Computes H-index, G-index, M-index, i10-index, citation velocity, and career trajectory. Maps co-author collaboration networks with Louvain community detection.

### Documents
Lists the most-cited papers in your corpus. Analyses reference lists to find the most frequently co-cited works. Full funding intelligence: funder lollipop chart, category breakdown (Government / Industry / Other), funding trends, and multi-funded paper table.

### Conceptual Structure
Identifies the most frequent keywords and concepts. Builds a Thematic Map  plotting keyword clusters by Callon centrality vs. density  to reveal motor, niche, emerging, and declining themes.

### Social Structure
Maps collaboration at three levels (Author, Institution, and Country) as interactive network graphs with community detection. Renders a choropleth world map of collaboration arcs. Shows a collaboration timeline across years.

### Intellectual Structure
Uncovers the citation backbone of your field. Identifies the most co-cited reference pairs. Builds a citation historiography showing how foundational works influenced one another over time.

### Clustering
Detects research communities using four network types  co-authorship, keyword co-occurrence, bibliographic coupling, and citation networks  each analysed for degree, betweenness, closeness, eigenvector, and PageRank centrality.

### Evidence Quality
Per-paper Evidence Score (0-100) and Reproducibility Score (0-100) computed from nine quality signals: trial registration, ethics approval, informed consent, funding declaration, PubMed indexing, open access, figures, tables, and data availability.

### Duplicate Detection
Detects duplicate papers by exact DOI match and fuzzy title similarity (>=90% token-sort ratio via rapidfuzz). Reports cluster count, duplicate rate, and a full table of suspect pairs.

### Provenance Explorer
Field-level data provenance heatmap showing which enrichment source filled each metadata field. Average confidence score per source and per field.

### Research Timeline
Interactive bubble scatter plot of publications over time  bubble size proportional to citation count, colour-coded by document type. Complementary year x document-type heatmap.

### Semantic Search
Meaning-based search over all paper titles and abstracts using `sentence-transformers` (`all-MiniLM-L6-v2`) and a FAISS cosine-similarity index. Also provides Paper Novelty Scores and Paper Similarity detection.

### Research Gap Detection
Clusters all paper abstracts with KMeans + FAISS embeddings, names each cluster with TF-IDF keywords, and scores it for research gap potential. Renders an interactive bubble chart and ranked gap card list.

### AI Features
Three AI-powered views running via local **Ollama**  zero data leaves your machine:

| View | What it does |
|---|---|
| **AI Summary:** | Corpus-level synthesis from your 50 most-cited abstracts: overall summary, main themes, emerging trends, research gaps, common methods, future directions, clinical implications. Cached per project. |
| **Research Chat:** | Multi-turn RAG chat  retrieves 8 semantically relevant papers per question, injects them as grounded context, returns cited answers with paper title references. |
| **Claim Extraction:** | Per-paper structured extraction for top-50 cited papers: main finding, study design, sample size, key limitation, future work, evidence level. |

---

## How Metadata Enrichment Works

Each imported paper passes through an enrichment waterfall a chain of open academic APIs where each source fills fields the previous one left empty:

```
CrossRef  →  OpenAlex  →  Semantic Scholar  →  PubMed  →  Unpaywall  →  ROR
```

- **CrossRef:** verifies DOI, fills publisher, document type, journal, page range
- **OpenAlex:** adds institutions with ROR IDs, country affiliations, full reference list, open-access status, grants
- **Semantic Scholar:** adds abstract backfill, references, fields of study
- **PubMed:** adds MeSH terms and grant IDs
- **Unpaywall:** adds open-access URL and licence
- **ROR:** resolves institution names to countries

Every field fill is recorded with its source, method, and confidence score in a `field_provenance` table.

---

## Multi-User and Production

BiblioDash ships with full multi-user support:

- **Authentication:** registration and login with PBKDF2-SHA256 password hashing; 30-day persistent sessions
- **First user = admin:**  the first account registered is automatically promoted to admin
- **Per-user project isolation:**  users see their own projects plus shared (public) projects
- **SQLite WAL mode:**  Write-Ahead Logging enables concurrent reads during imports; `busy_timeout = 15 s` prevents lock contention
- **Gunicorn:**  gthread workers (2xCPU+1 x 2 threads), 120 s timeout, automatic worker recycling

```bash
# Scale configuration via environment variables
WEB_CONCURRENCY=8       # number of Gunicorn workers
THREADS_PER_WORKER=4    # threads per worker
WORKER_TIMEOUT=120      # seconds before worker is killed
SECRET_KEY=<hex-string> # required for session signing
```

---

## Tech Stack

| Component | Technology |
|---|---|
| **Dashboard:** | [Dash](https://dash.plotly.com/) 4.x + [Plotly](https://plotly.com/) 5.x |
| **PDF extraction:** | Docling, PyMuPDF, pdfplumber |
| **Metadata enrichment:** | CrossRef, OpenAlex, Semantic Scholar, PubMed, Unpaywall, ROR |
| **Network analysis:** | [NetworkX](https://networkx.org/) 3.x  Louvain, centrality, spring layout |
| **Semantic search:** | sentence-transformers (`all-MiniLM-L6-v2`) + FAISS (`IndexFlatIP`) |
| **Gap detection:** | scikit-learn KMeans + TF-IDF + PCA |
| **Duplicate detection:** | rapidfuzz `token_sort_ratio` |
| **AI features:** | [Ollama](https://ollama.com) local LLM  qwen3 / llama3.1 (hardware-selected) |
| **Storage:** | SQLite WAL mode  normalised warehouse + checksum cache |
| **Auth:** | Flask sessions + werkzeug PBKDF2-SHA256 |
| **Production server:** | Gunicorn (gthread workers) |
| **Containers:** | Docker + docker-compose |
| **Data tables:** | dash-ag-grid, dash-table |
| **Geocoding:** | geonamescache, pycountry |
| **Language:** | Python 3.11+ |

---

## Current Limitations

| Area | Status |
|---|---|
| Author-institution attribution | Paper-level only  cannot isolate a specific co-author's affiliation |
| Citation Half-Life | Not computable  APIs return cumulative counts, not per-year breakdowns |
| Author/Journal co-citation | Not implemented  requires resolving reference metadata per paper |
| Bibliographic coupling | Accuracy bounded by DOI resolution in reference lists |
| Entity deduplication | Institution and country name variants across providers not yet merged |
| Evidence signals from PDF | Trial IDs, ethics statements, figure counts require PDF extraction; Scopus CSV imports show 0 for these fields |
| AI chat streaming | Responses appear after full generation  streaming not yet implemented |
| Report export (PDF/Word/PPT) | Planned |
| PostgreSQL backend | SQLite WAL handles ~100 concurrent users; PostgreSQL migration planned for larger deployments |

---

## Documentation

For a full walkthrough of every page and feature, see the [User Guide](docs/USER_GUIDE.md). Release history is tracked in the [Changelog](CHANGELOG.md).

---

## Author

**Robin Tomar** ([itsrobintomar@gmail.com](mailto:itsrobintomar@gmail.com))  
**Dr Prabudh Goel's Lab, AIIMS New Delhi**

---

<div align="center">
  <sub>BiblioDash &copy; 2026 &middot; Built for researchers, by researchers &middot; Made in India</sub>
</div>
