Metadata-Version: 2.5
Name: marginalia-ai
Version: 0.6.0
Summary: Build a searchable, citable research corpus and expose it over MCP
Project-URL: Homepage, https://github.com/John-Cusack/MarginaliaAI
Project-URL: Source, https://github.com/John-Cusack/MarginaliaAI
Project-URL: Issues, https://github.com/John-Cusack/MarginaliaAI/issues
Project-URL: Changelog, https://github.com/John-Cusack/MarginaliaAI/blob/main/CHANGELOG.md
Project-URL: Documentation, https://github.com/John-Cusack/MarginaliaAI/tree/main/docs
Author: MarginaliaAI
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: citations,corpus,information-retrieval,mcp,research
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.11
Requires-Dist: alembic<2,>=1.13
Requires-Dist: anthropic<1,>=0.40
Requires-Dist: asyncpg<1,>=0.29
Requires-Dist: httpx<1,>=0.27
Requires-Dist: jinja2<4,>=3.1
Requires-Dist: marginalia-ai-sdk<0.7,>=0.6
Requires-Dist: mcp<2,>=1.0
Requires-Dist: packaging>=23
Requires-Dist: pgvector<1,>=0.3
Requires-Dist: pydantic-settings<3,>=2.1
Requires-Dist: pydantic<3,>=2.5
Requires-Dist: pyyaml<7,>=6
Requires-Dist: rich<14,>=13
Requires-Dist: sqlalchemy[asyncio]<3,>=2.0
Requires-Dist: structlog<25,>=24.1
Requires-Dist: typer<1,>=0.12
Requires-Dist: uuid-utils<1,>=0.9
Provides-Extra: dev
Requires-Dist: fastapi>=0.110; extra == 'dev'
Requires-Dist: pytest-asyncio<1,>=0.23; extra == 'dev'
Requires-Dist: pytest-cov<6,>=5; extra == 'dev'
Requires-Dist: pytest<9,>=8; extra == 'dev'
Requires-Dist: ruff<1,>=0.5; extra == 'dev'
Requires-Dist: testcontainers[postgres]<5,>=4; extra == 'dev'
Provides-Extra: document-ai
Requires-Dist: docling<3,>=2.70; extra == 'document-ai'
Provides-Extra: documents
Requires-Dist: beautifulsoup4<5,>=4.12; extra == 'documents'
Requires-Dist: ebooklib<1,>=0.18; extra == 'documents'
Requires-Dist: lxml<6,>=5.1; extra == 'documents'
Requires-Dist: pymupdf<2,>=1.24; extra == 'documents'
Provides-Extra: embed-server
Requires-Dist: fastapi>=0.110; extra == 'embed-server'
Requires-Dist: sentence-transformers<4,>=3.0; extra == 'embed-server'
Requires-Dist: uvicorn>=0.27; extra == 'embed-server'
Provides-Extra: full
Requires-Dist: beautifulsoup4<5,>=4.12; extra == 'full'
Requires-Dist: docling<3,>=2.70; extra == 'full'
Requires-Dist: ebooklib<1,>=0.18; extra == 'full'
Requires-Dist: fastapi>=0.110; extra == 'full'
Requires-Dist: lxml<6,>=5.1; extra == 'full'
Requires-Dist: openai<2,>=1.10; extra == 'full'
Requires-Dist: pymupdf<2,>=1.24; extra == 'full'
Requires-Dist: sentence-transformers<4,>=3.0; extra == 'full'
Requires-Dist: uvicorn>=0.27; extra == 'full'
Provides-Extra: local-inference
Requires-Dist: sentence-transformers<4,>=3.0; extra == 'local-inference'
Provides-Extra: openai
Requires-Dist: openai<2,>=1.10; extra == 'openai'
Description-Content-Type: text/markdown

# Research Engine

Research Engine builds a searchable, citable research corpus and exposes it to MCP clients.
The base package includes the CLI, PostgreSQL schema/migrations, MCP server, remote inference
clients, lightweight text ingestion, extraction, entity/event services, and plugin host.

It does not install PostgreSQL, database extensions, local ML models, Docling, GPU drivers, or
third-party plugins.

## Install

```bash
python -m pip install marginalia-ai
# Everything, including local inference and document AI:
python -m pip install "marginalia-ai[full]"
```

Optional features are independently installable:

- `marginalia-ai[openai]` — OpenAI-compatible LLM adapter;
- `marginalia-ai[local-inference]` — sentence-transformers embedding and reranking;
- `marginalia-ai[documents]` — PDF text, EPUB, HTML, and TEI parsers;
- `marginalia-ai[document-ai]` — Docling layout/OCR and office/image conversion;
- `marginalia-ai[embed-server]` — FastAPI/Uvicorn plus its local inference runtime.

Local inference and Docling may download multi-gigabyte models and can require substantial disk,
RAM, and GPU capacity. A standard PyPI install does not select PyTorch's alternate CPU wheel
index; follow PyTorch's CPU installation instructions first when required.

## Database

Use PostgreSQL 15 or newer with `vector`, `pg_trgm`, and `ltree` available. Creating extensions
may require an elevated database role. Set the async URL explicitly:

```bash
export RE_DB_URL='postgresql+asyncpg://user:password@localhost:5432/research_engine'
ALEMBIC_INI="$(python -c 'from importlib.resources import files; print(files("research_engine").joinpath("adapters/storage/postgres/migrations/alembic.ini"))')"
alembic -c "$ALEMBIC_INI" upgrade head
research-engine doctor
```

`pg_dump` and `pg_restore` are external requirements for backup commands.

## Run over MCP

```json
{
  "mcpServers": {
    "research-engine": {
      "type": "stdio",
      "command": "research-engine",
      "args": ["serve"]
    }
  }
}
```

No checkout or repository working directory is required. `research-engine --help` and
`research-engine config --help` describe the installed command surface.

## Data and trust

Remote LLM providers receive prompts and selected corpus text and may charge per token. Review
provider settings and budgets before ingestion or extraction.

Plugins are ordinary Python distributions. Installation makes their static manifests available;
`research-engine plugin enable ID` displays and records the exact version, hash, contributions,
and permissions before code is imported. Enabled plugins execute in-process. Scoped clients are
the supported API boundary, not a security sandbox; enable only trusted artifacts.

See the [documentation](https://github.com/John-Cusack/MarginaliaAI/tree/main/docs),
[changelog](https://github.com/John-Cusack/MarginaliaAI/blob/main/CHANGELOG.md),
[issues](https://github.com/John-Cusack/MarginaliaAI/issues), and
[Apache-2.0 license](https://github.com/John-Cusack/MarginaliaAI/blob/main/LICENSE).
