Metadata-Version: 2.5
Name: bibr
Version: 0.5.1
Summary: Scientific paper extraction to structured JSON: metadata, text, references, tables, figures, and equations from PDF, DOCX, JATS XML, HTML, and ePub.
Project-URL: Homepage, https://bibr.org
Project-URL: Documentation, https://bibr.org
Project-URL: Repository, https://github.com/scienceverse/bibr
Project-URL: Issues, https://github.com/scienceverse/bibr/issues
Project-URL: Changelog, https://github.com/scienceverse/bibr/blob/main/CHANGELOG.md
Author-email: Scienceverse <jakub@jakubwerner.com>
License-Expression: AGPL-3.0-or-later
License-File: LICENSE.md
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU Affero General Public License v3
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Requires-Python: <3.15,>=3.11
Requires-Dist: beautifulsoup4>=4.14.3
Requires-Dist: google-genai>=1.0.0
Requires-Dist: html5lib>=1.1
Requires-Dist: httpx[http2]>=0.28.1
Requires-Dist: huggingface-hub>=0.26.0
Requires-Dist: instructor>=1.7.0
Requires-Dist: joblib>=1.3.0
Requires-Dist: jsonref>=1.1.0
Requires-Dist: litserve<0.3,>=0.2.16
Requires-Dist: numind<0.4,>=0.3
Requires-Dist: onnxruntime>=1.19.0
Requires-Dist: pandas>=2.3.3
Requires-Dist: pillow>=11.1.1
Requires-Dist: pydantic-settings>=2.14.2
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pypdfium2>=4.30.0
Requires-Dist: python-docx>=1.1.0
Requires-Dist: python-dotenv>=0.21.0
Requires-Dist: python-magic-bin>=0.4.14; sys_platform == 'win32'
Requires-Dist: python-magic>=0.4.27; sys_platform != 'win32'
Requires-Dist: python-multipart>=0.0.31
Requires-Dist: questionary>=2.1.0
Requires-Dist: rapidfuzz>=3.6.0
Requires-Dist: rich>=13.0.0
Requires-Dist: scikit-learn>=1.9.0
Requires-Dist: tenacity>=8.0.0
Requires-Dist: tokenizers>=0.21.0
Requires-Dist: uvicorn[standard]>=0.30.0
Requires-Dist: wordfreq>=3.0.0
Requires-Dist: wtpsplit-lite>=0.2.0
Provides-Extra: all
Requires-Dist: anthropic>=0.40.0; extra == 'all'
Requires-Dist: gradio>=6.15.0; extra == 'all'
Requires-Dist: mcp<3,>=2.2.0; extra == 'all'
Requires-Dist: opencv-python-headless>=4.8.0; extra == 'all'
Requires-Dist: pytorch-crf>=0.7.2; extra == 'all'
Requires-Dist: redis>=5.0.0; extra == 'all'
Requires-Dist: sentencepiece>=0.1.99; extra == 'all'
Requires-Dist: torch>=2.10.0; extra == 'all'
Requires-Dist: torchvision>=0.21.0; extra == 'all'
Requires-Dist: transformers>=5.0.0; extra == 'all'
Provides-Extra: batch
Requires-Dist: anthropic>=0.40.0; extra == 'batch'
Provides-Extra: cache
Requires-Dist: redis>=5.0.0; extra == 'cache'
Provides-Extra: demo
Requires-Dist: gradio>=6.15.0; extra == 'demo'
Provides-Extra: gpu
Requires-Dist: onnxruntime-gpu[cuda,cudnn]; (sys_platform == 'linux' or sys_platform == 'win32') and extra == 'gpu'
Provides-Extra: local
Requires-Dist: vllm-mlx<0.5,>=0.4; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'local'
Provides-Extra: local-mlx
Requires-Dist: vllm-mlx<0.5,>=0.4; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'local-mlx'
Provides-Extra: mcp
Requires-Dist: mcp<3,>=2.2.0; extra == 'mcp'
Provides-Extra: ml
Requires-Dist: opencv-python-headless>=4.8.0; extra == 'ml'
Requires-Dist: pytorch-crf>=0.7.2; extra == 'ml'
Requires-Dist: sentencepiece>=0.1.99; extra == 'ml'
Requires-Dist: torch>=2.10.0; extra == 'ml'
Requires-Dist: torchvision>=0.21.0; extra == 'ml'
Requires-Dist: transformers>=5.0.0; extra == 'ml'
Provides-Extra: torch
Requires-Dist: opencv-python-headless>=4.8.0; extra == 'torch'
Requires-Dist: pytorch-crf>=0.7.2; extra == 'torch'
Requires-Dist: sentencepiece>=0.1.99; extra == 'torch'
Requires-Dist: torch>=2.10.0; extra == 'torch'
Requires-Dist: torchvision>=0.21.0; extra == 'torch'
Requires-Dist: transformers>=5.0.0; extra == 'torch'
Provides-Extra: vllm
Requires-Dist: openai<3,>=2.54.0; (sys_platform == 'linux' and platform_machine == 'x86_64' and python_version < '3.14') and extra == 'vllm'
Requires-Dist: vllm==0.27.0; (sys_platform == 'linux' and platform_machine == 'x86_64' and python_version < '3.14') and extra == 'vllm'
Description-Content-Type: text/markdown

# [![bibr 🦫 — bibr chew paper.pdf | bibr.org](https://raw.githubusercontent.com/scienceverse/bibr/main/docs/assets/readme-banner.png)](https://bibr.org)

<!-- badges: start -->
[![PyPI version](https://img.shields.io/pypi/v/bibr.svg)](https://pypi.org/project/bibr/)
[![Docs](https://img.shields.io/badge/docs-bibr.org-blue)](https://bibr.org)
[![License: AGPL v3](https://img.shields.io/badge/License-AGPLv3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
![Made in Europe](https://img.shields.io/badge/Made_in_Europe-003399?logo=european-union&logoColor=FFCC00)
[![Lifecycle: experimental](https://img.shields.io/badge/lifecycle-experimental-orange.svg)](https://lifecycle.r-lib.org/articles/stages.html#experimental)
[![codecov](https://codecov.io/gh/scienceverse/bibr/graph/badge.svg?token=Mt0vQyE4qX)](https://codecov.io/gh/scienceverse/bibr)
<!-- badges: end -->

**bib**liography **r**odent 🦫 - a modern scientific extraction pipeline. Chews through papers, powered by open source and Metascience. Originally built for [Metacheck](https://github.com/scienceverse/metacheck) with accuracy as a priority.

- Reads PDF, DOCX, JATS XML, HTML, and ePub.
- Extracts metadata, references, full text, tables, figures, and equations into a
  [versioned JSON format](https://bibr.org/reference/schema/).
- Includes sentence and page references to help check extractions against the source.
- Works through the CLI, Python, an HTTP API, a web demo, or MCP.
- Lets you choose local or cloud models, limit page ranges, and skip extraction stages.

> **Alpha:** Expect bugs and uneven extraction quality. Current evaluation is strongest
> for English-language social science papers. See [known limitations](https://bibr.org/limitations/).

## Get started

Requires Python 3.11–3.14 and the
[system prerequisites](https://bibr.org/getting-started/install/#system-prerequisites).
Install from PyPI in a project managed by [uv](https://docs.astral.sh/uv/):

```bash
uv init --python 3.12 paper-extraction
cd paper-extraction
uv add bibr
uv run bibr setup
uv run bibr chew paper.pdf -o result.json
```

In an existing Python environment, you can also install with
`python -m pip install bibr` and run `bibr setup` / `bibr chew` directly.

The setup wizard detects your hardware, configures OCR and the LLM, and offers to
install any additional dependencies. Core installs run bibr's trained models through
ONNX Runtime; PyTorch, the demo, MCP, and hardware-specific serving runtimes are
[optional extras](https://bibr.org/getting-started/install/#extras). The first run may
download models and runtimes. See the [tester guide](https://bibr.org/tester-guide/)
for platform-specific instructions and the
[source installation guide](https://bibr.org/getting-started/install/#installing-from-source-contributors)
for development setup.

## Usage

### Command line

```bash
uv run bibr chew papers/ -o results/   # Process a directory
uv run bibr chew paper.pdf --dry-run   # Preview the processing plan
uv add 'bibr[demo]'                    # Add the optional web demo
uv run bibr demo                       # Open it locally
```

References are parsed locally by default. Use `--refs llm` to parse them with the
LLM, or `--refs off` to skip them. More options: [CLI reference](https://bibr.org/reference/cli/).

### Python

```python
import bibr

result = bibr.chew("paper.pdf")
print(result.title)
references = result.references.df  # pandas DataFrame
result.save("result.json")
```

See the [Python guide](https://bibr.org/guides/library/) for batch processing and
reusing loaded models with `bibr.Chewer`.

## LLM use

bibr uses LLMs selectively for tasks such as front-page metadata, with support
for small models tuned for extraction. You can disable downstream LLM extraction
with `--no-llm`, which returns structural output; PDF OCR may still use a
vision-language model. The [LLM use note](https://bibr.org/llm-use/) covers these choices
and how agentic LLMs helped develop bibr. It is a work in progress.

## Documentation

- [Configuration](https://bibr.org/guides/configuration/) — OCR, LLMs, reference parsing, and presets.
- [Deployment](https://bibr.org/guides/deployment/) — HTTP API (`bibr serve`), Docker, hardware, and authentication.
- [MCP server](https://bibr.org/guides/mcp/) — extraction tools for agents (`bibr mcp`).
- [JSON schema](https://bibr.org/reference/schema/) and [pipeline architecture](https://bibr.org/guides/architecture/).
- [Evaluating extraction quality](https://bibr.org/contributing/evaluation/) on papers from your workflow.

## Contributing

Bug reports, test papers, and contributions are welcome. See
[CONTRIBUTING.md](https://github.com/scienceverse/bibr/blob/main/CONTRIBUTING.md)
for development setup, tests, and pull requests.

Development began privately in December 2025. This public repository starts with
a clean source snapshot for the 0.5.0 launch; the earlier development history
remains private. Selected early design documents and their original contributions
are preserved in the [project history](https://github.com/scienceverse/bibr/tree/main/history).

---

## Acknowledgments

Special thanks to **Daniël Lakens** and **[Lisa DeBruine (@debruine)](https://github.com/debruine)**, for putting faith and patience in the project, and being generous with their time
 to help make bibr 🦫 better for everyone.

Lisa also contributed to the early paper-structure and metadata design documentation
preserved in the project history.

Also, to the whole [Metacheck](https://www.scienceverse.org/metacheck/) team, and **TU Eindhoven**.

We are grateful to the open-source projects that bibr builds on:

- [PaddleOCR-VL-1.6](https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6) (PaddlePaddle) — default OCR recognizer
- [GLM-OCR](https://huggingface.co/THUDM/GLM-OCR) (THUDM, Tsinghua University) — explicit compatibility backend and fallback
- [GROBID](https://github.com/kermitt2/grobid) — a major source of inspiration for structured scientific document parsing
- [LitServe](https://lightning.ai/docs/litserve/home) (Lightning AI) — serving infrastructure
- [PP-DocLayoutV3](https://github.com/PaddlePaddle/PaddleOCR) (PaddlePaddle) — document layout analysis
- [wtpsplit](https://github.com/segment-any-text/wtpsplit) — sentence segmentation
- [Crossref](https://www.crossref.org/) — reference metadata enrichment

---

## License

[AGPL-3.0-or-later](https://github.com/scienceverse/bibr/blob/main/LICENSE.md).
