Metadata-Version: 2.4
Name: parsosaur
Version: 0.1.0
Summary: Convert academic PDFs to JSON, TEI XML, and HTML.
Author-email: Anil Madhavapeddy <anil@recoil.org>
License-Expression: MIT
License-File: LICENSE
Keywords: academic,nemotron,ocr,pdf,tei
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Text Processing :: Markup :: XML
Requires-Python: <3.14,>=3.10
Requires-Dist: accelerate==1.12.0
Requires-Dist: beautifulsoup4==4.13.3
Requires-Dist: einops==0.8.1
Requires-Dist: huggingface-hub<2,>=1.24
Requires-Dist: lxml<7,>=5.3
Requires-Dist: markdown<4,>=3.7
Requires-Dist: numpy==2.2.6
Requires-Dist: openai<3,>=1.40
Requires-Dist: packaging>=24.2
Requires-Dist: pillow<13,>=12.2
Requires-Dist: pypdfium2<6,>=4.30
Requires-Dist: safetensors==0.8.0
Requires-Dist: timm==1.0.22
Requires-Dist: torch<2.14,>=2.11.0
Requires-Dist: torchvision<0.29,>=0.26.0
Requires-Dist: transformers<5.20,>=5.14.1
Requires-Dist: vllm==0.26.0
Description-Content-Type: text/markdown

# parsosaur

Convert academic PDFs to JSON, TEI XML, and HTML.

## Synopsis

```text
parsosaur [OPTIONS] DIR ...
```

## Description

Parsosaur extracts text, tables, figures, and bibliographic metadata from
academic PDFs. Each input directory must contain exactly one PDF and at most
one GROBID TEI file with a `.tei` or `.tei.xml` suffix. The TEI file supplies
context for image interpretation and metadata for bibliographic merging.

New parses require Linux x86-64, Python 3.10–3.13, a CUDA GPU, and an
OpenAI-compatible vision-model endpoint. Nemotron performs layout extraction
locally. PDF crops and prompts are sent to the selected endpoint.

The NVIDIA runtime and model are downloaded from a fixed Hugging Face
revision on first use. Allow several gigabytes of disk space for dependencies
and model files.

Outputs are written to `DIR/out/`. Existing compatible results are reused;
complete cached results require neither GPU inference nor endpoint requests.
Use `--force` to regenerate results after changing inputs or options.

## Installation

From a source checkout, install and run Parsosaur with uv:

```sh
uv sync --locked
uv run parsosaur --help
```

After publication to PyPI, run it without a checkout:

```sh
uvx parsosaur --help
```

Use `uv tool install parsosaur` to install a persistent `parsosaur` command.
Alternatively, install it with `pip install parsosaur`.

After publication, `uv run URL [OPTIONS] DIR ...` can run the launcher at a
raw URL for `scripts/run_parsosaur.py`. The launcher installs the library
version specified in that file.

## Options

- `-h`, `--help` displays help and exits.
- `--base-url URL` selects the vision-model endpoint. It defaults to
  `LLM_BASE_URL`, or `https://openrouter.ai/api/v1` if that variable is unset.
  A URL without a path has `/v1` appended.
- `--model MODEL` selects the endpoint's model. It is required for OpenRouter.
  Other endpoints default to the first model returned by their model list.
- `--no-context` omits TEI prose from image interpretation requests.
- `--no-merge` skips merging bibliographic metadata from the TEI file.
- `--force` deletes each input directory's `out/` subdirectory and regenerates
  all results.

## Environment

- `LLM_BASE_URL` sets the default vision-model endpoint.
- `LLM_API_KEY` supplies the endpoint's API key. `OPENROUTER_API_KEY` is used
  if `LLM_API_KEY` is unset. OpenRouter requires a key.
- `HF_HOME` sets the Hugging Face cache directory.
- `HF_HUB_OFFLINE=1` restricts model and runtime loading to cached files.
  It does not disable requests to the vision-model endpoint.

## Files

All generated files reside in `DIR/out/`.

- `document.json`, `metadata.json`, and `figures.json` contain the structured
  extraction results.
- `document.tei.xml` contains the document in TEI XML format.
- `index.html` provides a reading view with source-page highlights.
- `viewer.html` displays detected regions on the PDF.
- `pages/`, `crops/`, and `raw/` contain rendered pages, extracted images, and
  intermediate model outputs.
- `merge_report.json` records bibliographic merge decisions when merging is
  enabled and a TEI file is supplied.

## Examples

Parse one paper using a local vision-model endpoint:

```sh
uvx parsosaur paper/ --base-url http://localhost:8000
```

Parse several papers using the same endpoint:

```sh
export LLM_BASE_URL=http://localhost:8000
uvx parsosaur papers/*/
```

## Exit status

Parsosaur exits with status zero on success and a nonzero status on error.
During batch processing, individual paper failures are reported and
processing continues with the remaining papers.

## License

Parsosaur is distributed under the MIT license. NVIDIA's runtime and model
are subject to their upstream license and notices.

Please note that this is experimental research software, and is likely
to be rapidly iterated on with no backwards compatibility guarantees.
You should also carefully check the outputs to ensure that they are 
suitable for your usecases.
