Metadata-Version: 2.4
Name: langparse
Version: 0.1.0rc2
Summary: A developer-friendly document parsing toolkit with precise, source-grounded Excel understanding.
Author-email: syw2014 <syw2014@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/syw2014/langparse
Project-URL: Repository, https://github.com/syw2014/langparse
Project-URL: Issues, https://github.com/syw2014/langparse/issues
Keywords: llm,rag,parsing,chunking,agent,document-parsing,excel,spreadsheet,workbook,xlsx
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: pdf
Requires-Dist: pdfplumber>=0.11.10; extra == "pdf"
Provides-Extra: docx
Requires-Dist: python-docx>=1.1.0; extra == "docx"
Provides-Extra: excel
Requires-Dist: pandas>=2.0.0; extra == "excel"
Requires-Dist: openpyxl>=3.1.0; extra == "excel"
Requires-Dist: tabulate>=0.9.0; extra == "excel"
Requires-Dist: pillow>=10.0.0; extra == "excel"
Provides-Extra: model
Requires-Dist: openai<3.0.0,>=2.0.0; extra == "model"
Provides-Extra: ocr
Requires-Dist: rapidocr_onnxruntime>=1.3.0; extra == "ocr"
Requires-Dist: pdfplumber>=0.11.10; extra == "ocr"
Provides-Extra: mineru
Requires-Dist: mineru<4,>=3.4; extra == "mineru"
Provides-Extra: deepdoc
Requires-Dist: pdfplumber>=0.11.10; extra == "deepdoc"
Requires-Dist: opencv-python-headless>=4.9.0; extra == "deepdoc"
Requires-Dist: onnxruntime>=1.17.0; extra == "deepdoc"
Requires-Dist: pypdf>=6.15.0; extra == "deepdoc"
Requires-Dist: huggingface_hub>=0.20.0; extra == "deepdoc"
Requires-Dist: scikit-learn>=1.3.0; extra == "deepdoc"
Requires-Dist: shapely>=2.0.0; extra == "deepdoc"
Requires-Dist: pyclipper>=1.3.0; extra == "deepdoc"
Requires-Dist: jieba>=0.42.1; extra == "deepdoc"
Provides-Extra: all
Requires-Dist: pdfplumber>=0.11.10; extra == "all"
Requires-Dist: python-docx; extra == "all"
Requires-Dist: pandas; extra == "all"
Requires-Dist: openpyxl; extra == "all"
Requires-Dist: tabulate>=0.9.0; extra == "all"
Requires-Dist: pillow>=10.0.0; extra == "all"
Requires-Dist: openai<3.0.0,>=2.0.0; extra == "all"
Requires-Dist: rapidocr_onnxruntime; extra == "all"
Requires-Dist: mineru<4,>=3.4; extra == "all"
Requires-Dist: opencv-python-headless; extra == "all"
Requires-Dist: onnxruntime; extra == "all"
Requires-Dist: pypdf>=6.15.0; extra == "all"
Requires-Dist: huggingface_hub; extra == "all"
Requires-Dist: scikit-learn; extra == "all"
Requires-Dist: shapely; extra == "all"
Requires-Dist: pyclipper; extra == "all"
Requires-Dist: jieba; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: ruff>=0.6.0; extra == "dev"
Requires-Dist: pdfplumber>=0.11.10; extra == "dev"
Requires-Dist: python-docx>=1.1.0; extra == "dev"
Requires-Dist: pandas>=2.0.0; extra == "dev"
Requires-Dist: openpyxl>=3.1.0; extra == "dev"
Requires-Dist: openai<3.0.0,>=2.0.0; extra == "dev"
Requires-Dist: tabulate>=0.9.0; extra == "dev"
Requires-Dist: jieba>=0.42.1; extra == "dev"
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/syw2014/langparse/main/assets/langparse-mark-512.png" alt="LangParse logo" width="128">
</p>

<h1 align="center">LangParse</h1>

<p align="center"><strong>Documents in. Structure out.</strong></p>

<p align="center">
  <a href="README_cn.md">简体中文</a>
</p>

<p align="center">
  <a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" alt="Apache 2.0 license"></a>
  <a href="https://pypi.org/project/langparse/"><img src="https://img.shields.io/pypi/v/langparse?include_prereleases" alt="PyPI version"></a>
  <a href="https://github.com/syw2014/langparse/actions"><img src="https://github.com/syw2014/langparse/actions/workflows/tests.yml/badge.svg" alt="Tests"></a>
</p>

LangParse is a developer-friendly Python toolkit for turning documents into
structured results that programs, agents, and data pipelines can use directly.

It has two product pillars:

- **Easy document parsing:** one predictable interface for PDF, DOCX, Excel,
  CSV, Markdown, and text, with optional chunking, batch processing, quality
  checks, and pluggable PDF backends.
- **Precise, rich Excel understanding:** preserve workbook facts and reconstruct
  logical tables, forms, matrices, text regions, and cross-sheet relationships
  instead of flattening a workbook into plain text.

PDF and Word support make LangParse useful across a document pipeline. Excel is
where LangParse goes deliberately deeper.

---

## Project status

The current release candidate is `0.1.0rc2`. Core multi-format parsing,
structured OOXML workbooks, semantic chunking, batch processing, quality checks,
and CI are available today. LangParse remains pre-1.0; see
[docs/PROGRESS.md](docs/PROGRESS.md) for the module-by-module source of truth and
known gaps.

## Why LangParse?

Most document workflows do not need another complicated platform. They need a
small toolkit that is easy to install, easy to call, and honest about the
structure it can recover.

Excel also needs a different abstraction from PDF and Word. A workbook may
contain formulas, merged headers, repeated print fragments, forms, matrices,
hidden rows, comments, links, and tables continued across sheets. Converting it
straight to Markdown destroys information that later analysis cannot recover.

LangParse therefore keeps three layers separate:

```mermaid
flowchart LR
    A["Documents<br/>PDF · DOCX · XLSX · CSV · MD · TXT"] --> B["Simple parsing API"]
    B --> C["Consumable result<br/>Markdown · JSON · chunks"]
    X["Excel / OOXML"] --> F["Workbook facts<br/>cells · formulas · styles · visibility"]
    F --> S["Workbook structure<br/>tables · forms · matrices · relationships"]
    S --> C
    C --> D["Applications<br/>RAG · Agents · data pipelines"]
```

Rich structure is the source of truth. Markdown and retrieval chunks are useful
views derived from it, not replacements for it.

## Core capabilities

- **One entry point:** `AutoParser.parse_result(...)` routes supported formats
  through a consistent result contract.
- **Rich Excel IR:** `.xlsx` and `.xlsm` preserve coordinates, raw and display
  values, formulas, cached values, merges, style fingerprints, visibility,
  dimensions, print areas, comments, hyperlinks, and object anchors.
- **Semantic workbook reconstruction:** deterministic blocks distinguish
  logical tables, forms, matrices, text, and unclassified regions while keeping
  source ranges and confidence diagnostics.
- **General document coverage:** Markdown, DOCX, legacy DOC, CSV, text, and PDF;
  PDF currently supports `simple`, MinerU, and DeepDoc backends.
- **Downstream-ready output:** normalized Markdown/JSON, source-aware chunks,
  batch processing, metrics, and quality checks.
- **Optional model assistance:** explicit opt-in workbook disambiguation; the
  default path remains offline and deterministic.

## Installation

Install the current release candidate:

```bash
pip install --pre "langparse==0.1.0rc2"
```

Install only the optional capabilities you need:

```bash
pip install "langparse[excel]"
pip install "langparse[excel,model]"  # optional OpenAI workbook disambiguation
pip install "langparse[deepdoc]"
pip install "langparse[all]"
```

Calling an existing remote MinerU API needs only the core package. Install
`langparse[mineru]` only when this Python environment must provide and start a
local `mineru-api` orchestrator.

## Quick start

### Parse any supported document

```python
from langparse import AutoParser

result = AutoParser.parse_result("report.docx")

print(result.markdown_content)
print(result.metadata)
```

The same entry point accepts PDF, DOCX, Excel, CSV, Markdown, and text. For PDF,
select a backend only when you need one:

```python
result = AutoParser.parse_result("scan.pdf", engine="deepdoc")
```

### Inspect Excel as structure

```python
from langparse import ExcelParser
from langparse.workbooks import WorkbookIR

result = ExcelParser().parse_result("budget.xlsx")
workbook = result.structure
assert isinstance(workbook, WorkbookIR)
assert workbook.snapshot is not None

first_sheet = workbook.snapshot.sheets[0]
print(first_sheet.cells["B2"].formula)

for sheet in workbook.sheets:
    for block in sheet.blocks:
        print(block.kind, [ref.key for ref in block.source_refs])
```

The workbook IR remains linked to the original sheet and cell ranges. You can
derive Markdown or chunks without losing the facts needed for validation and
analysis.

### Use the CLI

```bash
langparse parse report.docx --format markdown
langparse parse budget.xlsx --format json --chunk
langparse parse docs/ --batch --chunk --metrics --output-dir out
```

### Chunking

Chunks respect a size budget while following Markdown structure. Sections come
from headings; within a section, blocks are packed up to `max_chunk_size`.

```python
SemanticChunker(max_chunk_size=1000, overlap=0, length_function=len)
```

- **`length_function`** measures chunk size. The default counts characters and
  pulls in no dependencies; pass a tokenizer's encoder to budget in tokens:
  ```python
  import tiktoken

  encoder = tiktoken.get_encoding("cl100k_base")
  SemanticChunker(max_chunk_size=512, length_function=lambda t: len(encoder.encode(t)))
  ```
- **`overlap`** is off by default. It duplicates content into the vector store,
  so it is opt-in.
- **Tables** that exceed the budget split by row with the header row repeated in
  each part, so every chunk stays readable on its own.
- **Code blocks** are never split — splitting would leave unterminated fences.
  An oversized one emits whole with `oversized: True` in its metadata.
- A `#` inside a fenced code block is not treated as a heading.

Each chunk carries `header`, `header_level`, `header_path`, `page_numbers` and
`chunk_index`.

From the CLI, `--chunk` adds a `chunks` array to JSON output (and separates
chunks with `---` in Markdown output), and activates the chunk metrics:

```bash
langparse parse paper.pdf --chunk --format json
langparse parse docs/ --batch --chunk --metrics --output-dir out
```

### Excel deep dive

OOXML workbooks are not treated as paginated pandas tables. Each sheet keeps a
stable compatibility ordinal, while the result sets `paginated=False` and
exposes lossless source facts, deterministic logical tables, coverage
diagnostics, semantic Markdown, and source-aware table-row chunks from one
parse:

```python
from langparse.services.parse_service import ParseService

parsed = ParseService().parse_result(
    "budget.xlsx",
    chunk=True,
    chunk_profile="retrieval",
)
analysis_chunks = ParseService().chunk_result(
    parsed,
    chunk_profile="analysis",
)
print(parsed.structure.snapshot.sheets[0].cells["B2"].formula)
print(parsed.diagnostics.coverage_ratio)
print([block.kind for block in parsed.structure.sheets[0].blocks])
print(parsed.diagnostics.source_ref_validity_ratio)
print(parsed.chunks[0].metadata["chunk_type"])
print(parsed.chunks[0].metadata["source_ranges"])
print(analysis_chunks[0].structured_payload.get("records"))

logical_tables = [
    block.logical_table
    for sheet in parsed.structure.sheets
    for block in sheet.blocks
    if block.logical_table is not None
]
cross_sheet_tables = [
    continuation.logical_table for continuation in parsed.structure.table_continuations
]
```

The parser deterministically separates tables across blank row/column bands,
merges repeated print fragments, builds multi-level header paths, classifies
sections/data/totals, and chunks complete logical rows without crossing section
boundaries. Candidate regions are conservatively classified as logical tables,
forms, matrices, text, or explicit unclassified raw grids; every kind has a
source-aware Markdown and chunk path. `structure.snapshot` and compatibility
tables retain the original cell-level view. High-confidence adjacent-Sheet
continuations expose one aggregate logical table through
`structure.table_continuations`; insufficient evidence keeps tables independent
and records an ambiguous or rejected diagnostic. Markdown and chunks remain
source-Sheet based rather than duplicating the aggregate, and source-member
chunks can be regrouped by `continuation_id`. Retrieval/analysis dual chunk
profiles are built into the library, batch service, and CLI. `retrieval` is the
default profile and uses a 1000-character budget; `analysis` uses 4000. Both
preserve complete rows and exact source references. Analysis chunks add
normalized, source-linked `records` while keeping cell-level facts, including
formulas and cached values, in `structure.snapshot`. A parsed result can
generate another profile repeatedly with `chunk_result()` without reparsing or
mutating its structure. The analysis profile is only available for OOXML
workbook results: CSV, legacy `.xls`, and non-workbook inputs keep their
compatibility paths. Use `structure.snapshot` for exact cell or formula
analysis rather than treating analysis chunks as a replacement for the fact
layer.

```bash
langparse parse budget.xlsx --chunk --chunk-profile analysis --format json
```

#### Optional workbook model disambiguation

Workbook model disambiguation remains explicitly opt-in. The default is `off`:
constructing `ExcelParser()` or calling `ParseService` without
`workbook_disambiguation` performs no model Adapter or cache construction, reads
no provider configuration, and creates no implicit model network work. Install
the official OpenAI SDK integration separately from the core parser:

```bash
pip install "langparse[excel,model]"
export OPENAI_API_KEY="..."
export OPENAI_MODEL="gpt-4o-mini"
# Optional for an OpenAI-compatible endpoint:
export OPENAI_BASE_URL="https://example.invalid/v1"

langparse parse budget.xlsx --model --disambiguation auto --format json
```

API keys are intentionally not accepted as CLI arguments because process
arguments and shell history are not secret stores. `--model` or an explicit
`--disambiguation auto|required` enables network work; environment variables
alone never enable it. Set `LANGPARSE_DISABLE_MODEL=1` for the runtime kill
switch.

The library interface may either use the built-in `OpenAIWorkbookStructureAdapter`
or inject another `WorkbookStructureModelAdapter`:

```python
from langparse.parsers.excel_parser import ExcelParser
from langparse.services.parse_service import ParseService
from langparse.workbooks.modeling import WorkbookDisambiguation

# `adapter` is supplied by the caller and implements
# WorkbookStructureModelAdapter.
direct = ExcelParser(disambiguation=WorkbookDisambiguation.auto(adapter)).parse_result(
    "budget.xlsx"
)

strict = ParseService().parse_result(
    "budget.xlsx",
    workbook_disambiguation=WorkbookDisambiguation.required(adapter),
)
```

Phase 4A is limited to **choice-only region-kind disambiguation**. Only a locally
ambiguous, unclassified region with at least two compatible registered kinds is
eligible. A response can only be `selected` with that case's registered
`case_id + choice_id`, or `abstained`; it cannot express a value, formula,
coordinate, range, header, row role, continuation, or arbitrary structure.
Provider-reported confidence is diagnostic only. The selected kind is applied
from the retained workbook snapshot and must still pass local materialization,
coverage, reconstruction, row-conservation, continuation, and source-reference
validation.

Model application is workbook-atomic. If any attempted selection cannot be
materialized, or the tentative workbook fails a continuation or structural
validator, every attempted selection is restored to its retained deterministic
block and all validators run again. `required` reports every reverted case as
unresolved.

`auto` keeps the deterministic local fallback and records sanitized diagnostics
when the provider, cache, limits, response contract, materialization, or final
validation fails, or when the provider abstains. `required` raises
`RequiredWorkbookDisambiguationError` for unresolved eligible ambiguity and the
typed error passes through `ExcelParser` and `ParseService`; a workbook with no
eligible ambiguity succeeds with zero calls in either mode.

An enabled `WorkbookDisambiguation` value owns a private, thread-safe,
process-local runtime/cache. Reusing that same value across `ExcelParser`,
`ParseService`, or batch calls allows a validated response to become a
re-decoded cache hit; `off` constructs no runtime or cache. `max_cases` limits
cases considered, while `max_calls` is a workbook-wide hard budget of actual
Adapter invocations, including retries; cache hits consume no calls. Policy
timeouts must be finite positive non-boolean real values, and count/byte limits
must be exact positive non-boolean integers.

The candidate request is deliberately narrow. It can include the target Sheet
name and source range, visible cell coordinates and display text, value type,
style fingerprint, merge geometry, local scalar features, and the registered
choices for that region. It omits hidden Sheets, formulas and cached formula
values, comments, hyperlinks, images, other regions, credentials, and provider
secrets. If any cell in the complete candidate envelope contains a formula—even
an unlisted cell or merged child—the whole case is locally unavailable and no
formula or cached result is projected. Cell text is treated as untrusted Prompt Injection data: the Adapter
port exposes no tool channel, and exact response fields, request checksum,
case/choice membership, size limits, and local validation prevent cell
instructions from expanding the operation. Duplicate JSON member names are
rejected at every response-object depth. Diagnostics do not retain prompts,
cell text, raw replies, or provider exception messages. The process-local,
non-persistent cache has a narrower but different contract: it retains only
response envelope bytes that have already passed strict response decoding, and
every hit is decoded and validated again. Nothing is written to disk, but
provider-supplied strings inside that envelope may remain in process memory
until the owning disambiguation value and its private runtime are released.
Each model-call audit records local schema, prompt, rule, validator, and privacy
versions plus the deterministic fallback rule confidence; these values never
come from the provider.

The workbook ambiguity evaluator is available with:

```bash
langparse eval \
  samples/workbook_ambiguity/public-manifest.json \
  --output-dir reports/workbook-ambiguity

# Live provider evidence is still explicit:
langparse eval private-manifest.json --model
```

Reports are immutable by digest and reject incomplete or modified replays.
`production_ready` additionally requires holdout data, at least 30 ambiguous
cases, and separate operational staging evidence; the bundled tuning seed can
never satisfy that release gate by itself.

Whole-workbook structural quality has a separate entry point, so model
disambiguation accuracy is never presented as final Excel accuracy:

```bash
langparse benchmark-workbook-quality \
  samples/workbook_quality/public-manifest.json \
  --output-dir reports/workbook-quality
```

The public tuning seed contains thirteen manually labelled workbooks covering
logical tables, multiple tables per sheet, adjacent native tables, visually
separated tables without blank columns, form side notes, matrices, text,
explicit fallback, cross-sheet continuation, repeated print fragments, chart
facts, formulas, named ranges, and hidden sheets. Reports score block precision/recall, header paths,
row roles, forms/matrices, continuations, source references, fallback, and
object fact/semantic coverage. A failed gate returns exit code `1`; reports omit
cell values and annotation content and include the complete truth digest in the
immutable run digest. The public seed prevents regression; production evidence
still requires a separate private holdout.

Cost circuit breakers never infer prices from a model name. Library callers
that set `max_cost_usd_per_workbook` must also supply
`input_cost_usd_per_million`, `output_cost_usd_per_million`, and a stable
`cost_pricing_version` in `WorkbookModelPolicy`. These rates should come from
the deployment's provider contract, including for OpenAI-compatible endpoints.

Phase 4B includes the optional OpenAI SDK Adapter, environment-based provider
configuration, a strict structured-response contract, observed usage/cost
circuit breakers, and an immutable evaluation report pipeline. This is a
usable provider path, but not production-effectiveness evidence by itself:
release still requires a representative private holdout, staging latency/cost
and failure-mode evidence, and a provider privacy review. The observed token
and cost budgets stop retries or later calls after reported usage reaches the
limit; they cannot prevent the first provider call from exceeding a budget and
therefore are circuit breakers rather than billing guarantees.

Summary/index chunks, rich `.xls`/`.xlsb` adapters, image/chart semantic blocks,
standard bundle output, and further production hardening remain follow-up work;
screenshots and VLM are Phase 4C, a second domain contract is Phase 4D, and
delimited and legacy inputs keep the compatibility adapter for now.

### Scanned PDFs

The `simple` engine falls back to OCR when a page turns out to be an image.
Detection needs both a page-covering image and a thin text layer — a scanned
page often carries a watermark, and that watermark text is enough to clear any
threshold low enough to avoid firing on genuinely sparse text pages.

```bash
pip install "langparse[ocr]"
```

```python
PDFParser(engine="simple", enable_ocr=True, ocr_min_chars=500)
```

Pages that took the fallback report `ocr_applied` and `ocr_text_chars` in their
metadata, which surface in `ParseMetrics`. Without `rapidocr_onnxruntime`
installed the parse still succeeds — the page simply keeps whatever text layer
it had, rather than failing.

### Measuring parse fidelity

Quality checks measure structure: page counts, whether any tables were found.
They say nothing about whether the content is *correct*. To measure that, give a
benchmark sample a reference:

```json
{
  "id": "report-01",
  "path": "samples/report.pdf",
  "expected_markdown": "samples/report.expected.md",
  "expected_tables": [[["Header A", "Header B"], ["1", "2"]]]
}
```

- **Text** is scored by word-level normalised edit distance. Words rather than
  characters, because a reflowed line break is not an error but a dropped word
  is.
- **Tables** are scored by TEDS. Cell substitution costs the normalised
  character distance between the two cells, so a typo scores better than a
  wrong value, and a dropped row costs more than a changed cell.

Samples without a reference are reported as unscored, never as perfect.

### MinerU Runtime

LangParse can run MinerU through `mineru-api`.

Runtime selection works like this:
- If you pass or configure `api_url`, LangParse calls that MinerU service directly.
- A remote `mineru-api` backed by a separate vLLM server also receives `backend` and `server_url` as `/file_parse` form fields. This path does **not** require the local `[mineru]` extra.
- If `api_url` is not set, LangParse will try to start a local `mineru-api` service and manage its lifecycle for the current parse.
- If `mineru-api` is not installed, pass `--auto-install-runtime` or `auto_install_runtime=True` to let LangParse install the configured runtime package in the current Python environment before starting the local service.

You can still control CPU/GPU selection and model/download directories through runtime parameters or configuration.

For local managed services:
- `model_dir` means "use this already-downloaded MinerU model directory"
- `download_dir` becomes the MinerU home root used by the local service, so MinerU will keep its default cache/config layout under that directory
- `model_policy="require_existing"` disables first-run download fallback and requires an existing local model setup

```python
from langparse import AutoParser

doc = AutoParser.parse(
    "paper.pdf",
    engine="mineru",
    api_url="http://mineru.example:25820",
    backend="vlm-http-client",
    server_url="http://vlm.example:21670",
    request_timeout=900,
)
```

```python
from langparse import AutoParser

cpu_doc = AutoParser.parse(
    "paper.pdf",
    engine="mineru",
    device="cpu",
    download_dir="./downloads",
)
```

```python
from langparse import AutoParser

local_doc = AutoParser.parse(
    "paper.pdf",
    engine="mineru",
    model_dir="./preloaded-models",
    model_policy="require_existing",
)
```

Environment variables:

```bash
export LANGPARSE_MINERU_API_URL=http://127.0.0.1:8000
export LANGPARSE_MINERU_BACKEND=vlm-http-client
export LANGPARSE_MINERU_SERVER_URL=http://vlm.example:21670
export LANGPARSE_MINERU_REQUEST_TIMEOUT=900
export LANGPARSE_MINERU_DEVICE=cuda
export LANGPARSE_MINERU_MODEL_DIR=./models
export LANGPARSE_MINERU_DOWNLOAD_DIR=./downloads
export LANGPARSE_MINERU_MODEL_POLICY=require_existing
export LANGPARSE_MINERU_AUTO_INSTALL_RUNTIME=true
```

### CLI Examples

The CLI handles every supported format, not just PDF. `--engine` applies to
PDFs only; other formats route to their own parser automatically:

```bash
langparse parse report.docx --format json
langparse parse notes.md --output notes.out.md
langparse parse mixed_folder/ --batch --output-dir out --metrics
```

Supported extensions: `.pdf`, `.docx`, `.doc`, `.xlsx`, `.xlsm`, `.xls`, `.csv`, `.md`, `.txt`.
Batch directory expansion picks up all of them; unsupported files exit with
code 2 and a one-line message.

Single-file parsing:

```bash
langparse parse paper.pdf --engine mineru \
  --api-url http://mineru.example:25820 \
  --mineru-backend vlm-http-client \
  --mineru-server-url http://vlm.example:21670 \
  --mineru-request-timeout 900 \
  --format json
```

Batch parsing:

```bash
langparse parse docs/ --engine mineru --batch --output-dir out --format json
```

Batch parsing with lightweight metrics and skip-existing behavior:

```bash
langparse parse docs/ --engine mineru --batch --output-dir out --format json --max-workers 4 --skip-existing --metrics
```

Run a product-readiness benchmark:

```bash
langparse benchmark samples/public.example.json --engine mineru --output-dir reports --max-workers 2
```

Benchmark reports include success rate, elapsed time, pages per second, table counts, OCR indicators, reading-order warnings, header/footer filtering counts, and image/caption metadata coverage.

If you want LangParse to manage a local MinerU service, omit `--api-url`. You can also override the local launch command and bind address:

```bash
langparse parse paper.pdf --engine mineru --api-command "mineru-api" --api-host 127.0.0.1 --api-port 8000
```

Install MinerU automatically in the current Python environment if `mineru-api` is missing:

```bash
langparse parse paper.pdf --engine mineru --auto-install-runtime --device cpu --format json
```

Use an existing local model directory without allowing implicit downloads:

```bash
langparse parse paper.pdf --engine mineru --model-dir ./preloaded-models --model-policy require_existing
```

## 🛠️ Development & Local Testing

LangParse uses [`uv`](https://github.com/astral-sh/uv) for environment and dependency management. The checked-in `.venv` is uv-managed and intentionally has **no `pip`**, so run everything through `uv run` (a bare `pip`/`python` on your shell may resolve to a different interpreter, e.g. Anaconda).

### Set up the environment

```bash
# Install all dependencies (including dev/test) from uv.lock
uv sync --all-extras

# Or install just what you need
uv sync                      # core only (no third-party dependencies)
uv pip install -e ".[pdf]"   # PDF parsing (pdfplumber)
uv pip install -e ".[docx]"  # Word parsing (python-docx)
uv pip install -e ".[excel]" # Excel parsing (pandas + openpyxl)
uv pip install -e ".[model]" # Optional OpenAI workbook disambiguation
uv pip install -e ".[ocr]"   # OCR (rapidocr_onnxruntime)
uv pip install -e ".[mineru]"# MinerU API/orchestrator (local backend is explicit)
uv pip install -e ".[deepdoc]"# DeepDoc runtime (OCR/layout/table ONNX weights, ~100MB download on first run)
uv pip install -e ".[all]"   # everything above
```

> Note: the core install has **no third-party dependencies**. The PDF/DOCX/Excel parsers require the optional extras above; without them a parse fails with an `ImportError` naming the missing package rather than crashing. `pip install -e ".[dev]"` is enough to run the test suite.

### Run the tests

```bash
uv run pytest -q
```

### Smoke-test locally

```bash
# Markdown parse + semantic chunk (no extra deps needed)
uv run python examples/basic_usage.py

# Parse a PDF of your own (requires the [pdf] extra)
uv run langparse parse your.pdf --engine simple --format json

# Run the benchmark on the bundled manifest template
uv run langparse benchmark samples/public.example.json --engine simple --output-dir reports
```

The repository ships `samples/public.example.json` as a benchmark manifest template. `data/` is where local test documents go; it is git-ignored, so bring your own.

## 💬 Contact

For questions, feature requests, or bug reports, the preferred method is to **open an issue** on this GitHub repository. This allows for transparent discussion and helps other users who might have the same question.

## Citing LangParse

If you use LangParse in your research, product, or publication, we would appreciate a citation! You can use the following BibTeX entry:

```bibtex
@software{LangParse_2026,
  author = {syw2014},
  title = {LangParse: A developer-friendly document parsing toolkit with source-grounded Excel understanding},
  month = {September},
  year = {2026},
  publisher = {GitHub},
  url = {https://github.com/syw2014/langparse}
}
```

## Changelog
See [CHANGELOG.md](CHANGELOG.md) ([中文](CHANGELOG_cn.md)) for release notes and the dated development history.

## License
This project is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
