Metadata-Version: 2.4
Name: trafipy
Version: 0.1.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Text Processing :: Markup :: HTML
License-File: LICENSE-MIT
License-File: LICENSE-APACHE
License-File: LICENSE-THIRD-PARTY.md
Summary: Python bindings for rs-trafilatura — fast heuristic main-content extraction
Keywords: html,content-extraction,boilerplate-removal,trafilatura,markdown,rust
Author: Hamid Asgari
License-Expression: MIT OR Apache-2.0
Requires-Python: >=3.12
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Upstream crate, https://github.com/Murrough-Foley/rs-trafilatura

# trafipy

Python bindings for the [`rs-trafilatura`](https://github.com/Murrough-Foley/rs-trafilatura)
crate — fast heuristic main-content extraction from HTML, in-process, no subprocess.

> Unofficial bindings, not affiliated with the `rs-trafilatura` project.

## Install

```bash
pip install trafipy
```

Linux x86-64 wheels are published (`manylinux_2_34`, abi3 for Python 3.12+), so
glibc 2.34 or newer is required. On anything else pip falls back to the sdist,
which needs a Rust toolchain to compile.

## Use

```python
import trafipy

result = trafipy.extract(html, url="https://example.com/page", favor_recall=True)
result.markdown     # GitHub Flavored Markdown, "" when nothing was extracted
result.text         # plain text of the same region
result.quality      # the crate's own predicted F1, in [0, 1]
result.confidence   # page-type classification confidence, or None
result.warnings     # list[str]
```

`url` is the document's own address, used by the crate as context for
canonical-URL and hostname handling. It does **not** absolutize links in the
returned markdown or text — those come back as the document wrote them. It is
never fetched: there is no HTTP client anywhere in the dependency tree, and this
library makes no network requests.

### Options

`extract(html, *, url=None, include_tables=True, include_links=True,
include_images=True, include_comments=False, favor_recall=True,
min_extracted_size=100, deduplicate=False)`

Every option is passed straight through to the crate and every value comes back
as the crate produced it. The binding makes no extraction decisions of its own.

## Two decisions worth knowing

**One call returns both textual forms.** The crate's `ExtractResult` carries
`content_text` and `content_markdown` together, so `output_markdown` is set
unconditionally and you pay for one parse. The Python `trafilatura` API returns
one format per call, so getting both there costs two parses.

**Metadata is not exposed.** The crate extracts titles, dates and authors, but
this binding returns content only. If you already derive your own metadata,
surfacing a second set invites mixing them, and two provenance stories for one
field is worse than one.

## Versioning

`rs-trafilatura` is pinned exactly (`=0.2.2`), and the pin is not a formality.
The crate is pre-1.0, so a minor release can change extraction output — meaning
the same page yields different content across an unpinned upgrade. Bump it
deliberately and re-check against your own fixtures when you do.

## Build from source

```bash
maturin develop                  # build and install into the current venv
maturin build --release --out dist --compatibility manylinux_2_28
```

Requires Rust 1.85+ (the dependency tree uses edition 2024).

## License

Dual-licensed under [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE), at your
option — matching the upstream crate.

The wheel statically links `rs-trafilatura` and its dependency tree; their
license notices are collected in [LICENSE-THIRD-PARTY.md](LICENSE-THIRD-PARTY.md).

