Metadata-Version: 2.4
Name: contextractor
Version: 0.4.3
Summary: Drive the Contextractor Node crawler/extractor from Python — clean main-content text in txt, markdown, json, or html.
Project-URL: Homepage, https://apify.com/glueo/contextractor
Project-URL: Repository, https://github.com/contextractor/contextractor
Author-email: glueo <company@glueo.com>
License-Expression: Apache-2.0
Keywords: content-extraction,crawlee,crawler,markdown,playwright,trafilatura,web-scraping
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Text Processing :: Markup :: HTML
Classifier: Typing :: Typed
Requires-Python: >=3.12
Requires-Dist: nodejs-wheel-binaries<25,>=24.16
Provides-Extra: test
Requires-Dist: pytest-asyncio>=1.4; extra == 'test'
Requires-Dist: pytest>=9; extra == 'test'
Requires-Dist: pyyaml>=6; extra == 'test'
Description-Content-Type: text/markdown

# contextractor

<p align="center"><img width="220" src="https://www.contextractor.com/media/cover-mini.svg" alt="Contextractor"></p>

Crawl web pages and extract clean main-content text — `txt`, `markdown`, `json`,
or `html` — from Python. Built on [`rs-trafilatura`](https://github.com/Murrough-Foley/rs-trafilatura)
(extraction) and [Crawlee](https://crawlee.dev/) + [Playwright](https://playwright.dev/)
(crawling).

This package is a thin, typed wrapper that **drives the bundled Node engine** — a
self-contained Node runtime ships with the wheel, so **no Node.js install is
required**.

## Install

```bash
pip install contextractor
python -m contextractor install   # one-time: download the Chromium browser
```

Platform wheels are published for macOS (arm64, x86_64), Linux (x86_64, aarch64;
glibc ≥ 2.28), and Windows (x64). Requires Python 3.12+.

## Quick start

Crawl one or more URLs and write extracted files plus a `manifest.json` index to
`output_dir`:

```python
import contextractor

summary = contextractor.extract(
    ["https://example.com"],
    save=["markdown-kvs"],
    output_dir="./out",
)
print(summary.succeeded, "of", summary.total)
```

An async variant, `aextract()`, takes the same arguments. Partial failures do
not raise — they are reflected in `summary.failed`; validation and real crawl
failures raise `ContextractorError`, and a missing browser raises
`MissingBrowserError` pointing you at `python -m contextractor install`.

## Single-page extraction

`extract_one()` crawls exactly one URL (no link-following) and returns the
extracted content directly — nothing is written to disk:

```python
import contextractor

markdown = contextractor.extract_one("https://example.com")          # -> str

contents = contextractor.extract_one(                                # -> dict[str, str]
    "https://example.com",
    formats=["markdown", "json", "original"],
)
```

With one requested format the return value is a `str`; with several it is a
`dict[str, str]`. `aextract_one()` is the async variant.

## Options

Every crawl option is a typed keyword argument, surfaced to editors and type
checkers via the `ExtractOptions` / `ExtractOneOptions` `TypedDict`s (e.g.
`save`, `mode`, `proxy`, `cookies`, `headers`, `max_crawl_depth`, `globs`). For
the full, authoritative option reference see the
[project documentation and CLI help](https://github.com/contextractor/contextractor#readme)
(`contextractor extract --help`); the Python keywords map one-to-one onto the
CLI flags.

Only `http`, `https`, `socks4`, and `socks5` proxy URLs are accepted; credentials
are never echoed in errors or logs. Set `CONTEXTRACTOR_NODE_PATH` to use a host
Node binary instead of the bundled runtime.

## License

Apache-2.0
