Metadata-Version: 2.3
Name: spextract
Version: 0.6.1
Summary: A declarative html scraper for python. Define your scraper spec in a yaml or json to extract data from html documents.
License: MIT
Author: Vincent Lonij
Author-email: 29819815+vincentropy@users.noreply.github.com
Requires-Python: >=3.10
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: beautifulsoup4 (>=4.14.3,<5.0.0)
Requires-Dist: click (>=8.3.2,<9.0.0)
Requires-Dist: lxml (>=6.0.4,<7.0.0)
Requires-Dist: pydantic (>=2.12.5,<3.0.0)
Requires-Dist: pyyaml (>=6.0.3,<7.0.0)
Description-Content-Type: text/markdown

# A Declarative HTML Scraper for Python

This package provides a simple way to declare what data should be extracted from an HTML document in a configuration file.

This enables sharing of scraping logic across projects and teams without the risk of executing untrusted code. It also allows for easier maintenance and updates to scraping logic without needing to modify the underlying codebase.

## CLI

The package includes a Click-based CLI with two commands:

```bash
decs parse spec.yaml <path to html file or directory>
decs validate spec.yaml expected-results.yaml
```

`parse` emits YAML in the same expected-results format used by `validate`, so you can capture known-good output and re-run validation later.

## How to use

### Build a configuration file

You can write a configuration file with the provided ParserSpec class.

```python
import py_decs

spec = py_decs.ParserSpec(
    name="example_parser",
    description="An example parser for demonstration purposes.",
    fields=[
        py_decs.FieldSpec(
            name="title",
            selector="h1.title::text",
            type=py_decs.FieldType.TEXT,
        ),
        py_decs.FieldSpec(
            name="links",
            selector="a.link::attr(href)",
            type=py_decs.FieldType.LINK,
            multiple=True,
        )
        py_decs.FieldSpec(
            name="author",
            selector="div.author",
            type=py_decs.FieldType.OBJECT,
            fields=[
                py_decs.FieldSpec(
                    name="name",
                    selector="span.name::text",
                    type=py_decs.FieldType.TEXT,
                ),
                py_decs.FieldSpec(
                    name="profile_url",
                    selector="a.profile::attr(href)",
                    type=py_decs.FieldType.LINK,
                ),
            ]
        ),
    ]
)
```

