Metadata-Version: 2.5
Name: melon-parse
Version: 0.1.0
Summary: Parse a Wikipedia page into Melon-mode: a clean, paginated, machine-readable reader JSON (ported from the Melon browser's encyclopedic extractor).
Project-URL: Homepage, https://github.com/alvations/melon-parse
Author-email: alvations <alvations@gmail.com>
License: MIT
License-File: LICENSE
Keywords: extraction,focus-mode,melon,parser,readability,reader,wikipedia
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Text Processing :: Markup :: HTML
Requires-Python: >=3.9
Requires-Dist: beautifulsoup4>=4.9
Requires-Dist: lxml>=4.6
Requires-Dist: requests>=2.25
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Description-Content-Type: text/markdown

# melon-parse

Parse a **Wikipedia page into Melon-mode**: a clean, distraction-free, **paginated focus-mode**
reader, emitted as **machine-readable JSON**.

This is a Python port of the encyclopedic reader from the [Melon browser](https://github.com/alvations/melon-browser)
— the same rules that strip Wikipedia's chrome (navboxes, hatnotes, edit links, `[1]` markers, the
"From Wikipedia, the free encyclopedia" tagline in ~50 languages), capture the infobox / data
tables / image gallery / references / interlanguage links as **structured data**, and split the
article into one coherent **top-level section per page** for focus-mode reading.

## Install

```bash
pip install melon-parse
```

## Use — library

```python
import melon_parse

article = melon_parse.parse("https://en.wikipedia.org/wiki/Watermelon")
# or a bare title:  melon_parse.parse("Watermelon", lang="en")

print(article.title, "—", len(article.sections), "focus-mode pages")
for s in article.sections:
    print(f"[{s.index}] {s.title}  ({len(s.blocks)} blocks)")

data = article.to_dict()          # fully JSON-serializable
```

## Use — CLI

```bash
melon-parse https://en.wikipedia.org/wiki/Watermelon        # JSON to stdout
melon-parse Watermelon --lang en -o watermelon.json         # bare title
melon-parse Watermelon --sections-only --indent 0           # just the pages, compact
melon-parse --html saved.html --url https://en.wikipedia.org/wiki/Watermelon   # offline
```

## Output schema (machine-readable)

```jsonc
{
  "url": "https://en.wikipedia.org/wiki/Watermelon",
  "title": "Watermelon",
  "lang": "en",
  "encyclopedic": true,
  "lead_image": "https://upload.wikimedia.org/…jpg",
  "section_count": 9,
  "sections": [                       // ← the paginated FOCUS MODE
    {
      "index": 0,
      "title": "Watermelon",          // lead page titled by the article
      "anchor": null,                 // original-page DOM id (scroll-sync), when present
      "blocks": [
        {"kind": "paragraph", "text": "Watermelon (Citrullus lanatus) is a flowering plant…"},
        {"kind": "heading", "level": 3, "text": "Etymology", "anchor": "Etymology"},
        {"kind": "list", "items": ["…", "…"]},
        {"kind": "image", "src": "https://…jpg", "caption": "A cut watermelon"},
        {"kind": "quote", "text": "…"}
      ]
    }
    // …one entry per top-level (H2) section; H3/H4 subsections stay inline via {kind:"heading",level:3}
  ],
  "infobox":   {"title": "Watermelon", "image": "…", "rows": [{"label": "Kingdom", "value": "Plantae"}]},
  "tables":    [{"caption": "…", "section": "Nutrition", "rows": [["Energy", "127 kJ"], …]}],
  "gallery":   [{"src": "…", "caption": "…"}],
  "references": {"1": "Smith, J. (2020)…", "2": "…"},          // [N] markers hidden from body
  "languages": [{"lang": "fr", "title": "Pastèque", "url": "https://fr.wikipedia.org/wiki/Pastèque"}]
}
```

### Block kinds
`paragraph` · `heading` (`level` 2–4, `anchor`) · `image` (`src`, `caption`) · `list` (`items[]`) · `quote`

### Pagination (focus mode)
Each **top-level (H1/H2)** heading starts a new page; **H3/H4** subsections stay *inline* within
their parent page. A page needs ≥ 220 chars of real body before the next heading spins up a new
page (empty/redundant sections like a structurally-captured "References" are dropped, not shown as
header-only fragments); a thin tail page folds back into the previous one. Nothing is dropped —
every real block lands in exactly one section.

## Scope
Currently Wikipedia / Wikipedia-style encyclopedic pages (`*.wikipedia.org/wiki/<Title>`). The
Melon browser applies the same reader idea to news, blogs, and other page types; those engines may
follow here.

## License
MIT © alvations
