Metadata-Version: 2.4
Name: piliwela
Version: 1.0.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Typing :: Typed
Requires-Dist: maturin>=1.7,<2.0 ; extra == 'dev'
Requires-Dist: pytest>=8,<9 ; extra == 'dev'
Requires-Dist: pymupdf>=1.24,<2.0 ; extra == 'dev'
Requires-Dist: pymupdf>=1.24,<2.0 ; extra == 'pdf'
Provides-Extra: dev
Provides-Extra: pdf
License-File: LICENSE
Summary: Rust-powered Sinhala legacy-font conversion for text, PyMuPDF pages, and PDF documents.
Keywords: sinhala,unicode,legacy-font,fmabhaya,pdf,pymupdf,nlp
Author: Naveen Chethiya
License-Expression: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/Naviya-C/piliwela
Project-URL: Issues, https://github.com/Naviya-C/piliwela/issues
Project-URL: Repository, https://github.com/Naviya-C/piliwela

<div align="center">

# Piliwela

**Fast Sinhala legacy-font to Unicode conversion for Python**

Convert FM-family Sinhala text while preserving English content, PDF layout metadata, and PyMuPDF's page structure.

[![PyPI version](https://img.shields.io/pypi/v/piliwela.svg)](https://pypi.org/project/piliwela/)
[![Python versions](https://img.shields.io/pypi/pyversions/piliwela.svg)](https://pypi.org/project/piliwela/)
[![CI](https://github.com/Naviya-C/piliwela/actions/workflows/ci.yml/badge.svg)](https://github.com/Naviya-C/piliwela/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/Naviya-C/piliwela/blob/main/LICENSE)

</div>

## Why Piliwela?

Many older Sinhala documents use legacy fonts where Sinhala characters are stored as Latin/ASCII codes. Extracting text from those PDFs can produce unreadable strings such as:

```text
Y%S ,xld
```

Piliwela converts that text into modern Unicode Sinhala:

```text
ශ්‍රී ලංකා
```

It uses PDF font metadata when available and preserves English words in mixed-language content.

## Features

- Rust-powered conversion engine with Python bindings
- FM-family legacy Sinhala to Unicode conversion
- Automatic conversion using PDF font metadata
- English and mixed-language text preservation
- Single-span and batched-span APIs
- PyMuPDF-compatible page dictionary conversion
- Complete-document and page-streaming PDF APIs
- Source font, bounding box, size, flags, block, and line preservation
- Optional raw-text and conversion metadata
- Per-document conversion reports
- Python 3.9–3.13 support

## Supported fonts

Piliwela 1.0 supports the FM legacy-font family, including fonts whose normalized names begin with `FM` and subset-prefixed PDF names such as:

```text
FMAbhaya
FMSamantha
FMEmanee
BCDEEE+FMAbhaya
```

DL, Wijeya, and Kaputa mappings are not yet part of the supported 1.0 API.

## Installation

For text and span conversion:

```bash
pip install piliwela
```

For PDF document conversion with PyMuPDF:

```bash
pip install "piliwela[pdf]"
```

## Quick start

```python
import piliwela

converted = piliwela.convert_auto_with_metadata(
    "Y%S ,xld",
    "FMAbhaya",
)

print(converted)
```

Output:

```text
ශ්‍රී ලංකා
```

## Convert plain text

Use `convert_auto()` when font metadata is unavailable:

```python
import piliwela

converted = piliwela.convert_auto("Y%S ,xld")
print(converted)
```

For PDF files, prefer the metadata-aware APIs because embedded font names provide safer detection.

## Convert one span with metadata

```python
import piliwela

result = piliwela.convert_span(
    "Y%S ,xld",
    "FMAbhaya",
)

print(result.text)
print(result.raw_text)
print(result.font_name)
print(result.family)
print(result.changed)
print(result.detection_source)
```

Example result:

```text
ශ්‍රී ලංකා
Y%S ,xld
FMAbhaya
FM
True
font_metadata
```

## Convert multiple spans efficiently

The batch API performs one Python-to-Rust call for the complete group:

```python
import piliwela

results = piliwela.convert_spans(
    [
        ("Y%S ,xld", "FMAbhaya"),
        ("English textbook", "Helvetica"),
    ]
)

for result in results:
    print(result.text)
```

Output:

```text
ශ්‍රී ලංකා
English textbook
```

## Convert a PyMuPDF page dictionary

If your application already opens the PDF with PyMuPDF, use `convert_page_dict()`:

```python
import pymupdf
import piliwela

with pymupdf.open("textbook.pdf") as document:
    page_data = document[0].get_text("dict")

converted_page = piliwela.convert_page_dict(page_data)
```

The returned value preserves PyMuPDF's structure:

```text
page
└── blocks
    └── lines
        └── spans
            ├── text
            ├── font
            ├── size
            ├── flags
            └── bbox
```

Only `span["text"]` is replaced. The input dictionary is not modified unless `in_place=True` is supplied.

### Preserve raw text and conversion metadata

```python
converted_page = piliwela.convert_page_dict(
    page_data,
    preserve_raw=True,
    include_metadata=True,
)
```

Each text span then includes:

```python
{
    "text": "ශ්‍රී ලංකා",
    "raw_text": "Y%S ,xld",
    "font": "FMAbhaya",
    "_piliwela": {
        "family": "FM",
        "changed": True,
        "detection_source": "font_metadata",
    },
}
```

## Convert an entire PDF

```python
import piliwela

result = piliwela.convert_pdf(
    "textbook.pdf",
    sort=True,
    include_images=False,
    preserve_raw=True,
    include_metadata=True,
)

for page in result.pages:
    for block in page["blocks"]:
        for line in block.get("lines", []):
            text = "".join(
                span["text"]
                for span in line.get("spans", [])
            )
            if text.strip():
                print(text)
```

`convert_pdf()` accepts:

- A filesystem path
- `bytes`
- `bytearray`
- `memoryview`
- A binary file object

`convert_document()` is an alias of `convert_pdf()`.

## Stream large PDFs

Use `iter_pdf()` for large textbooks so only one converted page is held at a time:

```python
import piliwela

for page in piliwela.iter_pdf(
    "large-textbook.pdf",
    sort=True,
):
    process(page)
```

## Conversion report

`convert_pdf()` returns a `ConvertedDocument` containing converted pages and a report:

```python
result = piliwela.convert_pdf("textbook.pdf")
print(result.report.to_dict())
```

Example:

```python
{
    "pages": 120,
    "text_blocks": 3912,
    "lines": 18440,
    "spans": 22108,
    "converted_spans": 17902,
    "unchanged_spans": 4206,
    "family_counts": {
        "FM": 17902,
        "Unknown": 4206,
    },
}
```

## Save converted text

```python
import piliwela

result = piliwela.convert_pdf("textbook.pdf", sort=True)
pages = []

for page_number, page in enumerate(result.pages, start=1):
    lines = []

    for block in page["blocks"]:
        for line in block.get("lines", []):
            text = "".join(
                span["text"]
                for span in line.get("spans", [])
            )
            if text.strip():
                lines.append(text)

    pages.append(
        f"--- Page {page_number} ---\n" + "\n".join(lines)
    )

with open("converted_textbook.txt", "w", encoding="utf-8") as file:
    file.write("\n\n".join(pages))
```

## API overview

| API | Purpose |
| --- | --- |
| `convert_auto(text)` | Convert text without font metadata |
| `convert_auto_with_metadata(text, font_name)` | Convert text using a PDF font name |
| `convert_span(text, font_name)` | Return converted text with typed metadata |
| `convert_spans(spans)` | Convert multiple spans in one Rust batch |
| `convert_page_dict(page_data)` | Convert a PyMuPDF page dictionary |
| `convert_page(page_data)` | Return a converted page and report |
| `convert_pdf(source)` | Convert a complete PDF into structured pages |
| `convert_document(source)` | Alias of `convert_pdf()` |
| `iter_pdf(source)` | Stream converted page dictionaries |
| `detect(text)` | Detect a legacy family from text heuristics |
| `detect_from_metadata(font_name)` | Detect a family from a PDF font name |
| `version()` | Return the installed Piliwela version |

## Important behavior

### Digital PDFs only

Piliwela converts text already present in a PDF's text layer. It does not perform OCR. Use an OCR engine first for image-only or scanned PDFs.

### Structured output, not a rewritten PDF

`convert_pdf()` returns converted text in PyMuPDF-compatible page dictionaries. It does not visually replace glyphs or generate a newly typeset PDF.

### Source geometry is preserved

Bounding boxes identify where the original legacy glyphs appeared. They are source PDF coordinates, not recalculated Unicode text dimensions.

### Images

Images are excluded by default to reduce memory usage. Preserve PyMuPDF image blocks with:

```python
result = piliwela.convert_pdf(
    "textbook.pdf",
    include_images=True,
)
```

### Reading order

PDF internal text order may differ from visual reading order. Pass `sort=True` to request PyMuPDF's top-left to bottom-right sorting:

```python
result = piliwela.convert_pdf("textbook.pdf", sort=True)
```

## Error handling

```python
import piliwela

try:
    result = piliwela.convert_pdf("textbook.pdf")
except piliwela.PDFDependencyError:
    print('Install PDF support with: pip install "piliwela[pdf]"')
except piliwela.DocumentConversionError as error:
    print(f"Invalid page structure: {error}")
```

## Use in a document-ingestion service

Add the dependency:

```text
piliwela[pdf]>=1.0.0,<2.0.0
```

If the service already has an open PyMuPDF page, avoid reopening the document:

```python
page_data = page.get_text("dict")
converted_page = piliwela.convert_page_dict(
    page_data,
    in_place=True,
)
```

## Development

```bash
git clone https://github.com/Naviya-C/piliwela.git
cd piliwela

python -m venv .venv
source .venv/bin/activate

pip install -U pip maturin
pip install ".[dev]"
maturin develop
pytest
cargo test --locked
```

Build a release wheel:

```bash
maturin build --release --out dist
```

## Contributing

Contributions are welcome, particularly:

- Verified mappings for additional Sinhala legacy-font families
- Real-world FM-family regression examples
- Mixed Sinhala-English conversion tests
- PDF extraction edge cases
- Documentation improvements

Please include tests and anonymized sample strings for conversion changes.

## License

Piliwela is released under the [MIT License](https://github.com/Naviya-C/piliwela/blob/main/LICENSE).

