---
name: pdf
description: Extract text and tables from PDF files, merge/split/rotate pages, and generate new PDFs (reports, factor tear-sheets) from text/tables/images. Use when the task involves reading data out of a PDF (broker research, index constituent lists, fundamentals), combining or splitting PDF documents, or producing a PDF report/tear-sheet from analysis results.
---

# PDF skill

Toolkit for reading and writing PDFs in Python. Installed libraries:

- **pdfplumber** (0.11.9) — best for **extracting text and tables** with layout awareness. First choice for reading.
- **pypdf** (6.13.1) — page-level ops: **merge, split, rotate, crop, read metadata, encrypt/decrypt**. Pure-Python, no system deps.
- **reportlab** (4.5.1) — **generate** new PDFs from scratch (paragraphs, tables, page layout). Pair with matplotlib to embed charts.
- **matplotlib** (3.10.9) — render plots to PNG/PDF for embedding (tear-sheets).

There is NO LibreOffice in this environment, so HTML→PDF / docx→PDF conversion is not available; build PDFs directly with reportlab.

## Quick decision guide

| Task | Tool / script |
|------|---------------|
| Pull all text from a PDF | `scripts/pdf_extract.py --mode text` |
| Pull tables into CSV/DataFrame | `scripts/pdf_extract.py --mode tables` |
| Merge several PDFs | `scripts/pdf_pages.py merge a.pdf b.pdf -o out.pdf` |
| Split / extract a page range | `scripts/pdf_pages.py split in.pdf --pages 1-3 -o out.pdf` |
| Rotate pages | `scripts/pdf_pages.py rotate in.pdf --deg 90 -o out.pdf` |
| Build a report / tear-sheet PDF | `scripts/make_pdf_report.py` (or reportlab snippet below) |

## Helper scripts

- **`scripts/pdf_extract.py`** — Extract `text` or `tables` from a PDF. Tables mode writes one CSV per detected table (or a chosen page). Handles page ranges.
- **`scripts/pdf_pages.py`** — `merge` / `split` / `rotate` page operations via pypdf.
- **`scripts/make_pdf_report.py`** — Build a titled PDF from a CSV table + optional PNG image (e.g. a matplotlib chart), the typical factor tear-sheet shape.

## Reading tables (the common ingest case)

```python
import pdfplumber, pandas as pd
with pdfplumber.open("research.pdf") as pdf:
    page = pdf.pages[0]
    table = page.extract_table()          # list[list[str]]
    df = pd.DataFrame(table[1:], columns=table[0])
    # text: page.extract_text()
```
`extract_tables()` (plural) returns every table on the page. Tune detection with
`table_settings={"vertical_strategy": "text", "horizontal_strategy": "text"}` when
borders are missing.

## Generating a PDF (reportlab)

```python
from reportlab.lib.pagesizes import A4
from reportlab.lib import colors
from reportlab.lib.units import cm
from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, Table, TableStyle, Image
from reportlab.lib.styles import getSampleStyleSheet

styles = getSampleStyleSheet()
doc = SimpleDocTemplate("tearsheet.pdf", pagesize=A4)
story = [Paragraph("Factor Tear-Sheet: Momentum_20d", styles["Title"]), Spacer(1, 0.5*cm)]
# embed a chart you saved with matplotlib (plt.savefig("cum.png", dpi=150, bbox_inches="tight"))
story.append(Image("cum.png", width=16*cm, height=8*cm))
# a metrics table
data = [["Metric", "Value"], ["IC mean", "0.043"], ["IR", "0.51"], ["Sharpe", "1.8"]]
t = Table(data, hAlign="LEFT")
t.setStyle(TableStyle([
    ("BACKGROUND", (0,0), (-1,0), colors.HexColor("#DDEBF7")),
    ("FONTNAME", (0,0), (-1,0), "Helvetica-Bold"),
    ("GRID", (0,0), (-1,-1), 0.5, colors.grey),
]))
story.append(t)
doc.build(story)
```

## Gotchas

- pdfplumber tables come back as **strings** — cast numeric columns with `pd.to_numeric(..., errors="coerce")`.
- Scanned/image-only PDFs have no text layer; extraction returns empty. OCR (tesseract) is NOT installed here — flag this to the user rather than silently returning nothing.
- pypdf merge preserves bookmarks/forms but not always tagging; verify page count after merge.
- reportlab measures in points (1 inch = 72 pt); `from reportlab.lib.units import cm, inch` for readable sizes.
- Chinese text in reportlab needs a CJK font: `from reportlab.pdfbase.cidfonts import UnicodeCIDFont; from reportlab.pdfbase import pdfmetrics; pdfmetrics.registerFont(UnicodeCIDFont("STSong-Light"))` then use fontName="STSong-Light".
