# SoulDoc

<p align="center">
  <a href="https://tentags.org">
    <img src="https://tentags.org/assets/img/souldoc-icon.png" width="300" alt="SoulDoc icon">
  </a>
</p>

**SoulDoc** is a compact, format-independent document layout and rendering
engine for Python. It describes document meaning, structure, and geometry
before choosing an output format.

> Define the document once. Render it everywhere.

## Project status

Version `1.0.3` is the current stable release. It provides a dependency-free
core, automatic layout, and official HTML, SVG, PDF, DOCX, and XLSX renderers.

The core:

- installs as a regular Python package without runtime dependencies;
- does not import PyCells, SQLAlchemy, FastAPI, or Playwright;
- supports strict and compatibility-oriented input normalization;
- provides a versioned JSON contract;
- preserves unknown fields from future schema versions;
- supports metadata and separately registered resources;
- understands the TenTags cell language without importing `tentags`;
- recognizes PyCells `MD(...)` records and renders Markdown as semantic HTML;
- measures content, performs flow layout, and paginates automatically;
- provides `Text`, `Markdown`, `Image`, `Table`, `Row`, `Column`, `Grid`, and
  `Absolute`;
- uses one `SoulStyle` model across all renderers;
- exposes a unified public API from the `souldoc` package.

Renderers with external dependencies are installed as optional extras. External
images are loaded only through an application-provided `resource_resolver`.
PDF output is printed from SoulDoc's HTML representation in Chromium with
HTTP and HTTPS requests blocked.

The `1.x` public API and schema version 1 form the stable compatibility
contract prepared for the first PyPI release.

## Why SoulDoc exists

Most export libraries are centered on one format:

- a DOCX library understands WordprocessingML;
- a PDF library understands pages and drawing commands;
- an HTML renderer understands the DOM and CSS;
- a spreadsheet library understands rows, columns, cells, and merges.

Without an intermediate model, application code often accumulates editor
coordinates, HTML fragments, PDF options, DOCX XML, and format-specific image
and table rules.

SoulDoc introduces a stable layer between application data and renderers:

```text
Content / templates / data
            |
            v
   SoulLayoutDocument
            |
            | layout_document()
            v
      SoulDocument IR
            |
            +----> HTML ----> PDF
            +----> SVG
            +----> DOCX
            +----> XLSX
            +----> JSON
```

An editor does not need to understand DOCX internals, and a DOCX renderer does
not need to understand an application's database. Both communicate through
SoulDoc.

## SoulDoc and TenTags

SoulDoc complements TenTags; it does not replace it.

- **TenTags** is a compact declarative language and export engine for tables.
- **SoulDoc** is a document layout engine that handles free positioning,
  geometry, pagination, and complete documents.

The `souldoc.tentags` module is independent from the DOCX renderer. It converts
TenTags input into a neutral table model and semantic HTML. TenTags `<cm>` and
`<rm>` markers become standard `colspan` and `rowspan` attributes, after which
each renderer can create its native merged cells.

The parser supports the cell tags currently used by TenTags, including `<b>`,
`<i>`, `<u>`, `<s>`, `<color>`, `<bg>`, `<fs>`, `<left>`, `<center>`,
`<right>`, `<cm>`, `<rm>`, `<url>`, `<img>`, `<mark>`, `<value>`, and `<br>`.
SoulDoc neither imports nor modifies the external `tentags` package.

The `scale(...)` preamble is also supported. Its vertical value controls row
height, and its horizontal value controls relative column width:

```text
TABLE(
  scale(A1=2,3;C1=1,2)
  data(A1=<b>Name</b>;B1=Amount;C1=Status)
)
```

In this example, the first row receives a height multiplier of `2`, column A a
width weight of `3`, and column C a width weight of `2`. Semantic HTML uses
`colgroup` and row heights; the DOCX renderer converts them into native Word
column widths and row heights.

## SoulDoc and PyCells Markdown

PyCells stores a Markdown formula in a cell's source data and its evaluated
HTML in the calculated value:

```text
data  = =MD("# Hello\n- First\n- Second")
value = <h1>Hello</h1><ul><li>First</li><li>Second</li></ul>
```

When such a record enters SoulDoc, the evaluated HTML is preserved and marked
as HTML content:

```python
from souldoc import document_from_dicts, save_document

document = document_from_dicts(
    [
        {
            "id": "markdown",
            "name": {
                "page": 1,
                "left_mm": 15,
                "top_mm": 15,
                "width_mm": 180,
                "height_mm": 80,
            },
            "data": r'=MD("# Hello\n## Markdown\n- First\n- Second")',
            "value": (
                "<h1>Hello</h1><h2>Markdown</h2>"
                "<ul><li>First</li><li>Second</li></ul>"
            ),
        }
    ]
)

save_document(document, "markdown.pdf")
save_document(document, "markdown.docx")
save_document(document, "markdown.xlsx")
```

If `value` is empty, SoulDoc decodes the `MD(...)` formula and converts common
Markdown itself. The dependency-free fallback supports headings, paragraphs,
line breaks, emphasis, strong text, strikethrough, links, images, ordered and
unordered lists, block quotes, fenced code, horizontal rules, and pipe tables.
Raw HTML is escaped in this fallback.

PyCells remains the preferred evaluator for its complete `extra` and `nl2br`
behavior. SoulDoc's fallback makes stored records portable when only the raw
formula is available.

The conversion helpers are public:

```python
from souldoc import (
    contains_markdown_formula,
    extract_markdown,
    markdown_formula_to_html,
    markdown_to_html,
)

source = extract_markdown(r'=MD("# Hello\n- Item")')
html = markdown_to_html(source)
```

Markdown that is not wrapped in `MD(...)` must be marked explicitly. This
prevents ordinary text containing `#`, `*`, `_`, or `-` from being
misinterpreted:

```python
from souldoc import (
    Markdown,
    SoulContentKind,
    SoulRecord,
)

record = SoulRecord(
    content="# Explicit Markdown\n\n- First\n- Second",
    content_kind=SoulContentKind.MARKDOWN,
)

component = Markdown(
    "# Layout Markdown\n\nThis is **bold**."
)
```

Explicit Markdown remains Markdown in the positioned IR and JSON contract.
Each visual renderer converts it to semantic HTML when producing output.
`Text("# Not a heading")` always remains plain text.

## Two document levels

### `SoulLayoutDocument`

`SoulLayoutDocument` is the high-level input model. Its component tree
describes flow content, rows, columns, grids, tables, and absolute layers.
`layout_document()` measures text, calculates geometry, wraps content,
paginates it, and returns a `SoulDocument`.

### `SoulDocument`

`SoulDocument` is the positioned intermediate representation. It contains
pages and boxes whose coordinates and dimensions are already known. Renderers
consume this model without repeating layout decisions.

### `SoulPage`

A page has a number, width and height in millimetres, and an ordered list of
positioned boxes. Arbitrary page sizes are supported.

### `SoulBox`

A box is the basic positioned unit. It contains:

- an identifier and page number;
- `left_mm` and `top_mm` coordinates;
- `width_mm` and `height_mm` dimensions;
- a `z_index`;
- source and computed values;
- final content and its kind;
- CSS-like style information;
- optional metadata and a resource reference.

Millimetres provide stable geometry across browsers, Word, PDF, and
spreadsheets without depending on screen DPI.

### Content kinds

```text
text      - plain text
markdown  - explicit Markdown source
html      - formatted HTML
table     - a semantic table
image     - an image resource
math      - an editable mathematical source expression
mixed     - mixed content
```

Content kinds let each renderer choose the closest native object. For example,
a table becomes a Word `w:tbl` element instead of a raster image.

### `SoulStyle`

`SoulStyle` is the shared model for dimensions, margin, padding, gaps, fonts,
colors, borders, alignment, overflow, and page-break behavior. Renderers
receive the same calculated geometry and CSS-like style information.

## Why the codebase can remain compact

SoulDoc separates automatic layout from format backends. The layout engine
answers four questions:

1. What is in the document?
2. On which page is it placed?
3. Where is it placed and in what stacking order?
4. How does it continue across pages?

Low-level file-format work belongs to individual renderers. This separation
keeps the core small, understandable, and testable. Compactness is an
architectural property, not a lack of capability.

## Installation

Install only the dependency-free core:

```bash
pip install souldoc
```

Install editable DOCX support:

```bash
pip install "souldoc[docx]"
```

Install visual XLSX support:

```bash
pip install "souldoc[xlsx]"
```

Install PDF support and its browser:

```bash
pip install "souldoc[pdf]"
playwright install chromium
```

Install every official renderer:

```bash
pip install "souldoc[all]"
```

For local development before publication:

```bash
pip install -e .
```

## Quick start

Create a high-level document and export it to several formats:

```python
from souldoc import (
    Column,
    SoulLayoutDocument,
    SoulStyle,
    Table,
    Text,
    save_document,
)

document = SoulLayoutDocument(
    name="Quarterly report",
    children=[
        Column(
            gap_mm=5,
            children=[
                Text(
                    "Quarterly report",
                    style=SoulStyle(
                        font_size_pt=24,
                        font_weight="bold",
                        text_align="center",
                    ),
                ),
                Text(
                    "The same component tree can be rendered to every "
                    "supported output format."
                ),
                Table(
                    rows=[
                        ["Product", "Quantity", "Amount"],
                        ["Service A", "12", "$1,200"],
                        ["Service B", "8", "$960"],
                    ],
                    header_rows=1,
                ),
            ],
        )
    ],
)

save_document(document, "report.html")
save_document(document, "report.svg")
save_document(document, "report.pdf")
save_document(document, "report.docx")
save_document(document, "report.xlsx")
```

`save_document()` selects the backend from the file extension.

## Positioned IR example

Use `SoulDocument` directly when the application already knows all
coordinates:

```python
from souldoc import (
    SoulBox,
    SoulContentKind,
    SoulDocument,
    SoulPage,
    dumps,
    loads,
)

document = SoulDocument(
    name="Contract",
    orientation="portrait",
    pages=[
        SoulPage(
            number=1,
            width_mm=210,
            height_mm=297,
            boxes=[
                SoulBox(
                    id="title",
                    page=1,
                    left_mm=20,
                    top_mm=15,
                    width_mm=170,
                    height_mm=20,
                    content="Contract",
                    content_kind=SoulContentKind.TEXT,
                    style=(
                        "font-size:24pt;"
                        "font-weight:bold;"
                        "text-align:center;"
                    ),
                )
            ],
        )
    ],
)

payload = dumps(document)
restored = loads(payload)
```

This object is not tied to any output format.

## Public API

The dependency-free package exports the core IR, layout model, serializers,
TenTags helpers, and unified export functions:

```python
from souldoc import (
    Absolute,
    ApproximateTextMeasurer,
    Column,
    EdgeInsets,
    Grid,
    Image,
    Markdown,
    PillowTextMeasurer,
    Row,
    SoulBox,
    SoulContentKind,
    SoulDocument,
    SoulLayoutDocument,
    SoulPage,
    SoulRecord,
    SoulResource,
    SoulStyle,
    Table,
    TableCell,
    Text,
    contains_markdown_formula,
    detect_content_kind,
    document_from_dict,
    document_from_dicts,
    document_from_records,
    document_to_dict,
    dumps,
    extract_markdown,
    layout_document,
    loads,
    markdown_formula_to_html,
    markdown_to_html,
    parse_tentags,
    render_document,
    save_document,
    tentags_table,
    tentags_to_html,
    validate_document,
)
```

Renderer-specific APIs remain available:

```python
from souldoc.docx import render_docx, save_docx
from souldoc.html import render_html, save_html
from souldoc.pdf import render_pdf, save_pdf
from souldoc.svg import render_svg, render_svg_pages, save_svg
from souldoc.xlsx import render_xlsx, save_xlsx
```

## Output behavior

- **HTML** produces printable, paginated HTML with fixed page geometry.
- **SVG** exposes the calculated geometry as native, editable vector
  primitives compatible with tools such as Inkscape. Rich HTML and Markdown
  are flattened to editable text. `render_svg_pages()` returns every page.
- **PDF** prints the same HTML representation through local Chromium.
- **DOCX** creates editable Word content, including native tables and
  positioned text boxes where possible.
- **XLSX** creates a visually close, editable workbook using cells, merged
  ranges, drawings, and images. A spreadsheet cannot reproduce every
  document-layout behavior exactly.
- **JSON** preserves the versioned, positioned SoulDoc IR.

## JSON contract

SoulDoc can move between Python projects and other languages through a
versioned JSON envelope:

```json
{
  "schema": "souldoc",
  "schema_version": 1,
  "producer": {
    "name": "souldoc",
    "version": "1.0.3"
  },
  "document": {
    "name": "Contract",
    "sheet_name": "Page",
    "orientation": "portrait",
    "width_mm": 210.0,
    "height_mm": 297.0,
    "pages": []
  }
}
```

The library version and schema version evolve independently:

```text
library version: 1.0.3
schema version:  1
```

Unknown fields inside `document`, `page`, `box`, and `resource` objects are
stored in `extra` and returned during serialization. Unknown fields beside
`schema`, `schema_version`, and `document` in the version 1 envelope are not
preserved.

## Resources and images

Binary resources are registered separately from boxes:

```python
from souldoc import SoulDocument, SoulResource

document = SoulDocument(
    name="Document with a logo",
    resources={
        "logo": SoulResource(
            id="logo",
            mime_type="image/png",
            source="data:image/png;base64,...",
        )
    },
)
```

An image box only needs a reference:

```python
SoulBox(
    id="logo-box",
    resource_id="logo",
    left_mm=15,
    top_mm=10,
    width_mm=30,
    height_mm=12,
)
```

When `content` is empty, `resource_id` automatically selects
`content_kind="image"`. A resource source may be a data URL, local path,
HTTP address, object-store identifier, or any application-specific value.
The core does not load it. Renderers receive a `resource_resolver` callback.

When a resource includes `sha256`, SoulDoc validates the digest format and
renderers verify the loaded bytes before embedding them.

## HTML security

SoulDoc Core preserves HTML as data; it is not an HTML sanitizer. Applications
must sanitize untrusted HTML before constructing a document.

HTML, SVG, DOCX, and XLSX load external images only through an explicitly
provided `resource_resolver`. PDF uses the same printable HTML and blocks
Chromium network requests.

## Package boundaries

The core must not import:

```text
PyCells
SQLAlchemy
FastAPI
Playwright
application database models
network clients
```

Integrations belong in adapters:

```text
application ORM model
        |
        v
application adapter
        |
        v
SoulRecord / SoulLayoutDocument / SoulDocument
```

The PyCells adapter lives in the PyCells project and is responsible for
database access. The dependency points in one direction: PyCells knows about
SoulDoc, while SoulDoc does not know about PyCells.

The fields `note` and `note_style` are intentionally outside SoulDoc because
they belong to a spreadsheet editor rather than a freely positioned document.

## Package structure

```text
souldoc/
    __init__.py      public API
    ir.py            positioned document model and JSON contract
    layout.py        component model, measurement, and pagination
    markdown.py      PyCells MD formula and Markdown conversion
    tentags.py       independent TenTags parser
    _render.py       renderer input normalization
    _resources.py    controlled resource resolution
    html.py          printable HTML renderer
    svg.py           SVG renderer
    pdf.py           Chromium PDF renderer
    docx.py          editable DOCX renderer
    xlsx.py          visual editable XLSX renderer
    export.py        unified format dispatch
```

## Implemented milestones

### Stage 1: dependency-free core

- generalized input record loading;
- strict geometry validation;
- JSON serialization;
- `schema_version=1`;
- unit tests.

### Stage 2: editable DOCX

- positioned text boxes;
- plain text, HTML, and hyperlinks;
- native Word tables;
- controlled resource loading;
- editable LaTeX source text.

### Stage 3: layout and HTML

- `SoulLayoutDocument` and shared `SoulStyle`;
- `Text`, `Markdown`, `Image`, `Table`, `Row`, `Column`, `Grid`, and
  `Absolute`;
- text measurement, wrapping, and pagination;
- repeated headers and footers;
- dependency-free printable HTML.

### Stage 4: multi-format export

- SVG as the geometry reference;
- PDF through printable HTML;
- visual XLSX with merged cells and drawing shapes;
- unified export API;
- TenTags table integration.
- PyCells `MD(...)` recognition and dependency-free Markdown fallback.
- Explicit `SoulContentKind.MARKDOWN` and `Markdown` layout component.

## Planned work

- document templates;
- layers and reusable components;
- block grouping;
- rotation and transparency;
- native OMML;
- more precise orphan, widow, and complex-font fallback handling;
- pixel-based regression testing in Microsoft Word, LibreOffice, and Excel.

## Design principles

1. **Documents before formats.** Describe meaning and geometry first, then
   choose an output format.
2. **Editability before snapshots.** Use native objects whenever an output
   format supports them.
3. **Millimetres as stable geometry.** Layout must not depend on screen DPI.
4. **A small core.** Databases, networking, and heavy renderers do not belong
   in the IR.
5. **A versioned contract.** Stored SoulDoc documents should remain readable
   by future versions.
6. **Extensibility without core rewrites.** New renderers consume the model
   instead of changing it.
7. **No single-project coupling.** PyCells is the first consumer of SoulDoc,
   not part of its core.

## Testing

The automated suite covers:

- portrait and landscape documents;
- page and `z_index` ordering;
- unknown fields from future versions;
- invalid coordinates and dimensions;
- Unicode and multiline text;
- HTML, native tables, hyperlinks, images, and formulas;
- measurement, flow layout, and automatic pagination;
- repeated headers and footers;
- table splitting;
- HTML, SVG, PDF, DOCX, and XLSX output;
- the unified export API;
- equivalent geometry in native coordinates across formats.

Opening generated DOCX and XLSX files in Microsoft Word, LibreOffice, and
Microsoft Excel remains a manual cross-application release check.

## Python compatibility

SoulDoc supports Python 3.10 through Python 3.14.

## License

SoulDoc is distributed under the Apache License 2.0. See `LICENSE` for the full
license and `NOTICE` for the copyright notice.

## Authorship

Copyright 2026 Zhandos Mambetali
