Metadata-Version: 2.4
Name: structx
Version: 0.6.0
Summary: Structured data extraction from text using LLMs and dynamic model generation
Author-email: blacksuan19 <py@aolabs.dev>
Classifier: Development Status :: 5 - Production/Stable
Classifier: Framework :: AsyncIO
Classifier: Framework :: Pydantic
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Database
Classifier: Topic :: Documentation
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: Office/Business
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Topic :: Text Processing :: Markup
Classifier: Topic :: Text Processing
Classifier: Topic :: Utilities
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: instructor>=1.15.4
Requires-Dist: litellm>=1.91.1
Requires-Dist: loguru>=0.7.3
Requires-Dist: openpyxl>=3.1.5
Requires-Dist: pandas>=2.2.3
Requires-Dist: pydantic>=2.11.7
Requires-Dist: pydantic-settings[yaml]>=2.14.2
Requires-Dist: tabulate>=0.9.0
Requires-Dist: tenacity>=9.1.2
Requires-Dist: tqdm>=4.68.4
Provides-Extra: dev
Requires-Dist: black>=25.1.0; extra == "dev"
Requires-Dist: bumpver>=2024.1130; extra == "dev"
Requires-Dist: pip-tools>=7.4.1; extra == "dev"
Requires-Dist: pytest>=9.1.1; extra == "dev"
Requires-Dist: wheel>=0.45.1; extra == "dev"
Provides-Extra: docs
Requires-Dist: docling-slim[convert-core,format-docx,format-html,format-latex,format-markdown,format-opendocument,format-pdf,format-pptx,format-web,models-local]>=2.111.0; extra == "docs"
Requires-Dist: torch>=2.13.0; extra == "docs"
Requires-Dist: torchvision>=0.28.0; extra == "docs"
Requires-Dist: weasyprint>=69.0; extra == "docs"
Provides-Extra: mkdocs
Requires-Dist: mkdocs>=1.6.1; extra == "mkdocs"
Requires-Dist: mkdocs-material>=9.7.6; extra == "mkdocs"
Requires-Dist: mkdocstrings[python]>=0.29.1; extra == "mkdocs"
Requires-Dist: mkdocs-git-revision-date-localized-plugin>=1.4.5; extra == "mkdocs"
Requires-Dist: mkdocs-material[imaging]>=9.7.6; extra == "mkdocs"
Requires-Dist: mkdocs-mermaid2-plugin>=1.2.3; extra == "mkdocs"
Dynamic: license-file

# structx

Advanced structured data extraction from any document using LLMs with multimodal
support.

[![Documentation](https://img.shields.io/badge/docs-mkdocs-blue.svg?style=for-the-badge)](https://structx.aolabs.dev "Documentation")
[![PyPI](https://img.shields.io/badge/PyPi-0.6.0-blue?style=for-the-badge)](https://pypi.org/project/structx "Package")
[![GitHub Actions](https://img.shields.io/badge/github%20actions-%232671E5.svg?style=for-the-badge&logo=githubactions&logoColor=white)](# "Build with GitHub Actions")

`structx` is a powerful Python library for extracting structured data from text,
tables, and documents using Large Language Models (LLMs). It passes existing
PDFs directly to vision-capable models; the optional `docs` extra converts other
document formats to PDF first.

## 🔔 Package rename notice (PyPI)

The PyPI distribution has been renamed from `structx-llm` to `structx`
(September 2025).

- Imports are unchanged: continue using `import structx`
- Document processing now lives in the optional `docs` extra
- Please update your environments and requirement files to use the new name

Upgrade commands:

```bash
pip uninstall -y structx-llm
pip install -U structx
```

If you previously pinned `structx-llm` in requirements or lock files, replace it
with `structx`. Install `structx[docs]` for non-PDF document conversion.

## ✨ Key Features

### 🎯 **Advanced Document Processing**

- **� Multimodal PDF Pipeline**: Passes PDFs directly to vision-capable models
  and converts supported non-PDF documents to PDF
- **🖼️ Vision-Enabled Extraction**: Native instructor multimodal support for
  PDFs and images
- **🔄 Smart Format Detection**: Automatic processing mode selection for best
  results
- **📊 Flexible File Support**: CSV, Excel, JSON, Parquet, raw text, and existing
  PDFs in the base install; DOCX, PPTX, images, and more via `structx[docs]`

### 🚀 **Intelligent Data Extraction**

- **🔄 Dynamic Model Generation**: Create type-safe Pydantic models from natural
  language queries
- **🎯 Automatic Schema Inference**: Intelligent schema generation and
  refinement
- **📊 Complex Data Structures**: Support for nested and hierarchical data
- **🔄 Natural Language Refinement**: Improve models with conversational
  instructions

### ⚡ **Performance & Reliability**

- **🚀 High-Performance Processing**: Threaded sync and native async row requests
- **🔄 Robust Error Handling**: Automatic retry mechanism with exponential
  backoff
- **📈 Token Usage Tracking**: Detailed step-by-step metrics for cost monitoring
- **Flexible Configuration**: Model settings from arguments, YAML, environment
  variables, dotenv files, and secrets through Pydantic Settings
- **🔌 Multiple LLM Providers**: Support through litellm integration

## Installation

```bash
pip install structx
```

For converting DOCX, PowerPoint, OpenDocument, markup, image, and other
non-PDF document formats:

```bash
pip install "structx[docs]"
```

### 🔧 What The Package Provides

- Structured readers for CSV, Excel, JSON, Parquet, and Feather
- Instructor multimodal vision support
- Optional Docling document parsing with CPU-only PyTorch resolution for uv on Linux
- Optional WeasyPrint PDF rendering for non-PDF document formats

## Quick Start

### Basic Text Extraction

```python
from structx import Extractor

# Initialize extractor
extractor = Extractor.from_litellm(
    model="gpt-4o",
    api_key="your-api-key",
    max_retries=3,      # Automatically retry on transient errors
    min_wait=1,         # Start with 1 second wait
    max_wait=10         # Maximum 10 seconds between retries
)

# Extract from text
result = extractor.extract(
    data="System check on 2024-01-15 detected high CPU usage (92%) on server-01.",
    query="extract incident date and details"
)

# Access results
print(f"Successful rows: {result.success_count}")
print(result.data[0].model_dump_json(indent=2))
```

### 📄 Document Processing with Multimodal Support

Install `structx[docs]` before using non-PDF document formats. Existing PDFs can
be passed directly through the multimodal path with the base installation.

```python
# Process a PDF invoice through the multimodal pipeline
result = extractor.extract(
    data="scripts/example_input/S0305SampleInvoice.pdf",
    query="extract the invoice number, total amount, and line items"
)

# Convert a DOCX contract and process with multimodal support
result = extractor.extract(
    data="scripts/example_input/free-consultancy-agreement.docx",
    query="extract parties, effective date, and payment terms"
)
```

### 📊 Token Usage Monitoring

```python
# Check token usage for cost monitoring
usage = result.usage
if usage:
    print(f"Total tokens: {usage.total_tokens}")
    for step, calls in usage.steps.items():
        print(step.value, [call.total_tokens for call in calls])
```

## 🚀 Why Multimodal PDF Processing?

The innovative multimodal approach provides significant advantages over
traditional text-based extraction:

- **📄 Context Preservation**: Full document layout and structure are maintained
- **🎯 Higher Accuracy**: Vision models can interpret tables, charts, and
  complex layouts
- **🔄 No Chunking Issues**: Eliminates problems with information split across
  chunks
- **📊 Universal Format**: Existing PDFs are passed through directly; supported
  non-PDF documents become processable through PDF conversion
- **🖼️ Visual Understanding**: Handles documents with visual elements,
  formatting, and structure

## 📚 Documentation

For comprehensive documentation, examples, and guides, visit our
[documentation site](https://structx.aolabs.dev).

- [Getting Started](https://structx.aolabs.dev/getting-started)
- [Basic Extraction](https://structx.aolabs.dev/guides/basic-extraction)
- [Working with Results](https://structx.aolabs.dev/guides/working-with-results)
- [Unstructured Text Processing](https://structx.aolabs.dev/guides/unstructured-text)
- [Async Operations](https://structx.aolabs.dev/guides/async-operations)
- [Multiple Queries](https://structx.aolabs.dev/guides/multiple-queries)
- [Custom Models](https://structx.aolabs.dev/guides/custom-models)
- [Token Usage Tracking](https://structx.aolabs.dev/guides/token-tracking)
- [API Reference](https://structx.aolabs.dev/api/extractor)

## Examples

Check out our [example gallery](https://structx.aolabs.dev/examples) for
real-world use cases,

## 📁 Supported File Formats

### 📊 Structured Data (Direct Processing)

- **CSV**: Comma-separated values with custom delimiters
- **Excel**: .xlsx/.xls with sheet selection and custom options
- **JSON**: JavaScript Object Notation with nested support
- **Parquet**: Columnar storage format for large datasets
- **Feather**: Fast binary format for data frames

### 📄 Unstructured Documents (Multimodal Pipeline)

| Format   | Extensions                                    | Processing Method                     | Quality    |
| -------- | --------------------------------------------- | ------------------------------------- | ---------- |
| **PDF**  | `.pdf`                                        | PDF → Multimodal                      | ⭐⭐⭐⭐⭐ |
| **Word** | `.docx`, `.doc`                               | Docling → HTML → PDF → Multimodal     | ⭐⭐⭐⭐⭐ |
| **PowerPoint** | `.pptx`, `.ppt`                       | Docling → HTML → PDF → Multimodal     | ⭐⭐⭐⭐   |
| **Text** | `.txt`, `.md`, `.py`, `.log`, `.xml`, `.html` | Docling → HTML → PDF → Multimodal     | ⭐⭐⭐⭐   |

### 🔄 Processing Pipeline

- **PDF passthrough**: Existing PDFs are sent directly to multimodal extraction
- **Docling parsing**: Reads non-PDF document-like inputs into a structured document model
- **WeasyPrint rendering**: Converts Docling HTML to a temporary PDF for non-PDF inputs
- **Multimodal extraction**: Sends the rendered PDF to instructor's multimodal API

## Contributing

Contributions are welcome! Please read our
[Contributing Guidelines](https://structx.aolabs.dev/contributing) for
details.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
for details.
