Metadata-Version: 2.5
Name: dataframeit
Version: 0.10.0
Summary: Enrich DataFrames with LLM-based text analysis. Extract structured information from text with Pydantic.
Project-URL: Homepage, https://github.com/bdcdo/dataframeit
Project-URL: Documentation, https://bdcdo.github.io/dataframeit/en/
Project-URL: Repository, https://github.com/bdcdo/dataframeit.git
Project-URL: Issues, https://github.com/bdcdo/dataframeit/issues
Project-URL: Changelog, https://github.com/bdcdo/dataframeit/blob/main/CHANGELOG.md
Author: Bruno da Cunha de Oliveira
Maintainer: Bruno da Cunha de Oliveira
License-Expression: MIT
License-File: LICENSE
Keywords: data-enrichment,dataframe,gemini,langchain,llm,nlp,pandas,pydantic,structured-output,text-extraction
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: langchain-core>=1.2.10
Requires-Dist: langchain>=1.2.11
Requires-Dist: pandas>=2.1.2
Requires-Dist: pydantic>=2.11.0
Requires-Dist: tqdm>=4.1.0
Provides-Extra: all
Requires-Dist: claude-agent-sdk>=0.1.48; extra == 'all'
Requires-Dist: langchain-anthropic>=0.3.21; extra == 'all'
Requires-Dist: langchain-exa>=1.0.0; extra == 'all'
Requires-Dist: langchain-google-genai>=2.1.11; extra == 'all'
Requires-Dist: langchain-groq>=1.0.0; extra == 'all'
Requires-Dist: langchain-openai>=0.3.34; extra == 'all'
Requires-Dist: langchain-tavily>=0.2.12; extra == 'all'
Requires-Dist: openpyxl>=3.1; extra == 'all'
Requires-Dist: polars>=0.20; extra == 'all'
Requires-Dist: pyarrow>=10; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: langchain-anthropic>=0.3.21; extra == 'anthropic'
Provides-Extra: claude-code
Requires-Dist: claude-agent-sdk>=0.1.48; extra == 'claude-code'
Provides-Extra: codex
Requires-Dist: filelock>=3.13; extra == 'codex'
Requires-Dist: openai-codex-cli-bin>=0.137.0a4; extra == 'codex'
Requires-Dist: openai-codex==0.1.0b3; extra == 'codex'
Provides-Extra: dev
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff<0.17,>=0.16; extra == 'dev'
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
Requires-Dist: mkdocs-static-i18n>=1.2.0; extra == 'docs'
Requires-Dist: mkdocs>=1.5.0; extra == 'docs'
Provides-Extra: excel
Requires-Dist: openpyxl>=3.1; extra == 'excel'
Provides-Extra: google
Requires-Dist: langchain-google-genai>=2.1.11; extra == 'google'
Provides-Extra: groq
Requires-Dist: langchain-groq>=1.0.0; extra == 'groq'
Provides-Extra: openai
Requires-Dist: langchain-openai>=0.3.34; extra == 'openai'
Provides-Extra: polars
Requires-Dist: polars>=0.20; extra == 'polars'
Requires-Dist: pyarrow>=10; extra == 'polars'
Provides-Extra: search
Requires-Dist: langchain-tavily>=0.2.12; extra == 'search'
Provides-Extra: search-all
Requires-Dist: langchain-exa>=1.0.0; extra == 'search-all'
Requires-Dist: langchain-tavily>=0.2.12; extra == 'search-all'
Provides-Extra: search-exa
Requires-Dist: langchain-exa>=1.0.0; extra == 'search-exa'
Description-Content-Type: text/markdown

# DataFrameIt

[![PyPI version](https://badge.fury.io/py/dataframeit.svg)](https://badge.fury.io/py/dataframeit)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**English** · [Português](https://github.com/bdcdo/dataframeit/blob/main/README.pt-BR.md) · [Español](https://github.com/bdcdo/dataframeit/blob/main/README.es.md)

**Enrich DataFrames with LLMs, simply and in a structured way.**

DataFrameIt processes text in DataFrames using Large Language Models (LLMs) and extracts structured information defined by Pydantic models.

**[Full Documentation](https://bdcdo.github.io/dataframeit/en/)** | **[LLM Reference](https://bdcdo.github.io/dataframeit/en/reference/llm-reference/)**

## Installation

```bash
pip install dataframeit[openai]  # OpenAI (default provider)
pip install dataframeit[google]  # Google Gemini
pip install dataframeit[anthropic]  # Anthropic Claude
pip install dataframeit[codex]  # Official Codex SDK (experimental)
pip install dataframeit[claude-code]  # Claude Code via the Claude Agent SDK
```

Set up provider authentication:

```bash
export OPENAI_API_KEY="your-key"  # or GOOGLE_API_KEY, ANTHROPIC_API_KEY
```

The experimental `codex` provider is optional, is not part of the `all` extra, uses the bundled runtime and requires local file-based authentication. See the [installation docs](https://bdcdo.github.io/dataframeit/en/getting-started/installation/) to set up the extra and the credentials.

## Quick Example

```python
from pydantic import BaseModel
from typing import Literal
import pandas as pd
from dataframeit import dataframeit

# 1. Define what to extract
class Sentiment(BaseModel):
    sentiment: Literal['positive', 'negative', 'neutral']
    confidence: Literal['high', 'medium', 'low']

# 2. Your data
df = pd.DataFrame({
    'text': [
        'Excellent product! Exceeded my expectations.',
        'Terrible service, never buying again.',
        'Delivery was fine, product is average.'
    ]
})

# 3. Process!
result = dataframeit(df, Sentiment, "Analyze the sentiment of the text.")
print(result)
```

**Output:**

| text | sentiment | confidence |
|------|-----------|------------|
| Excellent product! ... | positive | high |
| Terrible service... | negative | high |
| Delivery was fine... | neutral | medium |

Field and class names are arbitrary — the examples in the [`example/`](https://github.com/bdcdo/dataframeit/tree/main/example) notebooks use Portuguese ones.

## Features

- **Multiple providers**: Google Gemini, OpenAI, Anthropic, Cohere and Mistral via LangChain, plus Claude Code and Codex through their SDKs
- **Multiple input types**: DataFrame, Series, list, dict
- **Structured output**: Automatic validation with Pydantic
- **Resilience**: Automatic retry with exponential backoff
- **Performance**: Parallel processing, configurable rate limiting
- **Web search**: Tavily integration to enrich data
- **Tracking**: Token monitoring and throughput metrics
- **Per-field configuration**: Custom prompts and search parameters per field (v0.5.2+)

## Per-Field Configuration (New in v0.5.2)

Set field-specific prompts and search parameters using `json_schema_extra`:

```python
from pydantic import BaseModel, Field

class DrugInfo(BaseModel):
    # Field with the default prompt
    active_ingredient: str = Field(description="Active ingredient of the drug")

    # Field with a custom prompt (replaces the base prompt)
    rare_disease: str = Field(
        description="Rare disease classification",
        json_schema_extra={
            "prompt": "Search Orphanet (orpha.net). Analyze: {texto}"
        }
    )

    # Field with an additional prompt (appended to the base prompt)
    conitec_assessment: str = Field(
        description="CONITEC assessment",
        json_schema_extra={
            "prompt_append": "Search ONLY the CONITEC website (gov.br/conitec)."
        }
    )

    # Field with custom search parameters
    clinical_trials: str = Field(
        description="Relevant clinical trials",
        json_schema_extra={
            "prompt_append": "Search for recent clinical trials.",
            "search_depth": "advanced",
            "max_results": 10
        }
    )

# Requires search_per_field=True
result = dataframeit(
    df,
    DrugInfo,
    "Analyze the drug: {texto}",
    use_search=True,
    search_per_field=True,
)
```

**Available options in `json_schema_extra`:**

| Option | Description |
|--------|-------------|
| `prompt` or `prompt_replace` | Fully replaces the base prompt |
| `prompt_append` | Appends text to the base prompt |
| `search_depth` | `"basic"` or `"advanced"` (per-field override) |
| `max_results` | Number of search results (1-20) |

## Documentation

- [Quickstart](https://bdcdo.github.io/dataframeit/en/getting-started/quickstart/)
- [Guides](https://bdcdo.github.io/dataframeit/en/guides/basic-usage/)
- [API Reference](https://bdcdo.github.io/dataframeit/en/reference/api/)
- [LLM Reference](https://bdcdo.github.io/dataframeit/en/reference/llm-reference/) - Compact page optimized for coding assistants

## Examples

See the [`example/`](https://github.com/bdcdo/dataframeit/tree/main/example) folder for Jupyter notebooks with complete use cases.

## License

MIT
