Metadata-Version: 2.4
Name: sqlframe
Version: 4.9.0
Summary: Turning PySpark Into a Universal DataFrame API
Author-email: Ryan Eakman <eakmanrq@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/eakmanrq/sqlframe
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: SQL
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: prettytable<4
Requires-Dist: sqlglot<30.26,>=30.0.0
Requires-Dist: typing_extensions
Requires-Dist: more-itertools
Provides-Extra: bigquery
Requires-Dist: google-cloud-bigquery[pandas]<4,>=3; extra == "bigquery"
Requires-Dist: google-cloud-bigquery-storage<3,>=2; extra == "bigquery"
Provides-Extra: clickhouse
Requires-Dist: clickhouse-connect<2,>=0.13; extra == "clickhouse"
Provides-Extra: databricks
Requires-Dist: databricks-sql-connector[pyarrow]<5,>=3.6; extra == "databricks"
Provides-Extra: dev
Requires-Dist: chdb<5,>=4.4; sys_platform != "win32" and extra == "dev"
Requires-Dist: duckdb<2.1,>=1.2; extra == "dev"
Requires-Dist: findspark<3,>=2; extra == "dev"
Requires-Dist: mirakuru<3.0.5; extra == "dev"
Requires-Dist: ty>=0.0.18; extra == "dev"
Requires-Dist: openai<4,>=1.30; extra == "dev"
Requires-Dist: pandas<4,>=2; extra == "dev"
Requires-Dist: pandas-stubs<4,>=2; extra == "dev"
Requires-Dist: psycopg<4,>=3.1; extra == "dev"
Requires-Dist: pyarrow<27,>=10; extra == "dev"
Requires-Dist: pyspark<4.3,>=4; extra == "dev"
Requires-Dist: pytest<9.2,>=8.2.0; extra == "dev"
Requires-Dist: pytest-forked; extra == "dev"
Requires-Dist: pytest-postgresql<10,>=6; extra == "dev"
Requires-Dist: pytest-rerunfailures; extra == "dev"
Requires-Dist: pytest-xdist<3.9,>=3.6; extra == "dev"
Requires-Dist: pre-commit<5,>=3.7; extra == "dev"
Requires-Dist: ruff<0.18,>=0.4.4; extra == "dev"
Requires-Dist: types-psycopg2<3,>=2.9; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs==1.4.2; extra == "docs"
Requires-Dist: mkdocs-include-markdown-plugin==6.0.6; extra == "docs"
Requires-Dist: mkdocs-material==9.0.5; extra == "docs"
Requires-Dist: mkdocs-material-extensions==1.1.1; extra == "docs"
Requires-Dist: pymdown-extensions; extra == "docs"
Provides-Extra: duckdb
Requires-Dist: duckdb<2.1,>=1.2; extra == "duckdb"
Requires-Dist: pandas<4,>=2; extra == "duckdb"
Provides-Extra: openai
Requires-Dist: openai<4,>=1.30; extra == "openai"
Provides-Extra: pandas
Requires-Dist: pandas<4,>=2; extra == "pandas"
Provides-Extra: postgres
Requires-Dist: psycopg2<3,>=2.8; extra == "postgres"
Provides-Extra: redshift
Requires-Dist: redshift_connector<2.3.0,>=2.1.14; extra == "redshift"
Provides-Extra: snowflake
Requires-Dist: snowflake-connector-python[secure-local-storage]<6,>=4.7.3; extra == "snowflake"
Provides-Extra: spark
Requires-Dist: pyspark<4.3,>=4; extra == "spark"
Provides-Extra: standalone
Requires-Dist: pyarrow<27,>=10; extra == "standalone"
Provides-Extra: trino
Requires-Dist: trino<1,>=0.330; extra == "trino"
Dynamic: license-file

<div align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/eakmanrq/sqlframe/main/docs/images/wordmark-dark.svg">
    <img src="https://raw.githubusercontent.com/eakmanrq/sqlframe/main/docs/images/wordmark.svg" alt="SQLFrame" width="270" height="56">
  </picture>

  <h3>Your PySpark code. Your database.</h3>
  <p>Run the PySpark DataFrame API directly on your database.<br>No Spark cluster required.</p>

  <p>
    <a href="https://sqlframe.readthedocs.io/en/latest/quickstart/">Quickstart</a> ·
    <a href="https://sqlframe.readthedocs.io/en/latest/">Documentation</a> ·
    <a href="https://github.com/eakmanrq/sqlframe/issues">Report an issue</a>
  </p>
</div>

## Why SQLFrame?

- **Familiar API.** Write transformations with PySpark DataFrame operations, joins, and window functions.
- **Direct execution.** Run your transformations on your database engine.
- **Visible SQL.** Inspect, debug, and share the queries behind your DataFrame code.

## Try it locally

Install SQLFrame with DuckDB. You need Python 3.10 or later; no cloud account or Spark installation is required.

```bash
pip install "sqlframe[duckdb]"
```

```python
from sqlframe.duckdb import DuckDBSession

session = DuckDBSession()
df = session.createDataFrame(
    [("US", 120), ("UK", 80), ("US", 30)],
    ["country", "revenue"],
)
df.groupBy("country").sum("revenue").orderBy("country").show()
```

```text
+---------+--------------+
| country | sum(revenue) |
+---------+--------------+
|    UK   |      80      |
|    US   |     150      |
+---------+--------------+
```

To inspect a transformation's SQL, call `df.sql(optimize=True)`. Follow the [quickstart](https://sqlframe.readthedocs.io/en/latest/quickstart/) for a complete walkthrough.

## Choose your engine

| Engine | Install |
| :----- | :------ |
| [DuckDB](https://sqlframe.readthedocs.io/en/latest/duckdb/) | `pip install "sqlframe[duckdb]"` |
| [BigQuery](https://sqlframe.readthedocs.io/en/latest/bigquery/) | `pip install "sqlframe[bigquery]"` |
| [Snowflake](https://sqlframe.readthedocs.io/en/latest/snowflake/) | `pip install "sqlframe[snowflake]"` |
| [Postgres](https://sqlframe.readthedocs.io/en/latest/postgres/) | `pip install "sqlframe[postgres]"` |
| [ClickHouse (preview)](https://sqlframe.readthedocs.io/en/latest/clickhouse/) | `pip install "sqlframe[clickhouse]"` |
| [Databricks](https://sqlframe.readthedocs.io/en/latest/databricks/) | `pip install "sqlframe[databricks]"` |
| [Spark](https://sqlframe.readthedocs.io/en/latest/spark/) | `pip install "sqlframe[spark]"` |
| [Standalone](https://sqlframe.readthedocs.io/en/latest/standalone/) | `pip install sqlframe` |
| [Trino (preview)](https://sqlframe.readthedocs.io/en/latest/trino/) | `pip install "sqlframe[trino]"` |

Standalone sessions generate SQL without a database connection. For local Parquet support, install `sqlframe[standalone]`.

ClickHouse and Trino are preview integrations; see their guides for supported workflows and limits.

Community integrations: [GizmoSQL](https://github.com/gizmodata/sqlframe-gizmosql) · [Redshift](https://sqlframe.readthedocs.io/en/latest/redshift/) (`pip install "sqlframe[redshift]"`).

SQLFrame is also available on conda-forge: `conda install -c conda-forge sqlframe`.

## Already using PySpark?

Activate SQLFrame before importing PySpark to run supported transformations on another engine.

```python
from sqlframe import activate

activate("duckdb")

from pyspark.sql import SparkSession

session = SparkSession.builder.getOrCreate()
```

You can also import SQLFrame directly, as shown in the quickstart. See the [activation guide](https://sqlframe.readthedocs.io/en/latest/configuration/#activating-sqlframe) and your engine's [API compatibility reference](https://sqlframe.readthedocs.io/en/latest/reference/) for details.

## Keep exploring

- [Examples](https://sqlframe.readthedocs.io/en/latest/examples/): longer pipelines with aggregations and window functions.
- [Generated SQL](https://sqlframe.readthedocs.io/en/latest/configuration/#generated-sql): formatting, optimization, dialect overrides, and OpenAI enrichment.
- [Input and output dialects](https://sqlframe.readthedocs.io/en/latest/configuration/#input-and-output-dialect): configure SQLFrame for your engine's conventions.
- [Standalone SQL generation](https://sqlframe.readthedocs.io/en/latest/standalone/): generate queries without connecting to an engine.

SQLFrame is open source under the [MIT license](https://github.com/eakmanrq/sqlframe/blob/main/LICENSE).
