Metadata-Version: 2.5
Name: snowflake-spark-connect
Version: 0.1.2
Summary: Python client for Snowflake Runtime for Apache Spark
Project-URL: Homepage, https://github.com/snowflake-eng/sras-client
Project-URL: Repository, https://github.com/snowflake-eng/sras-client
Project-URL: Issues, https://github.com/snowflake-eng/sras-client/issues
Author-email: "Snowflake, Inc" <snowflake-python-libraries-dl@snowflake.com>
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries
Requires-Python: <3.13,>=3.10
Requires-Dist: googleapis-common-protos>=1.56.4
Requires-Dist: grpcio-status>=1.56.0
Requires-Dist: grpcio>=1.56.0
Requires-Dist: numpy<2,>=1.15
Requires-Dist: pandas>=1.0.5
Requires-Dist: py4j==0.10.9.7
Requires-Dist: pyarrow<26,>=4.0.0
Requires-Dist: pyjwt>=2.15.0
Requires-Dist: setuptools>=68; python_version >= '3.12'
Requires-Dist: snowflake-connector-python>=4.7.3
Requires-Dist: snowflake-snowpark-python
Requires-Dist: urllib3>=2.8.0
Description-Content-Type: text/markdown

# Snowflake Spark Connect Python Client

> [!IMPORTANT]
> This client is in Private Preview. It is not yet in production and is available only to selected accounts.

```bash
pip install snowflake-spark-connect
```

Supports Python 3.10, 3.11, and 3.12.

## Usage

```python
from snowflake.sparkconnect import SparkSession

spark = SparkSession.builder.getOrCreate()

spark.sql("SELECT 1").show()
spark.stop()
```

`getOrCreate()` starts a Spark Connect session in Snowflake, returning once the session is ready for us.

## Builder options

### Authentication

Use exactly one of the following authentication options when creating a session to customize the connection.

| Method | What it does |
| --- | --- |
| *(nothing)* | `getOrCreate()` returns the default connection in the environment |
| `.snowpark_session(session)` | Use an existing Snowpark session object |
| `.connection("prod")` | Named profile from `~/.snowflake/connections.toml` |
| `.remote(account=..., ...)` | Snowflake connection parameters (see below) |

Once a Spark Connect session is established, token refresh happens in the background to ensure the session stays authenticated.

### Session lifetime

| Option | Meaning |
| --- | --- |
| `.idlettl(timedelta(...))` | Stop the Spark server after this much idle time |
| `.sessionttl(timedelta(...))` | Hard cap on total session lifetime |
| `.tokenttl(timedelta(...))` | Lifetime of each Spark Connect JWT used for authentication |
| `spark_idlettl` / `spark_sessionttl` / `spark_tokenttl` inside `.remote(...)` | Same TTLs, passed as Snowflake kwargs |

### Database and schema

Spark Connect sessions must be started with a database and schema. `.database(name)` and `.schema(name)` on the builder can be used to specify a database and schema combination to use; an exception is raised if only one is provided. When neither is set, the user's [personal database](https://docs.snowflake.com/en/user-guide/personal-databases) `USER$<current user>.PUBLIC` is used by default.

| Option | Meaning |
| --- | --- |
| `.database(name)` | Use this Snowflake database for the Spark Connect server |
| `.schema(name)` | Use this Snowflake schema for the Spark Connect server |

Explicit `.database(...)` and `.schema(...)` values always override database/schema from a connection, configuration, or supplied/active Snowpark session.

### Artifact repository and Python packages

These are sent on `START_SPARK_CONNECT_SERVER` and installed on the Connect driver and on pushdown UDFs. They are omitted when unset. Packages require an artifact repository.

| Option | Meaning |
| --- | --- |
| `.artifact_repository(name)` | One Artifact Repository name, bound as `ARTIFACT_REPOSITORY` |
| `.python_packages([...])` | Pip specs, bound as `PYTHON_PACKAGES` |
| `spark_artifact_repository` / `spark_python_packages` inside `.remote(...)` | Same values, passed as Snowflake kwargs |

Attaching to an existing `sc://` URL rejects them, because attach does not START a server. `getOrCreate()` also rejects them when a session already exists. Leave both unset to reuse that session.

### `.remote(**kwargs)` Snowflake parameters

These are the same parameters as [`snowpark-python`](https://github.com/snowflakedb/snowpark-python).

Spark-specific kwargs:

| Parameter | Notes |
| --- | --- |
| `spark_idlettl` / `spark_sessionttl` / `spark_tokenttl` | Same as `.idlettl()` / `.sessionttl()` / `.tokenttl()` |
| `spark_artifact_repository` / `spark_python_packages` | Same as `.artifact_repository()` / `.python_packages()` |
