Metadata-Version: 2.4
Name: spark-logical-plan-capture
Version: 0.1.0
Summary: Python wrapper for Spark logical plan capture extension
Author: mt
License: MIT
Project-URL: Homepage, https://pypi.org/project/spark-logical-plan-capture/
Keywords: spark,pyspark,sql,logical-plan,extensions
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: License :: OSI Approved :: MIT License
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# spark-logical-plan-capture

Python package for connecting the Spark extension
`io.github.mt.logicplan.LogicalPlanCaptureExtension` via `spark-submit`
or from PySpark code.

## Install

```bash
pip install spark-logical-plan-capture
```

## Usage in PySpark

```python
from pyspark.sql import SparkSession
from spark_logical_plan_capture import get_spark_conf, get_jar_path

conf = get_spark_conf()

spark = (
    SparkSession.builder
    .config("spark.sql.extensions", conf["spark.sql.extensions"])
    .config("spark.jars", conf["spark.jars"])
    .getOrCreate()
)
```

## Usage with spark-submit

```bash
spark-submit \
  --conf "spark.sql.extensions=io.github.mt.logicplan.LogicalPlanCaptureExtension" \
  --jars "$(python -c 'from spark_logical_plan_capture import get_jar_path; print(get_jar_path())')" \
  your_job.py
```
