Metadata-Version: 2.4
Name: spark-logical-plan-capture-v2
Version: 0.1.1
Summary: PySpark package for Catalyst logical plan capture with Base64 logging marker SPARK_LOGICAL_PLAN_CAPTURE_V2.
Project-URL: Homepage, https://github.com/feature-oriented-method/catalyst_logic_plan
Project-URL: Repository, https://github.com/feature-oriented-method/catalyst_logic_plan
Author: mtfer
License-Expression: MIT
Keywords: base64,catalyst,logging,pyspark,spark,sql
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: <3.12,>=3.11
Requires-Dist: pyspark==3.5.2
Description-Content-Type: text/markdown

# spark-logical-plan-capture-v2

PyPI-пакет для PySpark 3.5.x, который подключает Spark SQL extension:

- перехватывает логические планы Catalyst
- сериализует payload в Java binary form
- кодирует payload в Base64
- пишет маркер в лог: `SPARK_LOGICAL_PLAN_CAPTURE_V2:<base64>`

## Installation

```bash
pip install spark-logical-plan-capture-v2
```

## Usage

```python
from pyspark.sql import SparkSession
from spark_logical_plan_capture import configure_spark_builder

builder = SparkSession.builder.appName("capture-demo").master("local[*]")
spark = configure_spark_builder(builder).getOrCreate()

spark.sql("select 1 as value").show()
```

## Decode SQL from log line

```python
from spark_logical_plan_capture import decode_captured_sql_from_logline

log_line = "SPARK_LOGICAL_PLAN_CAPTURE_V2:<base64>"
sql = decode_captured_sql_from_logline(spark, log_line)
print(sql)
```

## Build and publish

1. Соберите JVM jar:

```bash
sbt clean test package
```

2. Скопируйте jar в Python package data:

```bash
python scripts/prepare_python_package.py
```

3. Соберите wheel/sdist:

```bash
python -m build
```

4. Опубликуйте:

```bash
python -m twine upload dist/*
```
