Metadata-Version: 2.4
Name: airflow-provider-quanton-spark-k8s
Version: 0.0.2
Summary: Airflow provider for Onehouse QuantonSparkApplication Kubernetes CRDs
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: apache-airflow>=2.7.0
Requires-Dist: apache-airflow-providers-cncf-kubernetes>=8.0.0
Requires-Dist: kubernetes>=28.1.0

# airflow-provider-quanton-spark-k8s
This airflow provider will make it easy to run a quanton spark job on kubernetes using airflow. 

## Requirements
Apache Airflow >= 2.7.0
Python >= 3.10

## Installation
```
pip install airflow-provider-quanton-spark-k8s
```

## Usage
Basic Example DAG

```
from datetime import datetime

from airflow import DAG
from quanton_provider.operators.quanton_spark_application import (
    QuantonSparkApplicationOperator,
)

with DAG(
    dag_id="test_quanton_spark_pi",
    start_date=datetime(2026, 3, 1),
    schedule=None,
    catchup=False,
    tags=["quanton", "spark", "test"],
) as dag:
    run_spark_pi = QuantonSparkApplicationOperator(
        task_id="run_spark_pi",
        name="quanton-spark-pi-java-example",
        namespace="spark-operator",
        timeout=600,
        delete_on_success=True,
        spark_application_spec={
            "type": "Java",
            "mode": "cluster",
            "image": "apache/spark:3.5.0",
            "imagePullPolicy": "IfNotPresent",
            "mainClass": "org.apache.spark.examples.JavaSparkPi",
            "mainApplicationFile": "https://repo1.maven.org/maven2/org/apache/spark/spark-examples_2.10/1.1.1/spark-examples_2.10-1.1.1.jar",
            "arguments": ["100"],
            "sparkVersion": "3.5.0",
            "restartPolicy": {
                "type": "Never",
            },
            "driver": {
                "cores": 1,
                "coreLimit": "1200m",
                "memory": "1024m",
                "labels": {"version": "3.5.0"},
                "serviceAccount": "spark-service-account",
            },
            "executor": {
                "cores": 1,
                "instances": 2,
                "memory": "512m",
                "labels": {"version": "3.5.0"},
            },
        },
    )
```

## Testing

Install the package with test dependencies:

```
pip install -e .
pip install -r requirements-dev.txt
```

Run the test suite:

```
pytest
```

Run with coverage report:

```
pytest --cov=quanton_provider --cov-report=term-missing
```

The project enforces a minimum of 95% code coverage.
