Metadata-Version: 2.5
Name: streamcase
Version: 0.2.0a1
Summary: Deterministic testing for Apache Spark Structured Streaming pipelines
Project-URL: Homepage, https://github.com/SEPURI-SAI-KRISHNA/streamcase
Project-URL: Documentation, https://github.com/SEPURI-SAI-KRISHNA/streamcase/tree/main/docs
Project-URL: Issues, https://github.com/SEPURI-SAI-KRISHNA/streamcase/issues
Project-URL: Repository, https://github.com/SEPURI-SAI-KRISHNA/streamcase
Project-URL: Changelog, https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/CHANGELOG.md
Project-URL: Security, https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/SECURITY.md
Author: Sepuri Sai Krishna
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: apache-spark,pyspark,pytest,structured-streaming,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Provides-Extra: dev
Requires-Dist: build<2,>=1.2; extra == 'dev'
Requires-Dist: mypy<2,>=1.15; extra == 'dev'
Requires-Dist: pytest-cov<8,>=6; extra == 'dev'
Requires-Dist: pytest<10,>=8.3; extra == 'dev'
Requires-Dist: ruff<1,>=0.11; extra == 'dev'
Requires-Dist: twine<8,>=7; extra == 'dev'
Provides-Extra: pytest
Requires-Dist: pytest<10,>=8.3; extra == 'pytest'
Provides-Extra: spark
Requires-Dist: pyspark<4.3,>=4.2; extra == 'spark'
Provides-Extra: spark35
Requires-Dist: pyspark<3.6,>=3.5.9; extra == 'spark35'
Provides-Extra: test
Requires-Dist: pytest-cov<8,>=6; extra == 'test'
Requires-Dist: pytest<10,>=8.3; extra == 'test'
Description-Content-Type: text/markdown

# Streamcase

[![CI](https://github.com/SEPURI-SAI-KRISHNA/streamcase/actions/workflows/ci.yml/badge.svg)](https://github.com/SEPURI-SAI-KRISHNA/streamcase/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/LICENSE)

Streamcase is an early-development pytest-oriented toolkit for deterministic Apache Spark
Structured Streaming tests.

> **Alpha status:** Streamcase is available on PyPI, but its API remains
> experimental and may change in later alphas. The features in a source
> checkout can be newer than a published package. Check the release notes for
> the exact capabilities and tested combinations of the version you install.

The [0.2.0a1 release notes](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/releases/0.2.0a1.md)
describe its expanded alpha scope and limitations. Check
[PyPI](https://pypi.org/project/streamcase/) to confirm that version is
available before installing it; use a source checkout to try unpublished work.

## Problem

Ordinary DataFrame assertions do not exercise streaming semantics such as
micro-batch boundaries, late data, watermarks, checkpoint restarts, or state
growth. Streamcase will make those behaviors explicit and reproducible in Python
tests while executing against Spark's public Structured Streaming interfaces.

## Installation

Install the lightweight backend-independent package from PyPI. The `--pre`
option allows alpha releases; pin a version when reproducibility matters:

```shell
python -m pip install --pre streamcase
```

For the Spark runner, install the optional `spark` extra and provide a
compatible Java JDK separately:

```shell
python -m pip install --pre "streamcase[spark]"
```

The `0.2.0a1` release adds a separate `spark35` extra for PySpark 3.5.9 or
later 3.5 patches. Once that version is available on PyPI, install it with:

```shell
python -m pip install "streamcase[spark35]==0.2.0a1"
```

For the opt-in pytest fixture, combine its `pytest` extra with exactly one
Spark extra. For the Spark 4.2 line of `0.2.0a1`:

```shell
python -m pip install "streamcase[pytest,spark]==0.2.0a1"
```

See the [pytest integration guide](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/pytest-integration.md)
for explicit activation, a runnable example, and session ownership. For the
current source before `0.2.0a1` is published, use a checkout and install
`python -m pip install -e ".[pytest,spark]"` instead. The older `0.1.0a2`
distribution does not include the `pytest` or `spark35` extras.

These extras keep PySpark out of the core dependency set. See the
[Spark compatibility policy](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/spark-compatibility.md) for the supported
Python, Java, and Spark combination. Contributors working from a checkout can
use the [editable-install instructions](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/CONTRIBUTING.md#development-setup).

## Current source capabilities

Streamcase can describe immutable logical input batches and checkpoint-preserving
restart boundaries, represent captured output, and assert row equality, batch
counts, unique keys, and captured progress metrics without importing PySpark.
The public Spark runner executes batches and restarts against a caller-owned
session. It also offers an opt-in pytest runner fixture. See the
[scenario model guide](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/scenario-model.md) and
[results and assertions guide](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/results-and-assertions.md), plus the
[progress and state assertions guide](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/progress-and-state-assertions.md),
[two-batch Spark quick start and runner contract](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/spark-runner.md) and
[checkpoint restart example](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/restart-scenarios.md),
and [pytest integration guide](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/pytest-integration.md).

## Repository status

This bootstrap establishes:

- Python packaging with a `src` layout;
- Apache License 2.0 and project governance;
- issue and pull-request conventions;
- formatting, linting, type checking, tests, and package validation in CI;
- security and release policies.

See [ROADMAP.md](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/ROADMAP.md) for the planned delivery sequence and
[docs/architecture.md](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/docs/architecture.md) for the proposed technical shape.

## Local checks

```shell
python -m venv .venv
python -m pip install -e ".[dev]"
ruff format --check .
ruff check .
mypy
pytest -m "not spark"
python -m build
python -m twine check dist/*
```

Spark contributors can run the isolated integration suite with the setup in the
[contribution guide](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/CONTRIBUTING.md#development-setup).

## Contributing

All material work starts with a GitHub issue and reaches `main` only through a
reviewed pull request. Read [CONTRIBUTING.md](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/CONTRIBUTING.md) before making a
change.

## Trademark notice

Apache Spark, Spark, Apache Flink, and Flink are trademarks of the Apache Software
Foundation. Streamcase is independent software and is not endorsed by the Apache
Software Foundation.

## License

Licensed under the [Apache License 2.0](https://github.com/SEPURI-SAI-KRISHNA/streamcase/blob/main/LICENSE).
