Metadata-Version: 2.4
Name: turbine-stream
Version: 0.1.0a1
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: System :: Distributed Computing
Requires-Dist: pyarrow>=24
Requires-Dist: pydantic>=2
Requires-Dist: python-dotenv>=1.0
License-File: LICENSE
Summary: High-performance stateful stream processor with a Rust core and a Python SDK.
Keywords: kafka,streaming,stream-processing,stateful,arrow,rust,free-threading
Author-email: Guillaume Savary <g.savary@gmail.com>
License-Expression: Apache-2.0
Requires-Python: >=3.13
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://turbine.savary.name/
Project-URL: Homepage, https://github.com/elrik75/Turbine
Project-URL: Issues, https://github.com/elrik75/Turbine/issues
Project-URL: Repository, https://github.com/elrik75/Turbine

# Turbine

![Turbine](https://turbine.savary.name/turbine.png)

A high-performance stateful stream processor with a Rust core and a Python interface.

## Vision

Turbine aims to bring Kafka Streams-level stateful stream processing to the Python ecosystem, with performance that matches or exceeds JVM-based solutions. By writing the core engine in Rust and exposing a Python SDK, Turbine bridges the gap between raw throughput and the ergonomics that AI/ML practitioners expect.

## Goals

- **Rust core** — Low-latency, high-throughput stream processing engine with stateful operators (joins, aggregations, windowing).
- **Python SDK** — Idiomatic Python API so users can define topologies, transformations, and integrate with AI/ML tooling (LLMs, inference pipelines, etc.).
- **Queryable state** — Every node in the processing topology exposes its current state via an API. No more black-box pipelines.
- **Built-in observability** — Console UI and REST API to inspect the state of any node in real time, monitor throughput, lag, and topology health.
- **Kafka-compatible** — Consumes from and produces to Kafka and Kafka-compatible brokers (Redpanda).

## Positioning

Turbine competes with:

- **Kafka Streams** — Powerful but JVM-only. Turbine targets similar semantics with better performance and a Python-native experience.
- **Faust** — Python stream processing, but slow, unmaintained, and limited tooling. Turbine offers Rust-level performance with superior observability and stability.

## Why Python?

Python dominates the AI/ML landscape. Data scientists and ML engineers need to wire stream processing into inference pipelines, feature stores, and real-time AI applications. Turbine gives them a fast, stateful stream processor without leaving their ecosystem.

