Metadata-Version: 2.4
Name: storey
Version: 1.11.24
Summary: Async flows
Home-page: https://github.com/mlrun/storey
Author: Iguazio
Author-email: yaronh@iguazio.com
License: Apache
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: MacOS
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp~=3.8
Requires-Dist: v3io<0.8,>=0.6.10
Requires-Dist: pandas!=1.5.*,<2.2,>=1
Requires-Dist: numpy<1.27,>=1.16.5
Requires-Dist: pyarrow<18,>=1
Requires-Dist: v3io-frames!=0.11.*,!=0.12.*,>=0.10.14
Requires-Dist: fsspec>=0.6.2
Requires-Dist: v3iofs~=0.1.17
Requires-Dist: xxhash>=1
Requires-Dist: nuclio-sdk>=0.5.3
Provides-Extra: kafka
Requires-Dist: kafka-python~=2.0; extra == "kafka"
Provides-Extra: redis
Requires-Dist: redis~=4.3; extra == "redis"
Provides-Extra: sqlalchemy
Requires-Dist: sqlalchemy~=1.3; extra == "sqlalchemy"
Provides-Extra: psycopg
Requires-Dist: psycopg[binary,pool]~=3.2; extra == "psycopg"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Storey

Storey is an asynchronous streaming library for real-time event processing and feature extraction. It is part of the [MLRun](https://github.com/mlrun/mlrun) ecosystem.

## Features

- Async event processing with backpressure
- Rich set of built-in transformations: Map, Filter, FlatMap, Batch, Choice, JoinWithTable, and more
- Time-window aggregations with pluggable storage (V3IO, Redis, SQL)
- Sources and targets for files (CSV, Parquet), streams (Kafka, V3IO), and databases (Redis, TimescaleDB)
- Streaming (generator) support in Map steps

## Installation

```bash
pip install storey
```

Optional extras:

```bash
pip install storey[kafka]       # Kafka support
pip install storey[redis]       # Redis support
pip install storey[psycopg]     # TimescaleDB support
```

## Quick Example

```python
from storey import build_flow, SyncEmitSource, Map, Filter, ParquetTarget

controller = build_flow([
    SyncEmitSource(),
    Filter(lambda event: event["amount"] > 0),
    Map(lambda event: {**event, "amount_cents": int(event["amount"] * 100)}),
    ParquetTarget("output.parquet", columns=["user", "amount", "amount_cents"]),
]).run()

controller.emit({"user": "alice", "amount": 9.99})
controller.emit({"user": "bob", "amount": -1.00})
controller.emit({"user": "carol", "amount": 24.50})

controller.terminate()
controller.await_termination()
# output.parquet now contains the two events with positive amounts,
# each enriched with amount_cents.
```

## Documentation

See the [MLRun documentation](https://docs.mlrun.org/en/stable/) and the [storey transformations API reference](https://docs.mlrun.org/en/latest/api/storey.transormations/index.html).

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, testing, and coding conventions.

## License

Apache License 2.0
