Metadata-Version: 2.4
Name: unistream
Version: 0.1.2
Summary: Library for creating stream data producer / consumer client library.
Author-email: Sanhe Hu <husanhe@email.com>
Maintainer-email: Sanhe Hu <husanhe@email.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/MacHu-GWU/unistream-project
Project-URL: Documentation, https://unistream.readthedocs.io/en/latest/
Project-URL: Repository, https://github.com/MacHu-GWU/unistream-project
Project-URL: Issues, https://github.com/MacHu-GWU/unistream-project/issues
Project-URL: Changelog, https://github.com/MacHu-GWU/unistream-project/blob/main/release-history.rst
Project-URL: Download, https://pypi.org/pypi/unistream#files
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: MacOS
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: <4.0,>=3.10
Description-Content-Type: text/x-rst
License-File: LICENSE.txt
License-File: AUTHORS.rst
Requires-Dist: tenacity<10.0.0,>=8.2.0
Requires-Dist: func-args<2.0.0,>=1.0.2
Requires-Dist: vislog<1.0.0,>=0.1.3
Provides-Extra: dev
Requires-Dist: rich<14.0.0,>=13.8.1; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest<9.0.0,>=8.2.2; extra == "test"
Requires-Dist: pytest-cov<7.0.0,>=6.0.0; extra == "test"
Provides-Extra: doc
Requires-Dist: Sphinx<8.0.0,>=7.4.7; extra == "doc"
Requires-Dist: sphinx-copybutton<1.0.0,>=0.5.2; extra == "doc"
Requires-Dist: sphinx-design<1.0.0,>=0.6.1; extra == "doc"
Requires-Dist: sphinx-jinja<3.0.0,>=2.0.2; extra == "doc"
Requires-Dist: furo==2024.8.6; extra == "doc"
Requires-Dist: pygments<3.0.0,>=2.18.0; extra == "doc"
Requires-Dist: ipython<8.19.0,>=8.18.1; extra == "doc"
Requires-Dist: nbsphinx<1.0.0,>=0.8.12; extra == "doc"
Requires-Dist: rstobj==2.0.0; extra == "doc"
Requires-Dist: docfly==3.0.3; extra == "doc"
Provides-Extra: mise
Requires-Dist: PyGithub<3.0.0,>=2.8.0; extra == "mise"
Requires-Dist: httpx<1.0.0,>=0.28.0; extra == "mise"
Requires-Dist: tomli<3.0.0,>=2.0.0; python_version < "3.11" and extra == "mise"
Dynamic: license-file


.. image:: https://readthedocs.org/projects/unistream/badge/?version=latest
    :target: https://unistream.readthedocs.io/en/latest/
    :alt: Documentation Status

.. image:: https://github.com/MacHu-GWU/unistream-project/actions/workflows/main.yml/badge.svg
    :target: https://github.com/MacHu-GWU/unistream-project/actions?query=workflow:CI

.. image:: https://codecov.io/gh/MacHu-GWU/unistream-project/branch/main/graph/badge.svg
    :target: https://codecov.io/gh/MacHu-GWU/unistream-project

.. image:: https://img.shields.io/pypi/v/unistream.svg
    :target: https://pypi.python.org/pypi/unistream

.. image:: https://img.shields.io/pypi/l/unistream.svg
    :target: https://pypi.python.org/pypi/unistream

.. image:: https://img.shields.io/pypi/pyversions/unistream.svg
    :target: https://pypi.python.org/pypi/unistream

.. image:: https://img.shields.io/badge/✍️_Release_History!--None.svg?style=social&logo=github
    :target: https://github.com/MacHu-GWU/unistream-project/blob/main/release-history.rst

.. image:: https://img.shields.io/badge/⭐_Star_me_on_GitHub!--None.svg?style=social&logo=github
    :target: https://github.com/MacHu-GWU/unistream-project

------

.. image:: https://img.shields.io/badge/Link-API-blue.svg
    :target: https://unistream.readthedocs.io/en/latest/py-modindex.html

.. image:: https://img.shields.io/badge/Link-Install-blue.svg
    :target: `install`_

.. image:: https://img.shields.io/badge/Link-GitHub-blue.svg
    :target: https://github.com/MacHu-GWU/unistream-project

.. image:: https://img.shields.io/badge/Link-Submit_Issue-blue.svg
    :target: https://github.com/MacHu-GWU/unistream-project/issues

.. image:: https://img.shields.io/badge/Link-Request_Feature-blue.svg
    :target: https://github.com/MacHu-GWU/unistream-project/issues

.. image:: https://img.shields.io/badge/Link-Download-blue.svg
    :target: https://pypi.org/pypi/unistream#files


Welcome to ``unistream`` Documentation
==============================================================================
.. image:: https://unistream.readthedocs.io/en/latest/_static/unistream-logo.png
    :target: https://unistream.readthedocs.io/en/latest/

``unistream`` is a universal Producer / Consumer abstraction layer for stream systems. It lets you use a single, unified interface to send data to and pull data from any streaming backend — Apache Kafka, AWS Kinesis, AWS CloudWatch Logs, Apache Pulsar, or even local files — while the library handles batching, fault tolerance, retries, and exactly-once consumption behind the scenes.

The core library ships only local-file implementations. Vendor-specific integrations (AWS Kinesis, CloudWatch Logs, DynamoDB checkpoint, etc.) are released as **separate plugin packages**.


Features
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1. **Efficient Record Buffering:** Groups records into micro-batches with a Write-Ahead Log (WAL) for crash recovery, optimizing network bandwidth without risking data loss.
2. **Non-blocking Exponential Backoff:** Automatic retry with configurable backoff schedules. The retry check is time-based (no ``sleep``), so your application thread is never blocked.
3. **Checkpoint & Exactly-Once Consumption:** Per-record status tracking (pending → in_progress → succeeded / failed / exhausted), UUID-based concurrency locking with auto-expiry, and stream pointer persistence for fault-tolerant consumption.
4. **Pluggable Architecture:** Five core abstractions (Record, Buffer, Producer, CheckPoint, Consumer) with clean ABC → Base → Concrete layering. Implement ``send()`` for a new producer or ``get_records()`` for a new consumer — the framework handles everything else.


Core Abstractions
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- ``AbcRecord`` / ``DataClassRecord`` — the atomic data unit (id + create_at + serialize/deserialize).
- ``AbcBuffer`` / ``FileBuffer`` — WAL-backed batch buffer (put → should_i_emit → emit → commit).
- ``AbcProducer`` / ``BaseProducer`` / ``SimpleProducer`` — non-blocking put() event loop with retry.
- ``AbcCheckPoint`` / ``BaseCheckPoint`` / ``SimpleCheckpoint`` — per-record state machine + persistence.
- ``AbcConsumer`` / ``BaseConsumer`` / ``SimpleConsumer`` — consumption loop with tenacity retry + DLQ hook.


AI Agent Skill
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A self-contained `Claude Code skill <https://claude.com/claude-code>`_ is included at ``.claude/skills/unistream/SKILL.md``. It contains the complete API reference, protocols, and usage examples — any AI coding agent with this skill loaded can build custom producers, consumers, buffers, checkpoints, and records without reading the source code.


.. _install:

Install
------------------------------------------------------------------------------

``unistream`` is released on PyPI, so all you need is to:

.. code-block:: console

    $ pip install unistream

To upgrade to latest version:

.. code-block:: console

    $ pip install --upgrade unistream
