Metadata-Version: 2.5
Name: kafka-agent-bridge
Version: 0.1.0
Summary: Wire LangGraph and CrewAI agents into Kafka-backed event-driven systems.
Project-URL: Homepage, https://github.com/prajwalgowdahg/kafka-agent-bridge
Project-URL: Issues, https://github.com/prajwalgowdahg/kafka-agent-bridge/issues
Author: kafka-agent-bridge contributors
License-Expression: MIT
Keywords: agents,crewai,event-driven,kafka,langgraph
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Requires-Dist: aiokafka>=0.10
Requires-Dist: cachetools>=5.3
Requires-Dist: pydantic>=2.0
Requires-Dist: structlog>=24.0
Provides-Extra: all
Requires-Dist: crewai>=0.55; extra == 'all'
Requires-Dist: fastavro>=1.9; extra == 'all'
Requires-Dist: langchain-core>=0.2; extra == 'all'
Requires-Dist: langgraph>=0.2; extra == 'all'
Requires-Dist: redis>=5.0; extra == 'all'
Provides-Extra: avro
Requires-Dist: fastavro>=1.9; extra == 'avro'
Provides-Extra: crewai
Requires-Dist: crewai>=0.55; extra == 'crewai'
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: langgraph
Requires-Dist: langchain-core>=0.2; extra == 'langgraph'
Requires-Dist: langgraph>=0.2; extra == 'langgraph'
Provides-Extra: redis
Requires-Dist: redis>=5.0; extra == 'redis'
Description-Content-Type: text/markdown

# kafka-agent-bridge

`kafka-agent-bridge` is a small Python library for developers who already know LangGraph or CrewAI and want Kafka to be the event bus without hand-rolling consumers, retries, idempotency, result publishing, and dead-letter handling. It fills the space between heavyweight streaming-agent platforms and raw framework glue code.

## Quickstart

```bash
pip install -e '.[langgraph,redis,dev]'
docker compose up
```

```python
from kafka_agent_bridge import KafkaAgentBridge
from kafka_agent_bridge.adapters import LangGraphAdapter

adapter = LangGraphAdapter(graph=compiled_graph)

bridge = KafkaAgentBridge(
    bootstrap_servers="localhost:9092",
    consumer_group="reports-agent",
    idempotency_backend="redis://localhost:6379",
)
bridge.register(
    topic="reports.requested",
    handler=adapter,
    adapter="langgraph",
    output_topic="reports.completed",
    dlq_topic="reports.dlq",
)
await bridge.start()
```

Run the included LangGraph example:

```bash
make dev
make example-fraud
```

Kafka UI is available at `http://localhost:8080`.

## Architecture

```text
Kafka topic
    |
    v
KafkaAgentBridge
    |-- JsonDeserializer -> EventEnvelope
    |-- AgentRouter -> topic glob, X-Agent-Type, predicate
    |-- IdempotencyGuard -> Redis or in-memory TTL cache
    |-- LangGraphAdapter / CrewAIAdapter / custom handler
    |-- KafkaResultProducer -> output topic
    |
    +-- DeadLetterQueue -> *.dlq after retries

LangGraph graphs can use KafkaCheckpointer:
thread_id -> compacted Kafka topic agent.checkpoints
```

## What Is Included

- `KafkaAgentBridge` async runtime with multiple topic registrations.
- `AgentRouter` with glob, header, and predicate routing.
- `KafkaResultProducer` with JSON serialization and configurable batching.
- `EventEnvelope` Pydantic model for consumed messages.
- Redis-backed or in-memory idempotency guard.
- Retry and dead-letter publishing.
- `LangGraphAdapter`, `CrewAIAdapter`, and `KafkaCheckpointer`.
- Docker Compose with Kafka, Redis, and Kafka UI.

Avro and Schema Registry support are intentionally stubbed for v0.2.

## Comparison

| Approach | Best For | Tradeoff | How It Is Handled |
| --- | --- | --- | --- |
| kafka-agent-bridge | Python teams wiring LangGraph or CrewAI into Kafka quickly | Lightweight v0.1 surface; advanced Kafka tuning remains explicit | Sensible defaults cover the common path, while `consumer_kwargs`, `producer_kwargs`, retry settings, DLQ topics, and idempotency backends stay configurable when a production workload needs them. |
| Confluent Streaming Agents | Teams already on Confluent's managed ecosystem | More platform coupling and operational surface area | Use it when managed Confluent services are already the standard; use this library when teams want a smaller OSS Python layer over plain Kafka. |
| Hand-rolling | Highly custom Kafka and agent orchestration needs | Rebuilds routing, retries, DLQ, idempotency, and checkpointing yourself | Start with this library for the repeated plumbing, then drop down to custom handlers, predicates, adapters, and raw Kafka kwargs where the workflow genuinely needs custom behavior. |

The intent is not to hide Kafka forever. The bridge makes the default path small, then leaves explicit escape hatches for teams that need partitioning, offset, batching, security, or consumer-group tuning.

## Local Development

```bash
python -m venv venv
venv/bin/pip install -e '.[all,dev]'
make test
make lint
```

## Contributing

Issues and pull requests are welcome. For now, keep contributions focused on small, well-tested improvements: adapter behavior, Kafka operational safety, examples, and documentation. Please include tests for behavior changes and keep public APIs typed and documented.
