Metadata-Version: 2.4
Name: swarmstate
Version: 0.11.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Free Threading :: 2 - Beta
Classifier: Programming Language :: Rust
Classifier: Topic :: Software Development :: Libraries
Classifier: Typing :: Typed
Requires-Dist: swarmstate[langgraph,crewai,redis,disk,postgres,otel] ; extra == 'all'
Requires-Dist: swarmstate[langgraph,disk] ; extra == 'bench'
Requires-Dist: langgraph-checkpoint-sqlite>=2.0 ; extra == 'bench'
Requires-Dist: matplotlib>=3.8 ; extra == 'bench'
Requires-Dist: crewai>=0.70 ; extra == 'crewai'
Requires-Dist: pytest>=8.0 ; extra == 'dev'
Requires-Dist: pytest-cov>=5.0 ; extra == 'dev'
Requires-Dist: ruff>=0.16,<0.17 ; extra == 'dev'
Requires-Dist: mypy>=2.1,<3 ; extra == 'dev'
Requires-Dist: msgpack>=1.0 ; extra == 'disk'
Requires-Dist: langgraph>=0.2 ; extra == 'langgraph'
Requires-Dist: opentelemetry-api>=1.20 ; extra == 'otel'
Requires-Dist: psycopg[binary]>=3.1 ; extra == 'postgres'
Requires-Dist: psycopg-pool>=3.2 ; extra == 'postgres'
Requires-Dist: msgpack>=1.0 ; extra == 'postgres'
Requires-Dist: redis>=5.0 ; extra == 'redis'
Requires-Dist: msgpack>=1.0 ; extra == 'redis'
Provides-Extra: all
Provides-Extra: bench
Provides-Extra: crewai
Provides-Extra: dev
Provides-Extra: disk
Provides-Extra: langgraph
Provides-Extra: otel
Provides-Extra: postgres
Provides-Extra: redis
License-File: LICENSE
Summary: Drop-in state backend for LangGraph, CrewAI & custom agent loops - Rust core, framework-agnostic, built for production.
Keywords: agents,langgraph,crewai,checkpoint,state,rust,multi-agent
Author: Jose L. Salmeron
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/swarmstate/swarmstate/blob/main/CHANGELOG.md
Project-URL: Documentation, https://swarmstate.github.io/
Project-URL: Homepage, https://github.com/swarmstate/swarmstate
Project-URL: Issues, https://github.com/swarmstate/swarmstate/issues
Project-URL: Repository, https://github.com/swarmstate/swarmstate

# swarmstate

> Drop-in state backend for LangGraph, CrewAI & custom agent loops - Rust core, framework-agnostic, built for production.

> **Constant-time checkpoint reads** — `get_tuple` stays at ~7 µs whether a thread holds 5 or
> 2 000 checkpoints, where LangGraph's `InMemorySaver` climbs from ~5 µs to ~40 µs — and **O(1)**
> state snapshots: ~0.5 µs at any size, against 50 ms to `deepcopy` a 50 000-entry state.
> Durable writes land on par with `SqliteSaver` at the same fsync policy (~1.9× faster than
> its shipped default, which buys stronger durability).
> Method, hardware and raw numbers → **[`benchmarks/`](benchmarks/)**.

`swarmstate` is a **state and checkpointing backend** with a Rust core and a Python API for multi-agent
systems. It does not compete with visible agent frameworks; it acts as low-level infrastructure - much
like engines such as DuckDB, ClickHouse, Arrow, or Polars sit underneath data applications without
replacing them.

It solves three production pains:

1. **State lock-in across frameworks** - a framework-agnostic store so migrating frameworks doesn't lose state.
2. **Checkpoint reads that get slower as threads grow** - a Rust-backed implementation of LangGraph's
   checkpointer interface that resolves "the latest checkpoint" by lookup instead of scanning a thread's keys.
3. **Deterministic routing paid for in tokens** - a native handoff graph that resolves rule-based transitions in microseconds.

## Installation

```bash
pip install swarmstate            # prebuilt abi3 wheels, no compiler required
uv add swarmstate                 # or with uv
```

Optional extras: `swarmstate[langgraph]`, `swarmstate[crewai]`, `swarmstate[redis]`,
`swarmstate[disk]`, `swarmstate[postgres]`, `swarmstate[otel]`, `swarmstate[all]`.

## Usage

```python
import swarmstate as ss

store = ss.Store()                              # in-memory, msgpack codec
store.set("workflow", "onboarding", {"step": 3, "data": {...}})
snap = store.snapshot()                          # cheap, immutable snapshot
store.set("workflow", "onboarding", {"step": 4})
store.restore(snap)                              # rollback
store.get("workflow", "onboarding")              # -> {"step": 3, "data": {...}}

snap2 = store.snapshot()
snap2.diff(snap)                                 # {"added": [...], "removed": [...], "changed": [...]}

# Retention is opt-in: a snapshot the store keeps pins the state it saw
hist = ss.Store(max_history=10)                  # 0 (default) keeps none, None keeps all
hist.history()                                   # -> [Snapshot, ...], oldest first

# Batch ops: one GIL release / round-trip for the whole set
store.set_many([("workflow", "a", {...}), ("workflow", "b", {...})])
store.get_many([("workflow", "a"), ("workflow", "b")])   # -> [..., ...], order preserved

# Deterministic, LLM-free routing (resolved natively in Rust)
g = ss.HandoffGraph()
g.add_edge("triage", "billing", when="category == 'billing'")
g.add_edge("triage", "human")                    # unconditional default
g.route("triage", {"category": "billing"})       # -> "billing"
```

Drop-in LangGraph checkpointer (`pip install "swarmstate[langgraph]"`):

```python
from swarmstate.integrations.langgraph import SwarmStateSaver

graph = builder.compile(checkpointer=SwarmStateSaver())   # replaces SqliteSaver, 1 line
```

Bounded memory for long-running threads — checkpointers keep every step by
default, which for a service that never restarts means growth without end:

```python
saver = SwarmStateSaver(max_checkpoints_per_thread=8)     # keep the newest N per thread
```

Older checkpoints are dropped with their pending writes and channel blobs. On a
300-invocation thread that is **0.5 MB instead of 28 MB**, and the thread still
resumes; time travel is limited to the retained window, so size it to taste.

Optional metrics on checkpoint operations (opt-in, zero overhead when unused):

```python
from swarmstate.observability import InMemoryMetrics       # or OpenTelemetryMetrics

metrics = InMemoryMetrics()
saver = SwarmStateSaver(metrics=metrics)
# ... run the graph ...
metrics.summary()   # {"put": {"count": 12, "p50_ms": 0.006, ...}, "get_tuple": {...}}
```

OpenTelemetry tracing (each checkpoint op becomes a `swarmstate.checkpoint.<op>` span):

```python
from swarmstate.observability import get_tracer     # needs swarmstate[otel]

saver = SwarmStateSaver(tracer=get_tracer())         # composes with metrics=...
```

## Status

Early development.

- **M0 (scaffolding)** ✅ - Rust core builds; `import swarmstate` works.
- **M1 (Rust store)** ✅ - concurrent KV store, msgpack codec, O(1) immutable snapshots,
  incremental diffs, GIL released on hot paths.
- **M2 (HandoffGraph)** ✅ - deterministic conditional routing with a safe Rust condition
  evaluator (no `eval`), cycle detection.
- **M3 (LangGraph adapter)** ✅ - `SwarmStateSaver`, a drop-in `BaseCheckpointSaver`
  backed by the `Store`; snapshot/roll back the whole checkpoint DB at once.
- **M4 (Benchmarks)** ✅ - durable-vs-durable and in-memory-vs-in-memory comparisons on
  LangGraph's interface, read latency as a thread grows, `Store.snapshot()` vs `deepcopy`,
  and concurrency scaling. Reproducible: [`benchmarks/run.py`](benchmarks/run.py); method
  and results in [`benchmarks/README.md`](benchmarks/README.md).
- **M5 (CrewAI adapter + backends)** ✅ - persistent, drop-in checkpointer backends
  `RedisStore`, `DiskStore` (SQLite) and `PostgresStore`, all msgpack wire-format, plus
  `SwarmStateStorage` (portable memory backed by a shared `Store`).
- **M6 (docs · wheels · PyPI)** ✅ - full docs site, benchmarks, cross-platform abi3
  wheels, and PyPI publishing via Trusted Publishing (OIDC).
- **Observability** ✅ - opt-in metrics hooks and OpenTelemetry **tracing** on checkpoint
  ops (`put` / `put_writes` / `get_tuple`): an in-memory sink, an OpenTelemetry metrics
  sink, and per-op spans (`swarmstate[otel]`). Zero overhead when unused. Strict `mypy` in CI.
- **Free-threaded (no-GIL) ready** ✅ - the Rust core declares free-threaded support, so on
  a free-threaded CPython build (`cp313t`) the store **doesn't collapse under threads the way
  the GIL build does**: on a set+get workload at 8 threads it sustains **~1.8M ops/s vs ~130k
  on GIL Python (over 10x)**, where the GIL build gets *much slower* as threads are added.
  (These workloads are allocation-bound, so neither scales linearly with cores; the win is
  avoiding the GIL's collapse.) Version-specific `cp313t` and `cp314t` wheels ship for Linux
  (x86_64/aarch64), macOS (arm64) and Windows (x64) alongside the abi3 ones.
- **Batch API** ✅ - `Store.set_many` / `get_many` (and on every backend) amortize the
  per-call overhead over a batch: one GIL release for the in-memory core, one round-trip for
  networked backends. On free-threaded at 8 threads, `set_many` is ~3x the throughput of
  individual sets. `SwarmStateSaver` uses it internally: `put_writes` (and the `incremental`
  channel blobs) flush all writes of a step in a single `set_many`, so fan-out steps that emit
  many pending writes pay one lock/round-trip instead of one per write.

## Examples

Runnable, offline, deterministic demos in [`examples/`](examples/):

- [`support_triage.py`](examples/support_triage.py) - a LangGraph workflow tying together
  `HandoffGraph` routing, `SwarmStateSaver` checkpointing and snapshot/restore time-travel.
- [`state_portability.py`](examples/state_portability.py) - state as standard msgpack
  bytes, read back and cross-checked against the `msgpack` package.

## Documentation

Guide, tutorials and API reference: **[swarmstate.github.io](https://swarmstate.github.io/)**
— [the store](https://swarmstate.github.io/guide/store/),
[snapshots & diffs](https://swarmstate.github.io/guide/snapshots/),
[the LangGraph checkpointer](https://swarmstate.github.io/guide/langgraph/),
[persistent backends](https://swarmstate.github.io/guide/disk/),
[the handoff graph](https://swarmstate.github.io/guide/handoff/) and the
[benchmark method](https://swarmstate.github.io/benchmarks/). The site is built from
[`swarmstate/swarmstate.github.io`](https://github.com/swarmstate/swarmstate.github.io).

## Development

```bash
python -m venv .venv && source .venv/bin/activate
pip install maturin pytest
maturin develop --release     # compile the Rust core and install it locally
cargo test                    # Rust core tests
pytest -q                     # Python API tests

```

## Citing

If you use `swarmstate` in academic work, please cite it. GitHub's **"Cite this
repository"** button (from [`CITATION.cff`](CITATION.cff)) produces ready-made APA and
BibTeX entries. To cite the archived software release, use its Zenodo DOI
(`10.5281/zenodo.XXXXXXXX`):

```bibtex
@software{salmeron_swarmstate,
  author    = {Salmeron, Jose L.},
  title     = {{swarmstate}: A state and checkpointing backend for multi-agent
               systems with a Rust core},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.XXXXXXXX},
  url       = {https://github.com/swarmstate/swarmstate}
}
```

<sub>The DOI is minted when the first release is archived on Zenodo. Replacing
`10.5281/zenodo.XXXXXXXX` here, in [`CITATION.cff`](CITATION.cff) and on the docs site is
all it takes — the placeholder is deliberate, so that
nothing cites an identifier that does not resolve.</sub>

## License

MIT

