Metadata-Version: 2.4
Name: popoto
Version: 1.9.0
Summary: Agent memory for LLM apps on Redis or Valkey: decay, confidence, associations, and context assembly, built on a full Python ORM
Home-page: https://github.com/tomcounsell/popoto
Author: Tom Counsell
Author-email: Tom Counsell <other@tomcounsell.com>
License: MIT License
        
        Copyright (c) 2022 Tom Counsell
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://popoto.io/
Project-URL: Documentation, https://popoto.io/
Project-URL: Source, https://github.com/tomcounsell/popoto
Project-URL: Bug_Tracker, https://github.com/tomcounsell/popoto/issues
Keywords: agent-memory,agent memory,ai-agents,llm,llm-memory,long-term-memory,memory,context,context-engineering,retrieval,rag,bm25,embeddings,vector-search,semantic-search,redis,valkey,ORM,database
Classifier: Development Status :: 5 - Production/Stable
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Database
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: redis>=4.4.4
Requires-Dist: msgpack>=1.0.4
Provides-Extra: dataframe
Requires-Dist: pandas>=1.4.3; extra == "dataframe"
Requires-Dist: numpy>=1.23.1; extra == "dataframe"
Requires-Dist: msgpack-numpy>=0.4.8; extra == "dataframe"
Provides-Extra: ulid
Requires-Dist: ulid-py>=1.1.0; extra == "ulid"
Provides-Extra: ksuid
Requires-Dist: cyksuid>=1.0.0; extra == "ksuid"
Provides-Extra: embeddings
Requires-Dist: numpy>=1.23.1; extra == "embeddings"
Provides-Extra: voyage
Requires-Dist: voyageai>=0.3.0; extra == "voyage"
Requires-Dist: numpy>=1.23.1; extra == "voyage"
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == "openai"
Requires-Dist: numpy>=1.23.1; extra == "openai"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.40.0; extra == "anthropic"
Provides-Extra: monitoring
Requires-Dist: sentry-sdk>=2.0.0; extra == "monitoring"
Provides-Extra: mcp
Requires-Dist: mcp>=1.2; extra == "mcp"
Provides-Extra: benchmark
Requires-Dist: huggingface_hub>=0.23; extra == "benchmark"
Requires-Dist: sentence-transformers>=2.7; extra == "benchmark"
Provides-Extra: dev
Requires-Dist: pytest>=7.1.2; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: mypy>=0.971; extra == "dev"
Requires-Dist: black>=26.3.1; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: tiktoken>=0.5; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.6; extra == "docs"
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.25; extra == "docs"
Requires-Dist: griffe>=0.45; extra == "docs"
Requires-Dist: mkdocs-gen-files>=0.5; extra == "docs"
Requires-Dist: mkdocs-literate-nav>=0.6; extra == "docs"
Requires-Dist: mkdocs-open-in-new-tab>=1.0; extra == "docs"
Dynamic: license-file

### Status
[![pypi package](https://badge.fury.io/py/popoto.svg)](https://pypi.org/project/popoto)
[![total downloads](https://pepy.tech/badge/popoto)](https://pepy.tech/project/popoto)
[![deploy docs](https://github.com/tomcounsell/popoto/actions/workflows/deploy-docs.yml/badge.svg)](https://github.com/tomcounsell/popoto/actions/workflows/deploy-docs.yml)

### Documentation: [**popoto.io**](https://popoto.io/)


# Popoto: Agent Memory on Redis and Valkey

Memory for LLM agents, as primitives you program rather than a service you call. Records decay over time, confidence moves with evidence, associations form between things mentioned together, and a context assembler packs the result into a token budget before each turn.

It runs in your process against a Redis or Valkey server you already operate. Your memory data stays in your database, and the whole install is three packages with no API key.

Underneath, Popoto is a full Redis/Valkey ORM with Django-like model syntax. The memory system is built on it, and the [ORM half is documented below](#redis--valkey-orm).

## Install

```
pip install popoto
```

That pulls `popoto`, `redis`, and `msgpack`: 3 packages, 9.0 MB of site-packages measured 2026-09-04 in a clean Python 3.12 venv resolving redis-py 8.1.0. Point it at Redis or Valkey on `localhost:6379` and you are running.

## Memory around an LLM turn

`SubconsciousMemory` wraps a chat loop. It retrieves before the model call and stores after it, while your application keeps its own message list.

```python
from popoto import (
    Model, AutoKeyField, KeyField, StringField, FloatField,
    DecayingSortedField, ConfidenceField, BM25Field,
)
from popoto.recipes.subconscious_memory import SubconsciousMemory

class Memory(Model):
    memory_id = AutoKeyField()
    agent_id = KeyField()
    content = StringField(default="")
    importance = FloatField(default=1.0)
    relevance = DecayingSortedField(
        base_score_field="importance",
        partition_by="agent_id",
    )
    confidence = ConfidenceField(initial_confidence=0.5)
    content_bm25 = BM25Field(source="content")  # makes retrieval query-sensitive

memory = SubconsciousMemory(
    model_class=Memory,
    agent_id="agent-1",
    score_weights={"relevance": 0.6, "confidence": 0.3},
)

Memory(
    agent_id="agent-1",
    content="Deploys use a blue-green strategy behind the load balancer.",
    importance=2.0,
).save()

messages = [{"role": "user", "content": "What is our deployment strategy?"}]

# Pre-turn: relevant memories are retrieved and appended at the end of `messages`
messages, assembly = memory.inject_context(messages)

# ... call your LLM with `messages`, get back `response_text` ...

# Post-turn: facts in the response become new memories
memory.extract_memories(response_text, importance=0.6)

# Outcome: report which injected memories the agent actually used
memory.report_outcomes(assembly)
```

`BM25Field` is what makes retrieval respond to the query text. Leave it off and `SubconsciousMemory` ranks by importance and confidence alone, which is query-blind by design and right for some workloads. The [SubconsciousMemory recipe](https://popoto.io/guides/subconscious-memory-recipe/) covers when each mode applies.

Next steps: the [Agent Memory Quickstart](https://popoto.io/guides/agent-memory-quickstart/) builds the primitives up level by level, and the [Agent Memory overview](https://popoto.io/features/agent-memory/) is the full reference.

Running inside Claude Code, Codex, Hermes, or OpenClaw? [Harness Integration](https://popoto.io/features/harness-integration/) wires the same loop into hooks and MCP with no glue code: `pip install 'popoto[mcp]'`, paste a config block, and memory injects before every turn and captures after it.

## What is measured

Every number below comes from a harness in this repository, with its result JSON committed alongside. Method, per-category tables, and the runs that came out badly are in [Benchmarks](https://popoto.io/benchmarks/).

**Retrieval quality.** On LongMemEval-S (all 500 questions, no sampling, hybrid BM25 + vector with weighted RRF): Recall@1 0.892, Recall@5 0.986, MRR 0.931. Read the granularity before comparing this to anything: Popoto indexes one record per conversation turn, and a retrieved turn counts as a hit for its parent session, so these are session-level recall figures produced by turn-level ranking. Systems that rank whole sessions are answering a differently shaped question and the numbers are not interchangeable.

**Retrieval latency.** p50 3.0 ms at 1,000 records rising to 6.0 ms at 20,000 (p99 5.8 ms and 15.3 ms). In-process on the lexical path, one Apple-silicon machine, one representative run. Absolute milliseconds are machine-dependent; the shape of the curve is the durable part.

**LLM extraction was measured, and left off.** We ran Claude-based fact extraction against plain turn ingestion on the judged-answer harness, across three models plus a heuristic sentence splitter. Raw turn ingestion beat every extraction arm: judged accuracy 0.3636 for raw against 0.2078 for the heuristic splitter and 0.1948 / 0.1429 / 0.0519 for Sonnet, Opus, and Haiku. The mechanism is measured rather than guessed: the extraction prompt's instruction to skip filler discards a large share of the turns that hold the ground-truth evidence, so the answer is gone before retrieval ever runs. Extraction therefore ships as a documented opt-in that stays off by default. Every arm ran on the same 77 scored items from a 2-dialogue LoCoMo subset, so read the ordering, not the third decimal. Method and code: [LLM Memory Extraction](https://popoto.io/features/llm-memory-extraction/).

**Judged end-to-end accuracy trails retrieval quality by a wide margin.** On LoCoMo, judged answer accuracy is 0.3636 (28 of 77 scored items, 95% CI ≈ 0.25 to 0.47, `gpt-4o-mini` as both generator and judge under the pinned Mem0/GAM protocol), against retrieval that finds the right evidence far more often. Those are different metric families and are never combined. Finding the right evidence is far more reliable than answering from it, and closing that gap is the open work. Scope and interval: [Benchmarks](https://popoto.io/benchmarks/).

**Valkey is a first-class target.** Popoto uses core Redis data types and commands only, with no Redis-module dependency, and the suite carries explicit Valkey-safety tests asserting that indexes stay on plain types. Since August 2026 the test suite — everything except the `slow`-marked tests, chiefly the stress suite — also runs against a real Valkey server on every pull request and every push to `main`, as a separately named `pytest (Valkey)` check in [`tests.yml`](https://github.com/tomcounsell/popoto/blob/main/.github/workflows/tests.yml), pinned to `valkey/valkey:8-alpine` alongside the Redis job's `redis:7-alpine`. The job asserts via `INFO server` that the container really is Valkey before pytest starts, and there has been no Valkey-only failure across 60 completed runs. The same code runs against either server.

## Redis / Valkey ORM

Popoto started as an ORM and still is one. Every memory primitive above is a field on an ordinary model, so the same query syntax, indexes, TTLs, and pub/sub apply.

```python
from popoto import Model, KeyField, Field, SortedField

class Restaurant(Model):
    name = KeyField()
    cuisine = Field()
    rating = SortedField(type=float)

Restaurant.create(name="Burger Palace", cuisine="American", rating=4.5)

restaurant = Restaurant.query.get(name="Burger Palace")

print(f"{restaurant.name} serves {restaurant.cuisine} food.")
# => "Burger Palace serves American food."
```

### Features

 - very fast stores and queries
 - familiar syntax, similar to Django models
 - Async operations for asyncio-based applications
 - Geometric distance search
 - Timeseries for streaming data
 - compatible with Pandas, Xarray for N-dimensional matrix search
 - PubSub for message queues, streaming data processing
 - **Full Redis and Valkey support** - works with both out of the box
 - **[Agent Memory](https://popoto.io/features/agent-memory/)** - programmable memory primitives for AI agents (decay, confidence, associations, context assembly)
 - **[Content & Embeddings](https://popoto.io/features/content-and-embedding-fields/)** - large content storage, vector embeddings, and semantic search
 - **[Harness Integration](https://popoto.io/features/harness-integration/)** - subconscious memory for Claude Code, Codex, Hermes, and OpenClaw via hooks and MCP
 - **[Export & Import](https://popoto.io/guides/export-import/)** - move records between Redis instances with per-field round-trip fidelity and conflict/write-gate/embedding-mismatch policies, from Python or the `popoto-transfer` command line

**Popoto** is ideal for streaming data. The pub/sub module allows you to trigger state updates in real time.
Currently being used in production for:

 - trigger buy/sell actions from streaming price data
 - robots sending each other messages for teamwork
 - compressing sensor data and training neural networks

### Relationships, TTLs, and Meta options

``` python
import popoto
from popoto import Relationship, DatetimeField

class Restaurant(popoto.Model):
    name = popoto.KeyField()
    cuisine = popoto.Field()
    rating = popoto.SortedField(type=float)
    location = popoto.GeoField()

class Order(popoto.Model):
    order_id = popoto.AutoKeyField()
    restaurant = Relationship(Restaurant)
    total = popoto.SortedField(type=float)
    status = popoto.Field(default="pending")
    created_at = DatetimeField(auto_now_add=True)

    class Meta:
        order_by = "-created_at"
        ttl = 2592000  # 30 days
```

### Save instances

``` python
restaurant = Restaurant(name="Burger Palace")
restaurant.cuisine = "American"
restaurant.rating = 4.5
restaurant.location = (40.7128, -74.0060)
restaurant.save()

order = Order.create(restaurant=restaurant, total=24.99)
```

### Queries

``` python
from datetime import datetime, timedelta

midtown = (40.7549, -73.9840)
yesterday = datetime.now() - timedelta(days=1)

nearby_restaurants = Restaurant.query.filter(
    location=midtown,
    location_radius=5, location_radius_unit='km',
    rating__gte=4.0
)

print(len(nearby_restaurants))
# => 1

recent_orders = Order.query.filter(
    created_at__gte=yesterday,
    total__gte=10.00
)
```

Full ORM reference: [Models and Fields](https://popoto.io/fields/), [Making Queries](https://popoto.io/query/), [Async Operations](https://popoto.io/async/), [TTL](https://popoto.io/ttl/), [PubSub](https://popoto.io/pubsub/), [Multi-Tenancy](https://popoto.io/multi-tenancy/).

## Running locally

Popoto is a library, not a standalone service. It runs inside your application and talks to a Redis or Valkey server. To exercise it locally, and to run the test suite, you need a Redis/Valkey server listening on `localhost:6379`.

```bash
# 1. Start Redis (or Valkey), e.g. on macOS via Homebrew:
redis-server                     # or: brew services start redis

# 2. Install Popoto with dev dependencies:
uv venv && source .venv/bin/activate && uv pip install -e ".[dev]"

# 3. Run the test suite (isolated on Redis DB 15 by this repo's pytest config):
pytest
```

By default Popoto connects to `localhost:6379`; set `REDIS_URL` to point at a different server. The pytest plugin isolates tests on Redis DB 15, which this repo opts into with `popoto_test_db = "15"` in `pyproject.toml` (override with `POPOTO_TEST_DB=<n>`). In *your* project the plugin does nothing until you set one of those — see [Testing](https://popoto.io/testing/).


# Documentation

Documentation is available at [**popoto.io**](https://popoto.io/)

Please create new feature and documentation related issues at [github.com/tomcounsell/popoto/issues](https://github.com/tomcounsell/popoto/issues) or make a pull request with your improvements.


# License

Popoto is released under the MIT Open Source license.


# Popoto Community

Questions, bug reports, and feature requests are welcome on [GitHub Issues](https://github.com/tomcounsell/popoto/issues) and [GitHub Discussions](https://github.com/tomcounsell/popoto/discussions). Contributions via pull request are encouraged.

![Popoto](https://raw.githubusercontent.com/tomcounsell/popoto/main/static/popoto.png)

Popoto gets its name from the [Maui dolphin](https://en.wikipedia.org/wiki/M%C4%81ui_dolphin) subspecies, the world's smallest dolphin subspecies.
Because dolphins are fast moving, agile, and work together in social groups. In the same way, Popoto wraps Redis and Valkey to make it easy to manage streaming timeseries data and object persistence.
