Metadata-Version: 2.4
Name: curry-leaves-memory
Version: 1.0.0
Summary: cl_memory — a file-first Organized Knowledge Folder framework: markdown notes as the source of truth, with an optional SQLite/FTS5 (BM25) index and optional vectors in the same database. Point it at a directory and get a working knowledge base with stable ids, link integrity, history, health checks, and layered search.
Project-URL: Homepage, https://github.com/Curry-Leaves/curry-leaves-memory
Project-URL: Repository, https://github.com/Curry-Leaves/curry-leaves-memory
Project-URL: Issues, https://github.com/Curry-Leaves/curry-leaves-memory/issues
Project-URL: Changelog, https://github.com/Curry-Leaves/curry-leaves-memory/blob/main/CHANGELOG.md
Author: Ilayanambi Ponramu
License-Expression: MIT
License-File: LICENSE
Keywords: backlinks,bm25,fts5,full-text-search,knowledge-base,markdown,notes,okf,second-brain,vector-search,wiki
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Markup :: Markdown
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: pyyaml>=6.0
Provides-Extra: agent
Requires-Dist: curry-leaves>=2.0; extra == 'agent'
Provides-Extra: api
Requires-Dist: fastapi>=0.110; extra == 'api'
Requires-Dist: uvicorn>=0.27; extra == 'api'
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest>=8.2; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Provides-Extra: vector
Requires-Dist: sqlite-vec>=0.1.1; extra == 'vector'
Description-Content-Type: text/markdown

<p align="center">
  <img src="https://raw.githubusercontent.com/Curry-Leaves/curry-leaves-memory/main/assets/logo.png" alt="Curry Leaves logo" width="128" height="128">
</p>

<h1 align="center">Curry Leaves Memory</h1>

<p align="center">One memory substrate for agents — episodic, semantic, identity and whatever comes next — as plain markdown files, with a disposable index and layered search.</p>

<p align="center">
  <a href="https://pypi.org/project/curry-leaves-memory/"><img src="https://img.shields.io/pypi/v/curry-leaves-memory.svg" alt="PyPI version"></a>
  <a href="https://github.com/Curry-Leaves/curry-leaves-memory/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="license: MIT"></a>
  <a href="https://www.python.org"><img src="https://img.shields.io/badge/python-%3E%3D3.11-brightgreen.svg" alt="python: >=3.11"></a>
  <a href="https://github.com/Curry-Leaves/curry-leaves-memory/blob/main/src/cl_memory/py.typed"><img src="https://img.shields.io/badge/types-included-blue.svg" alt="types included"></a>
  <a href="https://github.com/Curry-Leaves/curry-leaves-memory"><img src="https://img.shields.io/badge/github-repo-181717.svg?logo=github" alt="GitHub repo"></a>
</p>

**Curry Leaves Memory** gives an agent a memory it can keep — and only one of them. Episodic,
semantic, identity, facts, learnings: they share a single store, a single write path,
and a single query surface, instead of a separate product per memory type. Point it at a
directory and you get stable ids, link integrity, edit history, health checks, layered search, a
knowledge-map walk, and a **semantic / episodic / consolidated** model with room for any type you
invent. Your memory is plain `.md` files — the SQLite index and embeddings are *derived,
disposable* state that rebuilds from them.

```python
from cl_memory import Bundle, FtsIndex, memory_conventions

mem = Bundle("~/agent-memory", index=FtsIndex(), conventions=memory_conventions())
mem.seed()

mem.remember("facts/jwt.md", type="semantic",
             meta={"title": "JWT auth", "description": "tokens signed with a rotating key"})
mem.recall("how does auth work")
```

- **Distribution:** `curry-leaves-memory` · **import:** `cl_memory` · **License:** MIT
- **Requires** Python ≥ 3.11. The base runtime dependency is `pyyaml` — nothing else.

---

## Table of contents

- [Why this exists](#why-this-exists)
- [Install](#install)
- [The three tiers](#the-three-tiers)
- [Memory model](#memory-model--semantic--episodic--consolidated)
- [Knowledge map](#knowledge-map--trace-and-path)
- [Web console & REST API](#web-console--rest-api)
- [LLM layer](#llm-layer-optional--contribcurryleaves)
- [Configuration](#configuration)
- [API reference](#api-reference)
- [Concurrency](#concurrency)
- [Docs](#docs)
- [License](#license)

## Why this exists

Agent memory is fragmenting. Episodic memory, semantic memory, identity and persona, facts,
learnings, knowledge graphs, the Organized Knowledge Folder convention — each is growing into its
own product, its own schema, its own store. Wire up three of them and you own three databases,
three query languages, three sync problems, and no way to ask a question that crosses them.

**An agent does not want five memory products. It wants a memory.** What happened last Tuesday,
what it knows to be true, what it learned, and who it is are not different systems — they are
different *questions* asked of one substrate.

That is the bet this framework makes. One store, one write path, one query surface; the
distinctions live in a `type:` field on an ordinary note, not in separate infrastructure:

```python
mem.remember("identity/self.md",   type="identity", ...)   # who I am
mem.remember("facts/jwt.md",       type="semantic", ...)   # what is true
mem.remember("events/2026-06.md",  type="episodic", ...)   # what happened
mem.remember("lessons/auth.md",    type="learning", ...)   # what I learned

mem.recall("auth incident", type="episodic")   # ask one type
mem.recall("auth")                             # or ask across all of them
```

**`type:` is an open string, not an enum.** `semantic` / `episodic` / `consolidated` get extra
machinery (event-time ordering, clustering, consolidation), but nothing is gatekept: an
`identity` or `learning` note is stored, indexed, ranked, linked, traversable and recallable on
exactly the same terms. A new memory category is a new string, not a new dependency — and because
the edges are plain markdown links, an episode can link to the fact it was about and `trace()`
will walk from one to the other.

Note the deliberate boundary: this is **memory, not skills**. *How to do something* — runbooks,
tools, executable procedures — belongs to the agent's skill layer (in this family, the
[`curry-leaves`](https://pypi.org/project/curry-leaves/) kernel), not to the memory substrate.
Memory records what happened, what is true, and what was learned; the skill layer decides what to
do about it. Nothing stops you storing a `type: procedural` note if you want one, but the
framework does not pretend that a stored document is a runnable capability.

The substrate underneath is deliberately boring: **your memory is plain `.md` files** you can
read, edit, `grep`, `git` diff, sync, and back up with any tool you already own. The SQLite index
and embeddings are *derived, disposable* state — delete `.index/` and it rebuilds from the files.
Nothing is trapped, there is no server to run, and the format outlives this library. That is what
makes it a foundation rather than another store to migrate off later.

Two consequences worth knowing up front:

- **No bundled models.** Embedders (`embed=`) and the consolidation summarizer (`summarize=`)
  are host-injected. cl_memory never downloads, calls, or bills a model on your behalf.
- **A bad index never blocks a write.** Index failures are best-effort; the markdown heals the
  index, never the reverse.

On disk a bundle is OKF-conformant (Open Knowledge Format v0.1): a directory of markdown + YAML
frontmatter + standard markdown links.

## Install

```bash
pip install curry-leaves-memory            # tier 0 + tier 1 (files + SQLite FTS)
pip install "curry-leaves-memory[api]"     # + FastAPI router and the runnable web console
pip install "curry-leaves-memory[vector]"  # + sqlite-vec acceleration for vector search
pip install "curry-leaves-memory[agent]"   # + the curry-leaves LLM layer
```

## The three tiers

Capability is additive and explicit — one `Bundle` facade, chosen by the `index=` argument. You
pay for indexing only when your *questions*, not just your data, outgrow scanning.

| Tier | Construction | You get | Costs |
|---|---|---|---|
| **0** | `Bundle(root)` | files + CRUD, pure-Python scan search | `pyyaml` only |
| **1** | `Bundle(root, index=FtsIndex())` | BM25 ranked search, O(1) backlinks/graph | stdlib SQLite |
| **2** | `Bundle(root, index=FtsIndex(vector=VectorIndex(embed=fn)))` | + semantic & hybrid retrieval | your embedder |

### Tier 0 — files + CRUD (default)

```python
from cl_memory import Bundle

kb = Bundle("~/my-notes")
kb.seed()                                    # index.md + CONVENTIONS.md skeleton (idempotent)

kb.write("topics/fastapi.md",
         "---\ntype: topic\ntitle: FastAPI\ndescription: async web framework\n---\n\nNotes…")
note = kb.read("topics/fastapi.md")          # {path, frontmatter, body, id, type, title, …}
kb.edit("topics/fastapi.md", [{"old": "Notes…", "new": "Rewritten."}])
kb.move("topics/fastapi.md", "apps/api")     # inbound + outbound links rewritten
kb.delete("notes/scratch.md", reason="superseded")   # soft-delete -> _archive/

kb.search("dependency injection")            # scan search at tier 0
kb.links("apps/api/fastapi.md")              # {outbound: [...], inbound: [...]}
kb.graph(); kb.health(); kb.history("apps/api/fastapi.md")
```

### Tier 1 — SQLite FTS5 (BM25), stdlib only

```python
from cl_memory import Bundle, FtsIndex

kb = Bundle("~/my-notes", index=FtsIndex())
kb.seed()
kb.search("how do we deploy")                # BM25: title > tags/aliases > description > body
```

The FTS index is **contentless** — it keeps only the inverted index, never a copy of your note
text (`index.db` ≈ 0.3–0.5× your corpus, not a second copy). It updates incrementally on every
write and is reconciled against the files on `seed()`. Delete `.index/` any time; `kb.reindex()`
rebuilds it. `FtsIndex(scope="metadata")` indexes only title/tags/description for a tiny index.

> **On `score`.** Results are ranked by BM25, a *collection-relative* statistic. A term that
> appears in every note has near-zero IDF, so on a tiny or homogeneous corpus scores legitimately
> collapse toward `0.0`. Trust the **ordering**, not the absolute value — it isn't comparable
> across corpora or between the keyword and vector legs.

> **SQLite ≥ 3.43** is needed for the contentless index (check with
> `python -c "import sqlite3; print(sqlite3.sqlite_version)"`). On older SQLite cl_memory
> automatically falls back to a **regular FTS5** index and logs it once: search, ranking,
> snippets, deletes and every API behave identically — the only difference is that `index.db`
> then holds a tokenized copy of your note text, so the "derived structure only" property no
> longer holds. `FtsIndex.contentless` reports which shape is live — read it *after* the index is
> bound to a bundle (i.e. after `seed()`), since the shape is chosen at bind time.

### Tier 2 — vectors in the same SQLite

Bring your own embedder — cl_memory never downloads a model.

```python
from cl_memory import Bundle, FtsIndex, VectorIndex

def embed(texts: list[str]) -> list[list[float]]:
    ...  # call Ollama, an API, or sentence-transformers; return one vector per text

kb = Bundle("~/my-notes",
            index=FtsIndex(vector=VectorIndex(embed=embed, model="bge-small", dim=384)))
kb.seed()
kb.search("how do we deploy", mode="hybrid")  # keyword | vector | hybrid (RRF fusion)
```

Embeddings live in the same `index.db` (`embeddings` table), keyed by content hash, and are
embedded lazily so writes stay cheap. By default only the note's summary (title + description +
tags) is embedded (`VectorConfig.scope="metadata"`); pass `scope="full"` to embed the body.
Hybrid mode fuses the two legs with Reciprocal Rank Fusion, which is rank-based — so the
incomparable BM25 and cosine scores never need normalizing.

## Memory model — semantic · episodic · consolidated

Three cognitive types share one substrate: **semantic** (timeless facts), **episodic** (dated
events), **consolidated** (durable summaries derived from episodes). They are just `type:`
frontmatter values on ordinary notes, so search, links, history, and `trace()` apply unchanged.

`type:` is an **open string**, never an enum. Those three get special handling, but `topic`,
`person`, `project`, `identity`, `learning` — or anything else you invent — is a first-class
note: stored, indexed, ranked, linked, traversable, and recallable on the same terms. Your whole
knowledge hub and your agent's memory live in one bundle. `MEMORY_TYPES`, `HUB_TYPES` and
`KNOWN_TYPES` are suggested vocabularies for docs and UIs, not constraints.

Because every type shares one substrate, memory that spans categories just works — an episode can
link to the fact it was about, and the knowledge map walks across the boundary:

```python
mem.remember("events/2026-06-01.md", type="episodic",
             body="Prod auth returned 500s. Root cause was the key in [JWT auth](/facts/jwt.md).",
             meta={"title": "Auth 500s", "description": "prod auth incident",
                   "occurred": "2026-06-01T14:30:00+00:00"})

mem.recall("auth incident", type="episodic")
# -> ['events/2026-06-01.md']                              one type

mem.recall("auth")
# -> ['facts/jwt.md', 'events/2026-06-01.md', ...]         across every type

mem.trace("auth incident")["path"]
# -> ['events/2026-06-01.md', 'facts/jwt.md']              episodic -> semantic, one hop
```

```python
from cl_memory import Bundle, FtsIndex, memory_conventions

mem = Bundle("~/agent-memory", index=FtsIndex(), conventions=memory_conventions())
mem.seed()

mem.remember("facts/jwt.md", type="semantic",
             meta={"title": "JWT auth", "description": "tokens signed with a rotating key",
                   "tags": ["auth"]})
mem.remember("events/2026-06-01.md", type="episodic",             # `occurred` defaults to now
             meta={"title": "Auth 500s", "description": "prod auth returned 500s",
                   "tags": ["auth", "incident"], "occurred": "2026-06-01T14:30:00+00:00"},
             body="Prod auth failing, see [JWT auth](/facts/jwt.md).")

mem.recall("how does auth work", type="semantic")   # relevance search, filtered by memory type
mem.timeline(since="2026-06-01T00:00:00+00:00")     # episodic recall, newest first
mem.forget("events/scratch.md")                     # soft-delete -> _archive/ (restorable)
```

Episodic notes carry `occurred` (when the event happened) distinct from the write-time
`timestamp` — an event that happened yesterday but is recorded today still sorts as yesterday.

### Restarting on existing memory

Point a `Bundle` at a directory that already has notes and call `seed()`. There is no separate
"open" call and no migration — the same code path creates a bundle or resumes one, so an agent
restarts with its memory intact:

```python
mem = Bundle("~/agent-memory", index=FtsIndex(), conventions=memory_conventions())
mem.seed()                       # resumes; overwrites nothing
mem.recall("what did we decide about auth")
```

Note content, stable ids, links, history and the index all survive. `seed()` reconciles by
content hash, so edits you made in your editor while the process was down are picked up, and a
deleted `.index/` is simply rebuilt from the files. Details and the recovery matrix are in
[docs/usage.md](docs/usage.md#21-booting-on-an-existing-bundle).

### Consolidation — fold recurring events into a durable fact

Cluster related episodes, summarize them into one `consolidated` note, then archive the raw
episodes to keep the active set small. Clustering is pure logic: episodes that share a tag
**and** a link within a time window (both required, so memories are never fabricated). The
summarizer is host-supplied; without it `consolidate()` is a no-op and `consolidation_candidates()`
just reports the clusters.

```python
def summarize(episodes: list[dict]) -> dict:       # your LLM here
    return {"title": "Auth Incidents",
            "description": "recurring prod auth 500s fixed by rotating the JWT key",
            "body": "…"}

mem = Bundle("~/agent-memory", index=FtsIndex(),
             conventions=memory_conventions(), summarize=summarize)
mem.consolidation_candidates()   # clusters (no LLM needed)
mem.consolidate()                # write consolidated notes + link provenance + archive raw
```

Consolidation is idempotent and also runs as a Gardener pass (`mem.garden()`).

## Knowledge map — trace() and path()

Start at a point, follow the strongest links, get the full story. `trace()` seeds from search and
best-first walks authored links (blending query-relevance with structure — mutual links, degree,
shared tags), bounded by a token budget, into a deterministic reading outline.

```python
t = kb.trace("how does checkout handle payment failures", budget=2000)
t["seed"]; t["path"]; t["edges"]; t["outline"]     # ordered reading path + why each hop
t = kb.trace("…", summarize=my_llm)                # optional prose `story` (BYO summarizer)

kb.path("people/priya.md", "topics/retries.md")    # shortest hop-chain between two notes
```

## Web console & REST API

```bash
pip install "curry-leaves-memory[api]"
python -m cl_memory.serve --root ~/my-memory --port 8000
```

Serves a self-contained browser console at `/` for browsing, searching, editing, tracing and
consolidating — plus the REST API under `/memory/*` and `/map/*`. This is a **dev/test harness,
not a hardened server**: no auth, single writer. Don't expose it to a network you don't trust.

To mount the API in your own app, or to wire an embedder / summarizer the CLI can't pass:

```python
from fastapi import FastAPI
from cl_memory import Bundle, FtsIndex
from cl_memory.contrib.fastapi import build_router

kb = Bundle("~/my-notes", index=FtsIndex()); kb.seed()
app = FastAPI()
app.include_router(build_router(kb))          # /knowledge/*, /memory/*, /map/*
```

## LLM layer (optional) — `contrib.curry_leaves`

Bridges the memory bundle to the [`curry-leaves`](https://pypi.org/project/curry-leaves/) agent
kernel. The core never imports it — the base runtime stays `pyyaml`-only.

```python
from cl_memory import Bundle, FtsIndex, memory_conventions
from cl_memory.contrib.curry_leaves import llm_summarize, memory_tools

# 1) LLM-backed consolidation: episode clusters fold into model-written durable notes
kb = Bundle("~/mem", index=FtsIndex(), conventions=memory_conventions(),
            summarize=llm_summarize(model="claude-sonnet-5"))
kb.consolidate()

# 2) Memory as agent tools: an agent reads and writes its own memory mid-conversation
from curry_leaves import Agent, Runner
agent = Agent(model="claude-sonnet-5", instructions="You have persistent memory.",
              tools=memory_tools(kb))   # remember, recall, timeline, trace, consolidate
result = await Runner(agent).run("What did we decide about auth last week?")
```

There is no `llm_embed` — `curry-leaves` is a tool-use kernel, not an embedding provider; bring
your own embedder for `VectorIndex(embed=…)`.

## Configuration

```python
from cl_memory import Bundle, Conventions, Guards, FtsIndex

kb = Bundle(
    root="~/team-kb",
    index=FtsIndex(),
    conventions=Conventions(
        areas=("apps", "topics", "people"),   # None = allow any top-level folder
        required_fields=("type",),
        id_prefix="kn_",
    ),
    guards=Guards(shrink_threshold=0.40),
    on_event=lambda kind, payload: print(kind, payload),   # neutral event names
    provenance_resolvers={"meeting": my_meeting_span_resolver},
)
```

All config objects are frozen dataclasses in `cl_memory.config`: `Conventions`, `Guards`,
`GardenerConfig`, `VectorConfig`, `TraceConfig`.

## API reference

Everything is a method on `Bundle`. Writes route through one unified write path that keeps
history, hubs, links and the index consistent under a reentrant lock. Full signatures, return
shapes and error behavior are in **[docs/usage.md](docs/usage.md)**.

**Read** — `read`, `read_raw`, `read_file`, `notes`, `dirs`, `resolve`, `established_tags`,
`conflicts`

**Write** — `write`, `write_raw`, `edit`, `upsert_meta`, `system_write`, `delete`, `move`,
`create_dir`, `move_dir`, `archive_dir`

**Search & graph** — `search`, `links`, `graph`, `trace`, `path`

**Memory** — `remember`, `recall`, `timeline`, `forget`, `consolidation_candidates`, `consolidate`

**Maintenance** — `seed`, `garden`, `health`, `history`, `provenance`, `reconcile`, `reindex`,
`regenerate_index`, `close`

**Ingest ledger** — `note_ingest`, `already_ingested`, `ingested_notes`

**Properties** — `root`, `tier`, `has_summarize`

Guard note: `write` enforces the shrink guard (rejects rewrites dropping >40% of a note) and the
area whitelist; `system_write` bypasses the shrink guard (system rewrites are authoritative);
`write_raw` is lenient (deliberate human edits from a UI).

## Concurrency

A reentrant write lock on each `Bundle` serializes mutations **in-process** (the multi-file
derived-state updates aren't individually atomic). SQLite uses one connection per thread in WAL
mode. Multi-process or multi-agent hosts should keep a **single writer per bundle directory**
(e.g. a width-1 work lane). Readers are unconstrained.

## Docs

- [docs/usage.md](docs/usage.md) — **module reference**: every method with its real signature and
  return shape, the errors and guards you will hit, extension points, and recipes
- [docs/architecture.md](docs/architecture.md) — design philosophy, structural layers, the write
  path, storage model, the `SearchIndex` protocol, concurrency contract
- [examples/quickstart.py](examples/quickstart.py) — runnable end-to-end tour
- [CONTRIBUTING.md](CONTRIBUTING.md) — dev setup and the invariants to respect
- [CHANGELOG.md](CHANGELOG.md)

## License

MIT © Ilayanambi Ponramu
