What Your Agent Remembers
Is What It Becomes.
Durable, cross-session memory with four-way fused recall. Core rules auto-loaded into every system prompt.
pip install luminary-memory
from luminary_memory import MemoryClient
client = MemoryClient(db_path="memory.db")
# 1. Ingest facts, preferences, or environment states
client.ingest(
"The deploy target is the staging cluster",
tags=["deploy", "infra"]
)
# 2. Recalled across semantic, keyword, temporal, & graph
result = client.recall("where do we deploy?")
for m, s in zip(result.memories, result.scores):
print(f"{s:.3f} {m.content}")
# → 0.942 The deploy target is the staging cluster
# 3. Autonomous maintenance sweep
client.run_lifecycle() # TTL + consolidation + prune + max cap# 1. Ingest a memory with metadata tags
luminary-memory add "The deploy target is the staging cluster" --tags deploy infra
# 2. Recall with 4-strategy parallel fusion
luminary-memory recall "where do we deploy?" --limit 5 --json
# 3. Fast keyword search across FTS5 index
luminary-memory search "staging"
# 4. Autonomous maintenance (TTL, dedup, pruning, max cap)
luminary-memory lifecycle
# 5. Store health report (0-100) + contradiction warnings
luminary-memory health
# 6. Check memory store stats and count
luminary-memory stats# Core memory: DB-backed MEMORY.md, auto-loaded every session
# Agent-side: pin a durable rule (via the luminary_core_add tool)
luminary_core_add("always use markdown tables for all reports")
# → {"result": "Core memory stored (id=42)."}
# See what is pinned
luminary_core_list()
# → {"core": [{"id": 42, "content": "always use markdown tables...", ...}]}
# Unpin a rule (keeps it in the store, drops from system prompt)
luminary_core_remove(id=42)
# → {"result": "memory 42 removed from core"}
# Config: chars budget + tag (env vars)
# LUMINARY_CORE_TAG=core LUMINARY_CORE_TOP_N=12
# LUMINARY_CORE_BUDGET=8000 (characters)LLM Tokens / Recall
Retrieval runs locally via CPU ONNX embeddings. You never burn API tokens to search memory.
End-to-End Recall (p50, 5k store)
Measured on 5,000-memory stores (deterministic pipeline). 4 parallel strategies fused, MRR 1.0.
Fused Strategies
Vector cosine, SQLite FTS5 keyword, temporal decay, and entity graph blended via weighted RRF (k=60).
Agent Tools
luminary_recall, luminary_ingest, luminary_list, plus luminary_core_add / luminary_core_remove / luminary_core_list.
Stateless Agents Forget Everything. Here Is How Luminary Fixes That.
When agents restart from an empty prompt every turn, they repeat past mistakes, re-query known data, and inflate token bills. Luminary provides an embedded, self-pruning memory store that sits directly inside your Python process. Every memory is embedded locally, indexed across four dimensions, and retrieved in parallel without cloud roundtrips.
- 01 Semantic Vector Search
- Local 384-dimensional ONNX vector similarity (
BAAI/bge-small-en-v1.5), vectorized matmul. Captures conceptual meaning on your CPU. - 02 Keyword Matching (FTS5 OR)
- SQLite FTS5 with BM25 ranking, terms joined with OR — a multi-word query like "use tables in reports" matches any term, and bm25 lifts the best document. No more empty results from AND joins.
- 03 Temporal Half-Life Decay
- Exponential decay curves weighted by access frequency and age (batched fetch, no N+1). Surfaces fresh and frequently accessed facts.
- 04 Co-Occurrence Entity Graph
- Relational entity graph with SQL aggregation. Discovers contextual links even when the query shares no direct keywords.
Parallel Fusion & Anti-Contradiction
All four strategies run concurrently and fuse via weighted Reciprocal Rank Fusion (semantic 0.4, keyword 0.3, graph 0.2, temporal 0.1). Short queries are expanded with graph entities — and when the graph is empty, with keywords from a durable rule on the same topic (v0.2.15). Jaccard dedup (0.85) removes paraphrases, adaptive cliff cutoff keeps only the relevant cluster, and a hard token budget protects the context window. Rules at importance ≥ 0.9 are pinned (never pruned or consolidated away), and a similar rule ingest auto-replaces the old one — "never use tables" never coexists with "always use tables".
Core Memory. Auto-Loaded Every Session.
The DB-backed equivalent of Hermes' native MEMORY.md — but stored in the database, patched by the agent, and always present from the very first prompt.
Always in the system prompt
Memories tagged core are injected into the system prompt every session, before persistent context and recall. A new session that asks "riset x, y, z" without ever mentioning "tabel" still has the table rule in context from turn one — no query match needed.
Patched by the agent
Three tools let the agent maintain its own core memory: luminary_core_add (pin a rule, importance ≥ 0.9), luminary_core_remove (unpin, keeps the memory in the store), and luminary_core_list (inspect what is pinned).
Character-budgeted
Configurable via LUMINARY_CORE_TAG (default core), LUMINARY_CORE_TOP_N (12 memories), and LUMINARY_CORE_BUDGET (8000 characters) — so core memory scales to your model's system-prompt appetite like MEMORY.md's char limit.
Never duplicated
Core memories are tracked in the injected-id set, so they never reappear in persistent context or query recall. Three-way dedup across core, persistent, and recall blocks.
Six Tools. The Agent Manages Its Own Memory.
In hybrid mode, the model can query, store, and curate memory on demand through function-calling tools.
luminary_recallluminary_ingestluminary_listluminary_core_addluminary_core_removeluminary_core_listTested at Scale. Zero Marketing Numbers.
Benchmarked with a reproducible harness on standard x86_64 CPUs. Numbers below are measured, not aspirational.
Want to fill in the pgvector numbers? Issue #6 is a real benchmark task.
Built for Agents That Do Real Work.
Coding agents that keep context
Repository conventions, flaky test workarounds, architecture decisions. Your coding agent persists them across CLI sessions, so it starts every morning with what it learned last week instead of a blank prompt.
Hermes agents with zero-token memory
Luminary is a first-class Hermes memory provider (memory.provider=luminary). Every turn recalls the relevant facts, core rules auto-load into the system prompt, and every session exit saves what mattered. Retrieval costs zero LLM tokens because embeddings run locally on CPU.
Background workers that survive restarts
Cron jobs and pipeline workers share outcomes, partial states, and rate-limit budgets across process restarts. Memory lives in a SQLite file, not in a process that dies at 3 a.m.
Format rules that never get forgotten
"always use markdown tables", "em dash 0", "no @ mentions" — pin them with luminary_core_add and they are in the system prompt of every new session, even when the user never mentions them.
Running in Four Steps.
-
1
Install Package
One lightweight package with bundled ONNX CPU embeddings. No GPU or cloud credentials required.
pip install luminary-memory -
2
Store Memories
Ingest durable facts, environment configurations, and preferences with metadata tags.
client.ingest("The deploy target is staging", tags=["deploy"]) -
3
Fused Recall
Query naturally. The four-strategy fused retrieval pipeline returns scored, deduplicated memories.
luminary-memory recall "where do we deploy?" --json -
4
Autonomous Lifecycle
Autonomous lifecycle sweep handles TTL expiration, near-duplicate consolidation, importance pruning, and the
max_memoriescap — pinned rules exempt.luminary-memory lifecycle
One Loop, Running Every Turn.
Memory is not a pipeline you run once. It is a loop your agent lives in: core rules loaded, recall, act, ingest, repeat.
Core rules loaded first
At session start, the <memory-context> block is built from core memory (tagged core) plus the top-N by importance. Injected into the system prompt — the agent always knows the durable rules.
Recall before you answer
At the start of every turn, four strategies run in parallel: vector cosine, FTS5 OR keyword, temporal decay, entity graph. Weighted RRF (k=60) fuses them, Jaccard dedup (0.85) drops paraphrases, and a token budget (4096) caps what enters the context window. The recalled block is appended to the turn's message as authoritative reference data.
Act with context
The agent answers with the recalled facts already in context. Zero LLM tokens were spent on retrieval: embeddings run locally on CPU via ONNX.
Ingest what mattered
After the turn, durable facts are extracted (optional LLM curation filters chit-chat; rule keywords are checked only against the curated summary, never raw transcripts), embedded locally, and stored with metadata tags and co-occurrence relations.
Lifecycle keeps it sharp
Periodically, TTL cleanup expires stale facts, semantic consolidation merges near-duplicates (pinned rules exempt), importance-based pruning drops low-value noise, and the max_memories cap is enforced. The store stays lean without you touching it.
SQLite by Default. pgvector When You Scale.
Luminary abstracts storage behind a pluggable interface. Start instantly with zero setup on your machine, then migrate seamlessly when your infrastructure demands it.
SQLite + FTS5
Default- Dependencies Python standard library, zero extra installs
- Vector Engine In-process cosine similarity on CPU
- Keyword Matching Built-in SQLite FTS5 index (BM25 ranking, OR join)
- Ideal Scale Single-user agents, local CLI, edge (under 100k memories)
- Setup Zero-config instant local
.dbfile
PostgreSQL + pgvector
- Dependencies PostgreSQL 14+ with pgvector extension
- Vector Engine pgvector cosine similarity (HNSW-ready)
- Keyword Matching ILIKE pattern matching on content and tags
- Ideal Scale Multi-agent fleets, backend services, enterprise
- Setup Database connection string (
LUMINARY_PG_DSN)
Frequently Asked Questions
Does any memory data or prompt text leave my machine?
No. Luminary is completely self-hosted. Embeddings are generated on your local CPU using an ONNX runtime (BAAI/bge-small-en-v1.5), and memories are persisted directly to your local SQLite file or your own PostgreSQL instance. There is zero telemetry and zero external API calls.
Do I need a GPU or third-party embedding API keys?
No. The default embedding engine runs locally on standard CPU hardware via ONNX with zero external API keys. It runs efficiently on laptops, Linux servers, and containerized CI environments.
Why four retrieval strategies instead of just a vector database?
Vector search alone struggles with exact identifiers (function names, file paths, error codes) and ignores temporal recency. Luminary combines vector similarity with SQLite FTS5 exact keyword matching (OR-joined so multi-word queries never come back empty), temporal decay curves, and entity graph co-occurrences. Results are merged via Reciprocal Rank Fusion (k=60) and pruned with Jaccard deduplication (0.85).
How does core memory differ from normal recall?
Recall is query-driven: it surfaces memories relevant to the current question. Core memory (tagged core) is auto-loaded into the system prompt every session, independent of any query — the DB-backed equivalent of Hermes' native MEMORY.md. Pin format rules, identity, and critical instructions with luminary_core_add; they are present from the very first prompt. Capped by core_top_n and core_budget (characters), and deduplicated against recall so nothing appears twice.
How does Luminary prevent memory store bloat over time?
Several layers keep the store lean: deterministic lifecycle passes (run_lifecycle()) that expire TTLs, consolidate near-duplicates via semantic cosine clustering, and prune low-importance entries (<0.2); a hard max_memories cap enforced by the lifecycle; and an optional LLM maintenance review (run_maintenance()) that cleans obsolete facts at session end. Rule pinning ensures durable rules survive every pass.
How do I check memory store quality?
Run luminary-memory health in your CLI. It evaluates duplicate rate (25%), staleness (25%), importance distribution (20%), graph relation density (15%), and store size (15%), outputting an overall 0-100 health score with actionable maintenance recommendations.
How do I switch from SQLite to pgvector for large-scale fleets?
Set LUMINARY_BACKEND=pgvector and configure your LUMINARY_PG_DSN connection string. All client methods (ingest, recall, run_lifecycle) share the identical Python API, letting you scale without modifying application code.
How does the Hermes agent integration work?
Configure memory.provider=luminary in your Hermes agent settings. The provider performs local fused recall on every conversational turn (zero LLM tokens), injects core memory + top-N important memories into context every turn as persistent context, exposes six memory tools to the model, and triggers automatic store maintenance when the session closes.
How does Luminary keep important rules from being forgotten?
Four safeguards: (1) Core memory — memories tagged core are auto-loaded into the system prompt every session (the DB-backed equivalent of MEMORY.md), so durable rules are present from the very first prompt. (2) Persistent context — the top-N memories by importance are injected every turn, merged with query recall under anti-duplication (by id and content hash, so identical text never appears twice). (3) Rule pinning — memories at importance ≥ 0.9 are never pruned or consolidated away. (4) Rule auto-replace — a similar rule ingest replaces the old one (anti-contradiction), so "never use tables" never coexists with "always use tables". Since v0.2.15 the store also learns from use: memories that keep getting recalled are re-estimated immediately and climb into the next turn's context.
Give Your Agents a Memory
That Lasts.
Self-hosted, private, and ready in seconds. Core rules always in context, facts always recalled.
pip install luminary-memory