Why BeliefState?

Building a simple chat loop with an LLM is easy. Building a production conversational agent that maintains consistent, long-term, and logical state is hard. BeliefState was built to bridge the gap between stateless LLM APIs and stateful conversational applications.

The Core Problem: LLMs are Stateless

Every request to an LLM API (like OpenAI's GPT-4o or Anthropic's Claude) is completely independent. The model does not remember the previous turn, the user's name, or what preferences were established unless they are sent in the prompt history. This leads to three fundamental issues:

Context Window is Not Memory

Developers often assume that expanding context windows (like Gemini's 1M+ tokens or Claude's 200k tokens) renders state tracking obsolete. In practice, long context windows are transport channels, not working memory.

• Reasoning Latency: Feeding hundreds of turns into an LLM increases the time-to-first-token (TTFT) and overall generation latency.
• Noise Sensitivity: Long histories contain conversational filler (e.g. "Got it!", "Thanks!"). LLMs struggle to separate key user facts from temporary conversational noise.
• Truncation Failure: If you truncate history to save cost or latency, you risk deleting critical user constraints (like dietary restrictions, database keys, or project rules).

BeliefState solves this by extracting facts into a structured, persistent belief state store in the background. Instead of feeding the raw history back to the model, you retrieve a compact, relevant, and clean summary of facts and inject it as system context.

Common Design Patterns

Here is how belief state tracking is used in real-world production architectures:

1. Customer Support & Concierge Agents

Ensure consistent enforcement of user-declared preferences (e.g., budget, allergies, shipping address) across multiple sessions.

# Turn 1: User specifies constraint
user: "I need to plan a trip to Tokyo. My budget is $3,000."
# beliefstate extracts: (USER, budget, "USD 3000")

# Turn 25: Assistant tries to suggest a premium hotel
assistant: "I recommend the Ritz-Carlton. It's only $1,200 per night!"
# beliefstate background check detects:
#   Contradiction: Proposed lodging budget exceeds user budget limits!
# Resolution: Injects conflict into prompt context to trigger model self-correction.

2. Developer Coding Agents

Maintain consistency in file structures, runtime environments, and architectural decisions throughout a complex coding workspace session.

# Turn 3: Agent decides framework
agent: "I will write the backend in FastAPI using aiosqlite for persistence."
# beliefstate extracts: (PROJECT, database, "aiosqlite"), (PROJECT, framework, "FastAPI")

# Turn 18: Agent forgets and attempts to import psycopg2
agent: "Let's import psycopg2 and connect to the postgres database..."
# beliefstate background checker triggers:
#   Contradiction: Project uses aiosqlite database, not postgres/psycopg2!
# Resolution: Prompt correction blocks the commit and keeps the agent aligned.

Why use beliefstate instead of standard Vector DBs?

Standard RAG (Retrieval-Augmented Generation) uses vector search to retrieve chunks of raw text history. This fails for state tracking: