Hearth · synthesis · 20 October 2026

What this project is, in language you can reason with

Hearth is a Python library that sits inside an agent’s own process, on the path between the agent’s code and the model company. On every call it keeps the stable beginning of the prompt identical, records what the cache did, and — only if you ask — moves the bits that would have broken that cache.

Package hearth-ai, import hearth, version 0.1.0b1. Public repo jishmisc28/hearth. Tag on GitHub: 0.1.0a1. Not on PyPI. Written 19 October 2026. This page is a reading copy of the code and the status log, dated 20 October 2026.

The idea

An agent is a loop. Step 30 sends the system instructions, the tool definitions, and every earlier message again, then adds one new piece. The model company already saw the beginning of that text. If the beginning is byte-for-byte the same, they charge about a tenth of the normal input price for it, and the first token often arrives sooner. If one byte near the start changes, they charge full price for everything after it.

Memory, shortening the history, parking a huge tool result, and switching to a cheaper model all look like separate features. Each one is a decision about that text, and each one has a cache price. Hearth is the place those decisions are made together, with the price written down in exact decimal arithmetic.

It is a library. The agent’s framework stays the agent’s framework. There is no server to run, no proxy in front of the API, and no user interface. You install it and wrap the client you already have.

The mechanism, without the product words

Providers match the cache from the first byte. A line like Current time: 2026-10-20T10:55:00 at the top of the system prompt changes on every step. The tools, the instructions, and the whole conversation after that line miss the cache, even though none of them changed.

The fix is placement. Text that must stay identical lives at the front. Text that changes every turn lives at the end, after the last cached position. The model still sees the time. The cache still sees a stable prefix.

Broken layout — the clock is first, so the cache misses the rest

time instructions tools history new turn

Stable layout — the clock is last, so the front can be read from cache

instructions tools history new turn time

A worked illustration from the handbook, labeled there as arithmetic, not a measurement: 30 steps, context growing from 10,000 to 70,000 tokens, about 1.2 million input tokens. With no cache hits that is 1.2 million units of the base price. If 85% is read at 0.1× and the rest is written at 1.25×, the bill is about 0.27 of the uncached cost. One timestamp at the front returns the bill to full price.

Three promises, chosen by a mode

Adoption is a mode, because changing an agent’s prompt is a behaviour change. The default in the design is safe. The function’s own default argument is still observe, which is the first-run setting.

Mode	What leaves the process	What you get
`observe`	The original request, byte for byte. Hearth does not “improve” it.	A measurement and a diagnosis. Useful on the first day, when you want to see the damage without touching the agent.
`safe`	The same messages, the same tool text, the same words. Order of tools may be put back to the order seen on the first call of the session. Cache markers may be added.	Protection against accidental reordering, plus a plan for where the provider should keep a cache breakpoint.
`optimize`	The same task, with documented moves: a timestamp pulled out of the system prompt into the latest user turn; a huge tool result replaced by a short stub, if the app registered a tool to read the original back.	The layout fix. This is opt-in because the bytes the model sees do change, even though the meaning is meant to stay.

strict=True is separate from the mode. If the guard sees a break that the application caused, it raises HearthPrefixBreak before the request is sent. That is for tests. If Hearth itself throws, the original request still goes out and the event is recorded as hearth_error. A bug in the library is not allowed to stop the agent.

What one call actually does

You write client = hearth.wrap(client, mode="safe"). The object you get back is a proxy. When the agent calls the method that sends a completion, this sequence runs. The response the agent receives is the provider’s response.

Intercept. The proxy catches the call. In observe it stops here for edits and sends the original arguments.src/hearth/wrapper.py · WrappedMethod.__call__

Restore tool order, only in safe and optimize. The first time a session sees a set of tools, that order is remembered. If a later call sends the same tools in a different order, they are put back. The tool text is unchanged. The restoration is written into the trace.wrapper.py · _process_safe_request

Relocate and offload, only in optimize. Lines in the system prompt that look like a timestamp, a UUID, a git commit, or a branch name are moved onto the latest user message. Tool outputs over the size threshold can be replaced with a content-addressed stub.src/hearth/layout.py · src/hearth/offload.py

Cut the request into segments. Tools, system text, earlier messages, and the newest message are labeled by how stable they are allowed to be.src/hearth/segments.py · segment_request

Compare with the previous call. The prefix guard finds the first real difference and names a cause. Some causes are the application’s. Some are the provider’s, or the prompt simply being too short to cache. The second group is never reported as your fault.src/hearth/guard.py · PrefixGuard

Decide whether a cache marker is worth sending. Anthropic allows at most four breakpoints, and a one-hour marker has to sit before a five-minute one. A one-shot call should not pay a cache-write premium. If you have said a human pause is coming, the stable front can take the longer lifetime.src/hearth/planner.py · plan_breakpoints, TTLOptimizer

Send, then read the bill. The provider’s usage fields are translated into three numbers that must add up: tokens read from cache, tokens written into cache, tokens charged in full. Cost is computed from the dated price table with Decimal, never with floating point.src/hearth/adapters/ · src/hearth/cost.py

Record the summary, not the prompt. A SQLite ledger stores token counts, cost, and break counts. A JSONL trace stores fingerprints. Prompt and response text are stored only when record=True. An OpenTelemetry span can carry the same counts.src/hearth/ledger.py · src/hearth/trace.py · src/hearth/otel.py

Five places a piece of text is allowed to live

This is the vocabulary the rest of the library uses. The class is a rule about when the text may change.

Class	What belongs there	The rule
STATIC	Tool definitions. Core instructions.	Set once for the session. Any change is a prefix break.
SESSION	Facts pinned when the session starts. Project rules. A user profile.	May change between sessions. A mid-session update goes to the tail, so the cached front stays put.
APPEND	The conversation so far.	Add to the end. Do not rewrite an earlier message, except through a recorded compaction.
VOLATILE	The clock, a git status, a snippet retrieved for this turn.	Always after the last cached position.
TURN	The new user input or the new tool result.	Last among the conversation messages.

In code these are StabilityClass and Segment in src/hearth/segments.py. A fingerprint is a deterministic hash of a message, so two processes agree on whether two prompts are the same without storing the text. That lives in src/hearth/fingerprint.py.

The pieces, named

Each name below is a real object in the tree. Together they are the whole library. There are 39 Python modules under src/hearth/.

wrap, session, volatile

src/hearth/__init__.py

wrap returns the proxy. session is a context manager that gives the call a name and a place to hang hints: expect_pause() for a human review, expect_calls(n) for how many steps you think remain. volatile(text) marks text that must ride in the tail. The active session is kept in a ContextVar, so concurrent async tasks do not share one session by accident.

Prefix guard

src/hearth/guard.py · BreakEvent

Compares this call with the previous one in a fixed order: too short to cache, idle longer than the lifetime, provider reported a miss on an identical prefix, then parameter changes, tool edits, tool reordering, a volatile pattern in the prefix, a system-prompt edit, a rewritten history. The first match wins. Four causes are defined as never the user’s fault: below_minimum, short_session, ttl_expired, provider_miss.

Cache planner and TTL optimizer

src/hearth/planner.py · CachePlan, Breakpoint, TTLOptimizer

A breakpoint is a mark that tells Anthropic “cache up to here.” The planner rations them. The optimizer chooses 5 minutes or 1 hour for the stable front. A 5-minute write costs 1.25× the input price. A 1-hour write costs 2×. If a pause longer than five minutes is coming, paying 2× once is cheaper than paying 1.25× again on every resume. The choice is made once per session, because changing the lifetime on an identical prefix is treated as a fresh write.

Provider adapters and the price table

src/hearth/adapters/ · src/hearth/cost.py · src/hearth/data/providers.toml

Anthropic, OpenAI, and Gemini report cache usage in different fields. Each adapter turns that report into read, write, and uncached, and those three must equal the provider’s total input. Prices, minimum prefix lengths, and lifetimes live only in providers.toml, each with a source URL and the date 2026-10-19. Nothing in the Python hard-codes a price.

Ledger and trace

src/hearth/ledger.py · src/hearth/trace.py

The ledger is SQLite: one row per call, with counts and a dollar cost, and no prompt text. The trace is JSONL for later replay. Bodies are omitted unless recording was turned on. That split is how you can debug a cache without copying customer prompts into a log.

Layout, offload, compaction

src/hearth/layout.py · src/hearth/offload.py · src/hearth/compaction.py

Layout is the timestamp move. Offload replaces a large tool result with a hash stub, and the application can read the original back through hearth.tools.read_offloaded. Compaction is the decision to summarize old history. Summarizing is not free: you pay to write the summary, and you pay a cache-write on the new prefix. The model computes a payback horizon. On the bench study, a 50,000-token history pays back only if about eight more calls remain. Below that, trimming costs more than it saves.

Memory

src/hearth/memory/store.py · checks.py · recall.py

Facts live in SQLite, default ~/.hearth/memory.sqlite, with full-text search. A fact is never overwritten in place. When it changes, the old row gets valid_to and superseded_by, and a new row starts. Writes are checked first: length, secrets, and phrases that look like “ignore previous instructions.” Facts pinned at session start go into the SESSION prefix. A fact written in the middle of a session goes to that turn’s tail, so it cannot invalidate the cache that is already warm.

Switch or stay

src/hearth/policies/route.py · RouteDecision

A cheaper model is not automatically cheaper mid-session. Switching means the new model must write the whole prefix cold. The library computes the number of remaining calls at which the switch pays for itself: if model B costs fraction q of model A, the threshold is q · w / (r · (1 − q)). Below that, the verdict is stay. A third verdict, delegate, runs a small sub-task on a cheap model with only the tokens that sub-task needs, and leaves the main prefix cached on the expensive model.

Simulator, fake provider, bench, replay, guard

src/hearth/sim/ · src/hearth/testing.py · src/hearth/replay.py · src/hearth/cli.py

FakeProvider returns canned SDK-shaped responses and invents usage numbers from the same rules as providers.toml, so tests need no API key. The simulator replays a trace and predicts read share. hearth replay runs a trace through observe, safe, and optimize and prints cost. hearth guard fails a CI run when a new user-caused break appears against a baseline. hearth report and hearth diagnose read a session and explain the cause.

Framework adapters

src/hearth/integrations/

LangGraph gets a middleware class. Pydantic AI gets a wrapper around the client the model object already holds. The OpenAI Agents SDK gets its default client replaced, so agents created afterwards inherit the wrap. The core package does not depend on any of those frameworks. You install the framework yourself.

Why a cache dies

The cause list is data, in src/hearth/data/breaks.yaml, and the human copy is docs/BREAKS.md. Fifteen codes. The grouping that matters for reasoning:

The bill, not a bug in your agent

The prompt is shorter than the provider’s minimum. The session was one call, so a cache write had nothing to read. The gap between calls was longer than the lifetime. The prefix was identical and the provider still reported zero reads.

Something in the application moved

Model or parameters changed. A tool was added, removed, or edited. The same tools arrived in a new order. A clock or id sat in the prefix. The system prompt was edited. History was rewritten. A session fact was injected into the front mid-conversation.

Safe mode repairs the reorder case. Optimize mode repairs the clock-in-the-prefix case and the oversized tool-result case. The other application causes are diagnosed and explained. Hearth does not silently rewrite history to hide them.

What has been built

The plan was eight milestones over ten weeks, with a kill test first. All eight were recorded on one day, 19 October 2026. The code for each milestone is in the tree and the local test run that day was green: 56 tests, 16 named invariants, strict type check clean, no required third-party packages on Python 3.11 and newer.

	Capability that exists	Recorded result
M0	Observe mode, traces, decimal costs, 20 canned tasks, a simulator kill test.	Marked GO. Arm D in the simulator: about 49% read share and 43% lower cost than a typical stock prompt that puts a timestamp first.
M1	Safe wrap, guard, planner, three adapters, ledger, report, diagnose, OpenTelemetry.	Invariants for identity, append-only history, usage reconciliation, fail-open, and money-as-decimal are present.
M2	Optimize layout, offload, compaction payback, the break catalogue.	Simulator: optimize cheaper than safe, 20 of 20 tasks passed. Compaction mean payback about 8 further calls.
M3	LangGraph, Pydantic AI, OpenAI Agents hooks.	Mini-suites recorded at 100% pass, offline.
M4	Replay, CI guard, fake provider, frozen report.	`docs/REPORT.md` says 42.0% cost reduction across 20 tasks.
M5	Temporal facts, write checks, cache-safe placement.	Memory tasks recorded at 100% pass.
M6	Stay / switch / delegate, and the 1-hour TTL choice. Version set to `0.1.0b1`.	Simulator: 18.1% savings on 5 mixed tasks, none lost.
M7	License, README, a latency microbench, providers file re-dated.	Docs are done. Release is waiting. See the last section.

Overhead measured in-process, with garbage collection paused for the timing loop: median 0.66 ms and 99th percentile 1.08 ms at about 50,000 tokens of context. The budget was 2 ms and 5 ms. That number is CPU time inside Hearth, not time-to-first-token from a provider.

How it was made

Language and runtime: Python 3.10 and newer. Packaging with hatchling. Day-to-day tool is uv. Checks are ruff, mypy --strict on src/hearth, and pytest, wrapped as make check.
Dependencies: the core imports the standard library. On 3.10 only, tomli reads the price file, because tomllib arrived in 3.11. Optional extras exist for the Anthropic, OpenAI, and Gemini SDKs, and for OpenTelemetry. The framework integrations do not pull those frameworks into the core install.
Specification: docs/BUILD_HANDBOOK.md is the spec. docs/STATUS.md is the log. docs/DECISIONS.md is the decision list. Provider facts are data, not code.
Correctness is a list of named tests. Observe must be byte-identical. Safe must not change content. Money must be Decimal. A miss caused by expiry, length, or the provider must not be blamed on the user. Those rules are what the 16 invariants lock.
The build was compressed. The handbook’s calendar is ten weeks. The git history from scaffold to the release-gate note is a single day of commits. The structure matches the handbook. The calendar does not.

What it can do today

On a machine with the repo, or once the package is installed from it:

Wrap a Gemini, OpenAI, or Anthropic client and leave the request untouched while recording cache accounting.
Wrap it in safe mode and have tool order restored when the same tools come back shuffled.
Wrap it in optimize mode and have timestamp-like lines moved out of the system prompt.
Ask, after a session, what broke the prefix and what it cost, with hearth report and hearth diagnose.
Replay a trace offline through the three modes.
Fail a test run when a new user-caused break shows up, with hearth guard.
Store facts that supersede instead of overwrite, and that refuse secrets and instruction-override text.
Ask whether staying, switching, or delegating is the cheaper next step, given the price table and the remaining calls.
Drop the same wrap into LangGraph, Pydantic AI, or the OpenAI Agents SDK without those libraries becoming dependencies of Hearth itself.

How to read the numbers

The 42% cost reduction, the 49% read share, the 18% routing saving, and the 0.00 calibration error are outputs of Hearth’s own simulator and fake provider. docs/BENCH.md records $0.00 of provider spend. No live kill test has been logged. The calibration error of zero is the simulator agreeing with itself.

The report file also prints the same read share and the same dollar figures for every framework row. That is what you get when one layout scenario is scored four times, not when four real agents were run on two companies’ APIs. The handbook’s kill test was three live runs per task per arm, on named models, with the spend written down. That run is the measurement that would turn the 42% into a claim.

The local tests are a different kind of evidence, and they are real: the identity rules, the usage arithmetic, the fail-open path, and the decimal costs were executed on 19 October and passed. They prove the library behaves as specified against fixtures. They do not prove a provider bill moved.

Can we continue

Yes. The library is far enough along that the next progress is proof and publication, not a new subsystem. The handbook’s release checklist, section 12, is the list. Here is where each line stands on 20 October 2026.

Release line	Standing
Invariant tests green	Green locally on 19 October. There is no GitHub Actions workflow, so GitHub is not re-running them.
Five nights of live contract tests	Not started. The contract target exists in the Makefile. The spend log is empty. Approved budget for this is $25.
Report reproduced from the bench command	The command exists and the file is frozen. It reproduces the simulator. It does not yet reproduce a live bill.
Three other people install `0.1.0b1`, wrap a real agent, and get a report	Not done. `gtm/TRIALS.md` does not exist.
Price table dated within 14 days of release	Dated 19 October 2026. Inside the window if a release happens this month.
PyPI, tag `v0.1.0`, your go	Not published. Trusted publishing is not configured. The version in the tree is `0.1.0b1`. You have not approved the tag.

Continuing means four pieces of work, in this order. Run the live kill test inside the $25 budget and write the token totals into docs/BENCH.md. Run the nightly contract test for five nights. Have three people outside the build try 0.1.0b1 and keep their notes. Then decide the tag and the PyPI upload with those notes in front of you. The code does not need another milestone before that decision.

After a real 0.1.0, the handbook’s own next item is a TypeScript port. It is explicitly after this release. Memory, routing, and the guard are already in the Python tree, so further product work is refinement of those, or the live evidence they still lack.