Prompt caching with LangChain on Amazon Bedrock

LangChain talks to Bedrock through the langchain-aws package, and its agents add caching with a middleware. I captured the exact requests langchain-aws 1.8.1 and LangChain 1.4.3 send while create_agent runs its real agent loop, checked them with CacheCanary, and ran every case on live Bedrock (Claude Sonnet 4.6, us-west-2, October 2026, re-checked October 30).

The short version: caching is off until you add the middleware, and it then works well, even with many tool calls at once. The thing that quietly throws it away is LangChain's own context editing.

The setup I'd use

from langchain.agents import create_agent
from langchain_aws import ChatBedrockConverse
from langchain_aws.middleware.prompt_caching import BedrockPromptCachingMiddleware

agent = create_agent(
    model=ChatBedrockConverse(model="us.anthropic.claude-sonnet-4-6"),
    tools=tools,
    system_prompt=SYSTEM_PROMPT,
    middleware=[BedrockPromptCachingMiddleware()],
)

You don't need cache points of your own on top, and if you clear old tool results, clear them in batches (finding 3).

What I found

1. Caching is off until you add the middleware

A plain ChatBedrockConverse in an agent sends no cache points. On live Bedrock, the same system prompt in two fresh agents:

Outside an agent, pass the same setting per call: model.invoke(messages, cache_control={"type": "ephemeral"}). With the middleware, ChatBedrockConverse puts a cache point after the tool list, after the system prompt, at the end of the newest message and at the end of the message before it.

2. Many tool calls at once: fine since October 2026

Bedrock only looks back about 20 positions from a cache point to find the previous cache entry. When I first tested this on October 7 2026, it counted every tool call and every tool result as its own position. Back then ChatBedrock (the InvokeModel API, one cache point at the end of the newest message) lost the whole cache at 11 tool calls at once, ChatBedrockConverse at 22, and with a cache point of your own on the system message it lost it at 11 again, because the middleware then skips its point on the message before the newest one. By October 28 Bedrock counted a group of tool calls side by side as one position, and the same for their results, as Anthropic documents for its own API. The same requests on live Bedrock, with a long customer history in the first message:

So either model class works. I'd still use ChatBedrockConverse: ChatBedrock sends the system prompt as plain text with no cache point of its own, so if something does change in the conversation, the system prompt is written again too. A step can still lose the cache if it adds more than about 20 other blocks at once, like 22 or more separate text blocks; cachecanary diff counts positions the way Bedrock does now.

3. Context editing rewrites the conversation on every call

ContextEditingMiddleware with ClearToolUsesEdit replaces old tool results with "[cleared]" once the conversation passes a token count (100,000 by default), keeping the newest 3. With its default settings, it does this again on every call without saving it, so each new tool result pushes one more old result out of the newest 3. An earlier part of the history changes on every call, and the cache points that come after it can't find the previous entry. Once it starts, every call writes most of the conversation to the cache again. (Setting clear_at_least stops the rewrites, because the same oldest results get cleared every time, but then it never frees more than that many tokens.)

The fix is to clear in batches, so the history only changes once per batch:

from dataclasses import dataclass

from langchain.agents.middleware import ClearToolUsesEdit, ContextEditingMiddleware
from langchain_core.messages import ToolMessage


@dataclass(slots=True)
class ClearToolUsesInChunks(ClearToolUsesEdit):
    """Clear old tool results in whole batches, not one per call."""

    chunk: int = 10

    def apply(self, messages, *, count_tokens) -> None:
        results = sum(isinstance(m, ToolMessage) for m in messages)
        cleared = max(0, results - self.keep) // self.chunk * self.chunk
        keep, self.keep = self.keep, results - cleared  # whole batches only
        try:
            ClearToolUsesEdit.apply(self, messages, count_tokens=count_tokens)
        finally:
            self.keep = keep


editing = ContextEditingMiddleware(edits=[ClearToolUsesInChunks(chunk=10)])
middleware = [editing, BedrockPromptCachingMiddleware()]

On live Bedrock, an agent making one tool call per step with long tool results, with clearing set to start at the seventh step:

Cost in full-price input tokens: reads count a tenth, writes 1.25.

So the stock edit, meant to save tokens, cost about 4 times as much per step as not clearing at all, while clearing in batches cost a little less. Clearing in batches rewrote the history on 2 calls out of 31 in my offline run, against every call for the stock edit, and the newest tool results are never cleared. Clearing in batches keeps up to keep + chunk - 1 recent results instead of exactly keep, so the model sees a few more. It also works with exclude_tools and clear_tool_inputs.

Smaller things

• SummarizationMiddleware replaces old messages with a summary whenever it runs, which rewrites the conversation once each time. On live Bedrock, each call right after a summary read only the system prompt from the cache, and the calls after it read normally again. The gap between its trigger and keep settings decides how often this happens; keep it wide.
• The older create_react_agent from LangGraph (deprecated in favour of create_agent) doesn't run middleware, and model.bind(cache_control=...) doesn't reach the request through it either: I got no cache points at all. A cache point in the system message caches only the system prompt. Move to create_agent with the middleware.
• With an application inference profile ARN, LangChain needs provider="anthropic" (it raises an error without it). It then looks up the model behind the profile with a GetInferenceProfile call when the model is created, which needs that IAM permission, or you can pass base_model_id="anthropic.claude-sonnet-4-6". Both cached on live Bedrock.
• If you add up cache writes from usage_metadata, read ephemeral_5m_input_tokens and ephemeral_1h_input_tokens. When Bedrock reports the cache lifetime, LangChain puts the writes there and sets cache_creation to 0, on purpose.
• Claude Haiku 4.5 needs at least 4,096 tokens before anything is cached. A 3,850-token prompt cached nothing on live Bedrock, with no error.
• Anything that changes in a @dynamic_prompt system prompt starts a new cache: a date once a day, a time or a request ID on every call. cachecanary lint warns about dates, times, IDs and timestamps.

What LangChain gets right

With ChatBedrockConverse, the middleware gives the tools and the system prompt cache points of their own, so a change later in the conversation never costs you those. The middleware doesn't save cache points into your conversation history or checkpoints, so restored threads stay clean. Structured output with response_format adds its tool to every call, so it doesn't break the cache. It worked with extended thinking turned on, and a cache point in a tool result's content is moved to where Bedrock accepts it.

Check your own agent

Save the exact requests your model sends. This works for ChatBedrockConverse and ChatBedrock, streaming or not; I checked each saved file is identical to what reached Bedrock.

import pathlib, time


def save_requests(model, folder="langchain-requests"):
    """Save every request this LangChain Bedrock model sends, for cachecanary."""
    out = pathlib.Path(folder)
    out.mkdir(exist_ok=True)

    def save(params, **kwargs):
        (out / f"{time.time_ns()}.json").write_bytes(params["body"])

    for op in ("Converse", "ConverseStream",
               "InvokeModel", "InvokeModelWithResponseStream"):
        model.client.meta.events.register(
            f"before-call.bedrock-runtime.{op}", save)


model = ChatBedrockConverse(model="us.anthropic.claude-sonnet-4-6")
save_requests(model)

Run a conversation, then compare two requests in a row with cachecanary diff. These are real outputs on requests LangChain built:

$ cachecanary diff edit-call8.json edit-call9.json --model us.anthropic.claude-sonnet-4-6
history-changed: Earlier conversation history was edited (not just appended). (messages[12].content[0])

$ cachecanary diff date-day1.json date-day2.json --model us.anthropic.claude-sonnet-4-6
system-changed: The system prompt changed inside the cached prefix (look for dates, IDs or per-user text). (system[0])

In plain words: the first pair is the stock context editing clearing one more tool result (finding 3). The second is a date in a dynamic system prompt.

Keep a few saved requests in your repo and run cachecanary lint on them in CI, and a LangChain upgrade that changes your cache points shows up in a pull request instead of on the bill. Setup is on the home page.

How I tested this

langchain-aws 1.8.1, LangChain 1.4.3 and LangGraph 1.2.14 in a clean Python environment. To capture requests, I pointed LangChain at a small local server standing in for Bedrock that speaks the Converse and InvokeModel formats, streaming or not, so create_agent ran its real agent loop and every request body was saved. The settings cases ran fully live on Bedrock (Claude Sonnet 4.6, us-west-2, October 7 2026), and I re-ran every case on October 30 2026 after Bedrock changed how it counts tool calls. A live model won't call exactly 21 tools on request, so for the agent-loop cases I sent the exact requests LangChain built, in order, to live Bedrock; caching depends only on the request. Every case uses made-up prompts with a fresh random tag, so old cache entries couldn't help. One run read nothing at all from the cache, system prompt included, and four reruns of the same case kept it. That pattern fits cross-Region inference: with a us. model ID, two calls can land in different Regions, and AWS notes this can mean more cache writes. The capture and the live checks are scripts in the repo: capture and live checks. Token counts move by a few tokens from run to run because each run uses a new random tag. LangChain changes quickly, so check your own version with the snippet above.

Same tests for other frameworks: LiteLLM and Strands Agents. More on why caching breaks on Bedrock: Five ways Claude prompt caching quietly breaks on Amazon Bedrock.

Want a quick number for your own app? Paste three totals from CloudWatch into the cache checker. It runs in your browser, nothing to install.

Using CacheCanary? I'd love to hear how it's going, good or bad, and whether you'd want the dashboard. Email me at hello@cachecanary.com. I read every message.