Prompt caching with LiteLLM on Amazon Bedrock

LiteLLM can add Bedrock cache points for you. I wanted to know what it really sends, so I captured the exact requests LiteLLM 1.104.0 builds for Bedrock, checked them with CacheCanary, and ran the important cases on live Bedrock (Claude Sonnet 4.6, us-west-2, October 2026, re-checked October 30).

The short version: LiteLLM puts cache points in valid places. The money is lost in where you ask it to put them, and in a few settings it drops without telling you.

The setup I'd use

Two cache points are enough: one on the system prompt and one on the last message, so the conversation point moves forward on every call.

response = litellm.completion(
    model="bedrock/converse/us.anthropic.claude-sonnet-4-6",
    messages=messages,
    tools=tools,
    cache_control_injection_points=[
        {"location": "message", "role": "system"},
        {"location": "message", "index": -1},
    ],
)

I ran a 7-step agent session with it on live Bedrock, with rounds of 12, 20 and 40 parallel tool calls: each step read almost the whole previous prompt from the cache.

Tokens. The last few tokens of each request are always new, so the cache can't hold them.

On October 7 2026 wide tool rounds needed two more points, and this page had a helper for them. Bedrock no longer needs it (finding 2).

If you use the LiteLLM Proxy, the same points go in the request body. With the OpenAI client:

response = client.chat.completions.create(
    model="claude-sonnet",
    messages=messages,
    tools=tools,
    extra_body={"cache_control_injection_points": [
        {"location": "message", "role": "system"},
        {"location": "message", "index": -1},
    ]},
)

What I found

1. role: "user" caches your oldest messages, not your newest

A common setup is one point on the system prompt and one on role: "user". That marks every user message. LiteLLM stops at 4 points, in the order it finds them, so once a chat has more than three user messages, the points sit on the first three and stay there. Everything after them is paid at full price on every call, and that part grows each turn. LiteLLM logs a warning ("Reached the provider limit of 4 cache breakpoints") and sends the request anyway.

On live Bedrock, turn 10 of a chat:

Use index: -1 (the last message) instead of role: "user".

2. Many parallel tool calls: fine since October 2026

With points on the system prompt and the last message, the conversation point moves to the newest tool result on each step. Bedrock only looks back about 20 positions from a cache point for the previous cache entry. When I first tested this on October 7 2026, it counted every tool call and every tool result as its own position, so 12 parallel calls put the previous entry out of reach and the conversation was written to the cache again, at 1.25 times the normal input price instead of a tenth of it. By October 28 Bedrock counted a group of tool calls side by side as one position, and the same for their results, as Anthropic documents for its own API. The same requests on live Bedrock, with a long customer history in the first user message:

A step can still lose the cache if it adds more than about 20 other blocks at once, like 22 or more separate text blocks; cachecanary diff counts positions the way Bedrock does now.

3. A cache point on an assistant tool-call message is dropped

If you add a point on the assistant message that holds the tool calls, and that message has only tool calls and no text, LiteLLM has nothing to attach the point to and sends the request without it. No warning. A point on a tool result does work: LiteLLM turns it into a valid Bedrock cache point after the result.

4. The 1-hour cache lifetime can be dropped without a warning

You can ask for a 1-hour cache with "ttl": "1h" in cache_control, or in an injection point's control. LiteLLM only sends it for models whose entry in its price list says they support it. Otherwise it sends a normal 5-minute cache point. I found two cases where that happens to a model that does support 1 hour:

• Application inference profiles. When the model is a profile ARN (bedrock/converse/arn:aws:bedrock:...), LiteLLM can't tell which model it is, so it drops the 1-hour setting. Name the model and pass the ARN separately, and it's kept:

  # PROFILE_ARN = "arn:aws:bedrock:REGION:ACCOUNT:application-inference-profile/ID"
  litellm.completion(
      model="bedrock/converse/us.anthropic.claude-sonnet-4-6",
      model_id=PROFILE_ARN,
      ...
  )

  On the InvokeModel route (bedrock/invoke/), a profile ARN as the model fails with an error, so model_id is the way to use one there too.
• Claude Sonnet 5.5 with LiteLLM's built-in price list. LiteLLM downloads its price list when it starts. If you set LITELLM_LOCAL_MODEL_COST_MAP=True, or the download fails (it falls back with a "Falling back to local backup" warning), it uses the copy inside the package, and 1.104.0's copy has no Sonnet 5.5. Sonnet 4.6 and Opus 5.5 are in it and keep 1 hour. LiteLLM 1.105.0 (October 11) fixed this: its copy includes Sonnet 5.5, so upgrade if you're on 1.104.

I checked both on live Bedrock. Bedrock's response says which lifetime it used, and in both cases it was 5 minutes. With the fix above (or the downloaded price list, or LiteLLM 1.105.0 for Sonnet 5.5), it was 1 hour. The request still works either way, so the only sign is more cache misses after 5 quiet minutes. To check, look for "ttl": "1h" in the request LiteLLM sends (see below).

Smaller things

• location: "tool_config" puts a point after your tool list. Bedrock caches tools first, so if the tool list alone is shorter than the model's minimum (1,024 tokens on Sonnet 4.6), that point caches nothing and uses up one of your 4.
• A 1-hour point after a 5-minute point is rejected by Bedrock with an error. At least that one is loud. cachecanary lint catches it before it ships.

What LiteLLM gets right

Every cache point LiteLLM sent was in a place Bedrock accepts. A point on a tool result goes after the tool result, not inside it (inside, Bedrock ignores it). The InvokeModel route (bedrock/invoke/) keeps cache_control as it is. The LiteLLM Proxy behaves the same: I ran it with points in config.yaml, with cache_control sent by the client, and with points sent per request, and all three reached Bedrock.

Check your own app

Save the exact requests LiteLLM sends to Bedrock. This callback writes each one to a litellm-requests folder. It works for normal, async and streaming calls; I checked each saved file is identical to what reached Bedrock.

import json, pathlib, time
import litellm
from litellm.integrations.custom_logger import CustomLogger

class SaveBedrockRequests(CustomLogger):
    def log_pre_api_call(self, model, messages, kwargs):
        body = (kwargs.get("additional_args") or {}).get("complete_input_dict")
        if not body:
            return
        if isinstance(body, bytes):
            body = body.decode()
        if not isinstance(body, str):
            body = json.dumps(body, default=str)
        folder = pathlib.Path("litellm-requests")
        folder.mkdir(exist_ok=True)
        (folder / f"{time.time_ns()}.json").write_text(body)

litellm.callbacks = [SaveBedrockRequests()]

Run one conversation, then compare two requests in a row with cachecanary diff. These are real outputs on requests LiteLLM built:

$ cachecanary diff chat-turn6.json chat-turn7.json --model us.anthropic.claude-sonnet-4-6
prefix-identical: The cached prefix is identical. If B still missed, the entry likely expired (TTL elapsed between calls) or cross-region inference routed to a Region without the entry.
checkpoint-not-moved: The cached part was read, but the last cache point didn't move while 8 new blocks were added after it. Those are paid at full price on every call. Put the cache point at the end of the newest message. (messages[4].content[0])

$ cachecanary diff with-download.json without-download.json --model us.anthropic.claude-sonnet-5-5
ttl-changed: Checkpoint TTL changed (1h -> 5m); entries are keyed by TTL.

In plain words: the first pair is the role: "user" chat from finding 1: the cache works, but the newest messages are never added to it. The second is the same Sonnet 5.5 request with and without LiteLLM's price list download (finding 4).

Keep a few saved requests in your repo and run cachecanary lint on them in CI, and a LiteLLM upgrade that changes your cache points shows up in a pull request instead of on the bill. Setup is on the home page.

How I tested this

LiteLLM 1.104.0 in a clean Python environment. To capture requests, I pointed LiteLLM at a small local server standing in for Bedrock that saved every request body. For the live numbers I sent the same requests to Bedrock (Claude Sonnet 4.6, us-west-2, October 7 2026, and again on October 30 2026 after Bedrock changed how it counts tool calls) with made-up prompts and a fresh random tag in each run, so old cache entries couldn't help. The capture and the live checks are scripts in the repo, so you can run them yourself: capture and live checks. I also ran LiteLLM's own proxy server against live Bedrock, called with the OpenAI client as shown above, and got the same numbers as the library. Token counts move by a few tokens from run to run because each run uses a new random tag. On October 12 I captured the same requests again with LiteLLM 1.105.0: findings 1 to 3 and the profile ARN case are unchanged, and the Sonnet 5.5 case is fixed. LiteLLM changes quickly, so check your own version with the callback above.

Using Strands Agents or LangChain instead? Same tests: Strands Agents and LangChain.

More on why caching breaks on Bedrock, with the bug reports behind each case: Five ways Claude prompt caching quietly breaks on Amazon Bedrock.

Want a quick number for your own app? Paste three totals from CloudWatch into the cache checker. It runs in your browser, nothing to install.

Using CacheCanary? I'd love to hear how it's going, good or bad, and whether you'd want the dashboard. Email me at hello@cachecanary.com. I read every message.