Caching and tracing

Caching answers: cache=True

cache=True saves each answer on disk and reuses it when the same inputs come again, so the model is asked once.

@thunc.function(cache=True)
def category(ticket: str) -> Literal["bug", "billing", "other"]:
    """Classify this support ticket."""
    ...

It's off by default, because it only suits some functions:

• Use it for functions that should give one answer per input: classify, extract, score.
• Don't use it for functions meant to vary (drafting a reply, brainstorming), or whose answer depends on something that isn't an input, like today's date. Make that an input instead, def overdue(deadline: date, today: date) -> bool, and caching becomes safe.

When a saved answer is reused

Only for the exact same function, prompt, backend and model, so changing the docstring, the return type or the model asks again. A saved answer is checked against the return type and ensure= before it's reused, and failed calls are never saved. The function's name is part of the key, so renaming a function starts its cache fresh.

Where answers go

In .thunc_cache/ in the working directory; change it with configure(cache_dir=...) or THUNC_CACHE_DIR. Each call is one JSON file holding the full prompt in plain text, inputs included, so treat the folder like the data you send.

Clearing the cache

Clear everything, or one function's answers, from Python:

thunc.clear_cache()                                # everything
thunc.clear_cache(urgency)                         # one function
thunc.clear_cache("urgency")                       # the same, by name
thunc.clear_cache(older_than=timedelta(days=30))   # answers saved more than 30 days ago
thunc.cache_info()                                 # what's saved, one group per function

Or from the command line:

thunc cache list                                   # saved answers per function
thunc cache clear                                  # everything
thunc cache clear --function urgency               # one function (repeat for several)
thunc cache clear --older-than 30d --dry-run       # what would go, without deleting

A name is the function's name (urgency, or Triage.urgency for a method), optionally with its module (support_inbox.urgency). For thunc.call, pass name="..." to group its answers the same way; unnamed calls are cleared only with everything or by age. Ages count from when the answer was saved. clear_cache returns how many answers it deleted.

Clearing deletes only cache entries, never other files in the folder, and it's safe while another process is using the cache. The thunc command (also python -m thunc) reads THUNC_CACHE_DIR, or takes --cache-dir; it can't see a configure(cache_dir=...) in your code.

Tracing every call

thunc.configure(trace="calls.jsonl")

Every call is appended to the file as one JSON line, or set THUNC_TRACE instead. An agent run is one line too, with every model reply in it. log_triage.py and repo_guide.py read their traces back.

Testing code that calls a model

There's no record/replay switch for tests yet: cache=True is per function, and nothing serves every call from disk and fails on a miss. thunc's own offline tests use a fake backend instead of a real model.

Edit this page on GitHub