Eleven days of thunc

The releases

1. 3 Oct0.1.0 First release: typed functions on Claude, Claude Code and Codex
2. 3 Oct0.1.1 The OpenAI backend and local models
3. 4 Oct0.1.2 cache=True, system= and a parser hardened over twelve rounds
4. 4 Oct0.1.3 The Jev backend
5. 4 Oct0.2.0 Agents
6. 4 Oct0.2.1 Durable agents with Temporal
7. 5 Oct0.2.2 Native tool calls on Claude Code, after a benchmark
8. 6 Oct0.2.3 Native tool calls everywhere agents run; the last of 0.2
9. 7 Octthunc-watch 0.1.0 A live dashboard in the terminal
10. 7 Oct0.3.0 thunc watch, and thunc write as an experiment
11. 7 Oct0.3.1 The changes that missed 0.3.0's package
12. 10 Oct0.3.2 A stability pass
13. 11 Oct0.4.0 Enum returns, the async API and record and replay
14. 12 Oct0.5.0 thunc.testing and your own backends
15. 13 Oct0.6.0 The agent API frozen; versioned formats; durable fingerprints that survive upgrades

Each release has its full notes in the changelog. This post is the story around them.

A function that is a prompt

The idea behind thunc is that calling a model from code shouldn't need a prompt template, a JSON schema and a page of parsing code. You write the function you want. The docstring is the prompt, the parameters are the inputs, and the return type is the contract: thunc checks the reply against it, sends a wrong answer back to the model to try again, and raises a ThuncError if it still doesn't fit.

import thunc


@thunc.function
def urgency(ticket: str) -> int:
    """Rate how urgent this ticket is,
    from 1 (can wait) to 5 (customer is blocked)."""
    ...


urgency("I was charged twice!")  # 4, a checked int

The name is think + function. Two constraints were set on the first day and haven't moved since: thunc has no dependencies beyond the standard library, and it runs on whatever you already have. 0.1.0 shipped with three backends, the Claude API and the Claude Code and Codex command-line tools, so anyone logged in to one of those could try it without an API key. It also had thunc.call for prompts built in code, thunc.map to run many calls at once, ensure= for your own checks, and JSONL tracing.

The first commit with code landed at 13:33. 0.1.0 was on PyPI at 13:49, through PyPI's Trusted Publishing: creating a GitHub release builds and uploads the package, with no tokens to manage. By the end of the afternoon 0.1.1 added an OpenAI backend on the Responses API, and with it local models through OPENAI_BASE_URL, tested with LM Studio and gpt-oss-20b. The same afternoon added a CONTRIBUTING guide, GitHub Discussions and this website.

Twelve rounds of trying to break the parser

Models don't always follow the contract. They wrap JSON in a code fence, open with a <think> block, answer {"rating": 4} when asked for an int, or reply NaN. The parser has to read the harmless near-misses, send the ambiguous ones back, and never return a wrong value as if it were right.

To find out where it failed, independent agents attacked it in rounds. Each round wrote 25 to 37 new tests against the contract, offline with scripted backends, on Python 3.10 and 3.11. Every finding was fixed with a regression test or written down as a known issue. Rounds 11 and 12 were regression hunts that compared the branch with main over more than 150,000 (reply, type) pairs. Against a fixed set of 102 realistic bad replies:

The last rounds taught the most useful lesson. They mostly found bugs in the branch's own cleverest rules: an exactness check for whole-number floats, unwrapping under a field name, an elaborate rule for a code fence's opening line. Instead of patching them again, the final commit removed them. The test suite went from 61 tests to 205, and only ThuncError can now escape a call because of a bad reply. The same release added cache=True, which asks the model once per input and keeps the answer on disk, and system= for your own system prompt.

Agents that hand back a typed answer

The morning's 0.1.3 added a backend for Jev, a small judgment model that answers yes/no, labels and ratings in about 0.3 seconds. It also made the Codex backend ignore your own Codex config and tools, so a call behaves the same on every machine.

Then the big one. A function answers from what you pass it; an agent can look around first. 0.2.0 introduced thunc.Agent: a typed task, written like a function, that lists, searches and reads files in a working directory and, where its permissions allow, edits them and runs commands, before it finishes with a checked value of its return type.

fixer = thunc.Agent(
    "fixer",
    workdir=".",
    permissions=["write:src/**", "run:pytest"],
)


@fixer.task
def fix_failing_tests() -> bool:
    """Run the tests and fix any problems that they surface."""
    ...

The pieces landed as seven stacked pull requests and merged within two minutes of each other:

• Permissions as rules with globs: write:, read:, run: and ! to deny. Commands run without a shell and with a minimal environment.
• A record of every run: agent.run returns the value with the files changed, the commands run and each step, and an AgentError carries the partial record when a run fails.
• follow= gives the agent your AGENTS.md or CLAUDE.md as instructions, and agents keep notes between runs in a memory file.
• Native tool calls on the Claude and OpenAI APIs, with the fixed part of the prompt cached. On the command-line backends, agents used a text protocol instead: one JSON action per reply.
• Your own functions as tools with tools=, and system prompt presets for coding, code review and analysis.

The agent's default system prompt was a deliberate choice. Copying the prompts of Claude Code or Codex was ruled out early: their tools differ from thunc's, they assume a person at the keyboard, and they change between versions. thunc has a short prompt of its own, and live_tests/eval_prompts.py compares no prompt, the default and each preset on fixture repositories; 90 of 90 runs passed on the Claude API and Claude Code. CI started running the offline tests on Windows the same day.

Durable agents, the same afternoon

An agent that edits files and runs commands for minutes at a time shouldn't lose its work when a process dies. 0.2.1 added an optional runtime on Temporal, installed with thunc[temporal]. Each model turn and tool call is recorded in a workflow, so a run survives a worker restart and can be picked up from another process. File writes go through an intent and a receipt, so a retried step doesn't write twice, and a command that may or may not have run before a crash waits for you to say what happened with resolve() rather than guessing.

To make that possible, the agent loop's decisions moved into one engine shared by local and durable runs. Local thunc stays dependency-free; the Temporal SDK is only needed by those who ask for it. The evening went to the README (quickstart first, with demo GIFs) and a new look for this site, with the docs section you can read today.

Measuring thunc's own cost

Before tuning anything, thunc got two ways to measure itself. thunc run --profile app.py runs a program and reports where its time went: per function, calls, cache hits, retries and model time against thunc's own; per agent task, steps and time in each tool. And benchmarks/ is a suite that times thunc against stand-in models that answer at once, so the numbers are thunc's work, not a model's. It found three speedups:

The benchmark that changed the plan

The next question was harder: when a thunc agent fails or runs slowly, how much of that is thunc rather than the model? The tool-use benchmark (live_tests/bench_tooluse.py) answers it with eight small repositories, each pressing on one part of a harness: a value hidden two hops from its call site behind a stale build copy, a bug near line 1,900 of a 2,400-line file, a rename across 14 files, a module that is mostly backslashes and quotes, tab-indented near-duplicates, one real error in 4,000 lines of build output, tests that only pass from a subfolder, and a structured answer gathered from four files with two decoys. The same model ran every task through thunc and through Claude Code itself.

The answer was uncomfortable. On Claude Sonnet 5.5, thunc agents on the Claude Code backend passed 12 of 24 runs. Claude Code passed 24 of 24. The model was the same in every column; only the harness changed.

The report traced most of the gap to the text protocol. Current models are trained to call tools natively, and asked instead to write one JSON action as plain text, they slip back. They wrote a correct action and kept going, making up the tool's result and the next steps. One failed model call ended a whole run. search and list waded through what git ignores, and run had no working directory and kept only the end of long output, where the first error wasn't.

The fix was to stop fighting the model. On Claude Code, thunc now serves the agent's tools as an MCP server that one claude -p process per run calls natively; a small relay hands each call back to thunc, which carries it out with its own tools, permissions and records. Failed steps are retried. The tool gaps closed. And the Claude Code backend now loads none of your Claude Code settings, so files in an agent's working directory can't give it instructions.

Same model, same tasks: thunc agents now pass every run, slightly faster than Claude Code and at about a third of its cost. That shipped as 0.2.2 on the morning of 5 October, together with thunc run --profile.

Closing out 0.2

The benchmark left a list of open items, and 0.2.3 was scoped to finish all of them before anything new: eight workstreams, one pull request each, stacked on one another. A few worth telling:

• Agents renamed symbols by writing throwaway scripts. Faced with 26 separate edits, models wrote a Python script to do the rename instead. edit can now replace every occurrence or make several changes in one call.
• edit no longer needs a prior read. With a shell, agents read files with cat, and edit refused them 7 times in 24 runs. It only changes text the agent quotes exactly, so it can't overwrite what the agent hasn't seen.
• The text protocol reads messy replies. A quarter of Sonnet's text-protocol replies had been sent back as invalid with a correct action inside them. Now the first complete action is used, and the model is told to send the JSON alone next time. One run had batched [search, read, finish 0.0], a guess written before its own read came back, so finish alongside other calls is now refused.
• A reply that stalled for an hour. The first live run on the Claude API hung on one Opus reply: the stream sent nothing, while the API's keep-alive pings kept the SDK's read timeout from firing. A reply that sends nothing for timeout seconds is now stopped and asked again. Replies are streamed with room for large writes, max_tokens no longer ends a run, and agents think at effort high by default.
• Native calls on Codex, through the same relay as Claude Code: 30 seconds a task against 45 for the text protocol. The third round of runs hit the Codex subscription's usage limit, so that table has two runs per task instead of three.
• Durable runs with native calls on Claude Code. This one was gated on a spike: kill Claude Code in the middle of a tool call, then resume its session. It worked, with a twist. Claude Code marks the interrupted call as having an unknown outcome and the model asks for it again, so the durable runtime maps that repeated call to the same journal entry: replayed if it finished, held for resolve() if nobody knows.

In the rerun, every harness passed 24 of 24, the text protocol included (20 of 24 in 0.2.2) in about half its previous time, and through the Claude API both Sonnet 5.5 and Opus 5.5 passed 8 of 8. 0.2.3 shipped on 6 October as the last of the 0.2 line.

Watching a program

A program that makes dozens of calls and runs agents is hard to follow from its logs. thunc watch app.py runs it with a dashboard in the terminal: the calls waiting on a model, retries and why each reply was rejected, timings for each function, each agent's steps as they happen, and the profile report when the program ends. --agents follows agent runs from any process, and --plain prints one line per event for CI.

The dashboard is written in Rust with ratatui, which raised a packaging question. Bundling a compiled binary would have turned thunc into a set of platform wheels and broken installs where no wheel exists. So the dashboard ships as its own package, thunc-watch, installed with pip install "thunc[watch]", and thunc stays pure Python. thunc only writes one JSON line per event to the file named by THUNC_EVENTS, and nothing at all when it isn't set. See Watching a program.

Functions that write themselves

The other 0.3 feature started from a simple question: if the model can answer a function's calls, why not have it write the function? With @thunc.function(write=True), the first call asks for three things side by side: this call's answer, a draft of the body from the docstring and signature, and five test calls from a separate request, each answered by the model. The draft is linted, run on all six inputs and must match every answer. A passing draft goes into your file in place of ..., the decorator is removed, the checked calls become doctest examples, and from then on the function is plain Python with no model calls.

thunc: writing minutes() in durations.py (first call)
thunc: checked against 6 model answers: all agree
thunc: wrote durations.py lines 5-27 in 14s (answer 4s, draft 11s, test calls 9s; side by side). Removed @thunc.function. Review: git diff durations.py

It took two tries to build the right thing. The first attempt, a day earlier, built something else: hand-written function bodies that fall back to the model when they're unsure. It was a reasonable feature but not the one that had been asked for, so it was reverted the same day, unreleased. The second attempt started by writing down which idea it delivered. Its first version used a whole agent as the writer and took about 39 seconds on Codex; replacing it with one typed call, and asking for the test calls separately so they don't share the draft's blind spots, brought that to 20.

thunc write edits your source files, so it ships as an experiment. It's refused in CI, in installed code, in files outside the project and in files changed since they were imported, and every change is a diff to review. See thunc write.

A stability pass

After a few days of new features, a review of 0.3.1 looked for ways a run could fail that it shouldn't. It found two.

When the Claude API sends an error in the middle of a streamed reply, say an overloaded server, the SDK raises it with the stream's HTTP status, 200. thunc read that as permanent and the agent run failed, though the docs promise such errors are retried. Now the error's type decides. In a simulation of 10-step runs with real thunc agents and a scripted client:

The second was a trace file that couldn't be written, in a missing folder for example. The error replaced the call's outcome: a successful call lost its answer, already paid for, and a failed one hid its own error. Now thunc warns once per path and the call's result stands.

Handing work to a cheaper model

The benchmark had shown a thunc agent on Sonnet doing the same work as Claude Code for much less. That raised a practical question: could Claude Code itself hand work to one? The answer is thunc-offload, a skill you can download from the site. It tells Claude Code to give long, self-contained jobs with a checkable outcome (a wide refactor, a debugging hunt, a feature with tests) to a thunc agent on Sonnet, then read the diff and run the tests itself; to run batch work, like classifying hundreds of log lines, as typed functions with thunc.map, so it stays out of the conversation; and to do small tasks itself, because a handoff costs three or four turns of its own.

The first number was 78%: in the tool-use benchmark's Opus 5.5 rerun, a thunc agent cost $0.039 a task against $0.174 for Claude Code, both passing 24 of 24. But that compares the agent with Claude Code, each working alone. Handing off still costs the main session its turns to explain the job and check the result, so a second test counted both. Claude Code on Opus 5.5 built a multi-module feature from a spec, graded by 24 hidden tests, three runs per setup:

That's 21% cheaper overall at the same pass rate, and every handoff run cost less than every run of Claude Code alone. On small three-file fixes it broke even. The page now leads with 21%, and keeps 78% labelled as the agent's own saving. The test also caught a bug in the skill: it told Claude Code to start long agent runs in the background, which in headless claude -p ends the turn and cuts the agent off. Two runs passed 6 and 0 of 24 hidden tests that way. One more finding: in every test, Claude Code didn't hand off unless asked, so the page tells you to ask. See Claude Code skill.

Functions complete, and tests without a model

0.4 went back to the issue tracker and closed three of the five open feature requests. A function can return an Enum and gets the member back (#6). Values are matched like a Literal's options; the prompt lists them, with the name beside a value that doesn't say what it means (3 (HIGH)); and a name sent in place of a value is asked again, not guessed. thunc.call(..., returns=Literal["a", "b"]) is now a Literal["a", "b"] to your type checker instead of Any (#10), through PEP 747's TypeForm, imported for type checkers only, so thunc still has no dependencies at run time. A test runs mypy and basedpyright over every public signature, and fails both when TypeForm is swapped back. And thunc.acall and thunc.amap are the async API, with calls run in worker threads so the event loop stays free.

The biggest piece was record and replay (#8). Code that calls a model is hard to test: a real model is slow, costs money, needs a login in CI and doesn't answer the same way twice. Now you record a test suite's answers once, commit them, and replay them from then on:

THUNC_RECORD=1 pytest          # ask the model, and save every answer in tests/recordings
pytest                         # replay: no model is asked, and a call that wasn't recorded fails
THUNC_RECORD=missing pytest    # replay what's there; ask for, and save, only what's missing

A call is matched on its function, system prompt and request; the backend and model are saved but not matched, so CI needs neither a backend nor an API key. A replayed answer is still checked against the return type and ensure=. Agent runs replay the model's replies in order while their tools run for real, so a run that goes differently runs out of replies and fails rather than passing by accident. While replaying, nothing at all reaches a model: every place thunc would start a request or a CLI checks first, so a durable run, which isn't recorded, fails loudly instead of spending money. Replaying a call takes about 37 µs, the same as a cache hit, because recordings use the cache's format.

0.4 also put a deprecation policy in place before anything needs it. A part of the API that's going away keeps working for at least two minor releases and warns where it's used, naming the release that removes it, and a test fails once a deprecation's removal release arrives. Nothing is deprecated yet; it's there for the changes ahead of 1.0.

Testable and extensible

Recordings suit tests where the answer should be a real model's. For unit tests where the test decides the answer, 0.5.0 adds thunc.testing: a scripted model that answers every call and agent run, whatever backend= they name, and keeps every request for the test to check.

from thunc import testing


# triage() is your code; it calls two thunc functions, urgency() and category()
def test_triage():
    with testing.fake_model({"urgency": 4, "category": "bug"}) as model:
        assert triage("I was charged twice") == ("bug", 4)
    assert model.calls[0].inputs == {"ticket": "I was charged twice"}

Replies go by function name, in order, or come from a function of the request. A string is sent as the model's raw text, so a bad one makes the call retry as a real one would; anything else is sent as JSON, so 4, Priority.HIGH or a dataclass work as written. act() and finish() script an agent's tool calls. The thunc_model and thunc_isolated pytest fixtures come with thunc. The test for them runs a separate pytest, so it proves the fixtures load the way they will for you, and it was checked by breaking the fixture's cleanup on purpose and watching it fail.

The same release opened thunc to backends it doesn't ship. thunc.register_backend(name, fn) takes a function that turns a prompt into text, or, with typed=True, one that answers typed questions as Jev does, and makes it usable anywhere a built-in backend name is. Its signature is checked when it's registered, not at its first call. Writing the tests for it turned up a bug that no built-in backend could reach: a typed backend without a default model failed every call with a KeyError. The docs now also cover Ollama, which serves the same Responses API as the openai backend, with a live test in live_tests/test_ollama.py. That test hasn't been run for this release yet, and the release notes say so.

Agents settle

0.6 adds little you can call. It's about what stops changing. The agent API is frozen: thunc.Agent and its methods, @thunc.agent, thunc.Run and thunc.AgentError change from here on only through the deprecation policy 0.4 put in place. A freeze is only as good as what enforces it, so tests/api.txt now lists every public signature, the agent API and the rest of thunc.__all__ alike, and a test fails on any difference. The file is written the same way on Python 3.10, 3.13 and 3.14, so the test means the same thing on every Python CI runs. Two small additions went in before the freeze: agent.acall, to match thunc.acall, and protocol= and effort= typed as their choices, so a type checker catches a typo the run would refuse anyway.

Three things other programs read got a written promise. An agent run's record now carries a format number on its first line, as the THUNC_EVENTS stream always has, and within one number both only gain events and fields. A test checks what thunc writes against the fields the docs list, and fails on one they don't; it was checked by adding one. The third is durable task fingerprints. A worker refuses to continue a run whose task's fingerprint changed, so a thunc upgrade that changed fingerprints would strand every run in flight. Upgrading thunc now never changes the fingerprint of a task registered the same way, and a test pins four task shapes. Before writing that promise down, the fingerprints were computed again with 0.3.2's own code: they were the same, so none of 0.4 to 0.6 had broken it.

The release also closed one of the tool-use benchmark's open questions. On deep_fix, a bug in a 2,000-line file, a thunc agent takes 8 turns to Claude Code's 5, reading the file in 400-line pages. Would a bigger default read help? On Opus 5.5, three runs each:

Only the whole file halves the turns, and it doubles the cost, because it stays in the context for every turn after. thunc's edge on these tasks is cost, so 400 stays, and a model that wants more can ask for it. Two other open questions stay open, on purpose. On native Codex runs, steps count tool calls rather than model replies, and no tokens are reported: codex exec --json marks no model reply and reports tokens only when a turn ends, which a run that ends on finish never reaches. Fixing either would mean reading Codex's undocumented session files, and a guess there could break a path that works today. And thunc write stays experimental through 1.0, outside the freeze, until there's more use to judge it by.

What we learned

• Measure the harness, not only the model. The biggest improvement in these eleven days, 12 of 24 to 24 of 24, didn't come from a better model. It came from a benchmark that held the model fixed and asked what thunc was doing wrong.
• Every change comes with a number. Each release note says what was measured and how: runs passed, seconds and dollars per task, runs that survive an error rate. It keeps claims honest and shows which ideas didn't pay off.
• Attack your own code, then simplify it. Adversarial rounds found real bugs, and then found bugs in the fixes. Removing the clever rules did more than patching them again.
• Releasing is code too. Two releases went out from the wrong commit. Both times the fix was a written-down order of steps, not more care.
• Count the whole cost. The skill's first number, 78%, was true but compared the wrong things. Counting the session that hands off the work and checks it gave 21%, which is the number you'd actually see.
• Keep the core small. Temporal and the dashboard are real dependencies, so they're an extra and a separate package, and the precise types for returns= are imported for type checkers only. pip install thunc still installs nothing else.
• A promise needs a test that can break it. The frozen API, the record's format and the fingerprints each have one, and each was checked by breaking the promise on purpose. The fingerprints were also checked against the oldest code that set them.
• CI is a different machine. 0.4's first CI run failed on a type check that passed locally: in GitHub Actions, basedpyright ignores the option that hides warnings. Setting GITHUB_ACTIONS=true locally reproduced it, and the fix was a config of the check's own.
• Say which idea you're building. One day's work went to the wrong feature because the plan didn't say plainly what it would and wouldn't deliver.

What's next

thunc is in beta, and with 0.6 the agent API is frozen for 1.0. What's left on the way is your feedback: 0.7 to 0.9 are for the API changes that come from people using thunc, made through deprecations, then a security review, a stability page and an upgrade guide, and a release candidate in December with only bug fixes. Two of the original feature requests are still open on the issue tracker: Pydantic models as return types (#7), which a contributor has taken on, and more examples (#11). The next thunc-watch release will show replayed calls as replayed rather than cached, and skip run records in a format it doesn't know. If you run thunc on Ollama, results from live_tests/test_ollama.py are welcome in an issue. thunc write stays experimental through 1.0; whether it leaves the label after that depends on how it holds up in your code. If you try it, or anything else here, tell us how it went.

Edit this page on GitHub