Metadata-Version: 2.5
Name: atif-make
Version: 0.8.0
Summary: Convert agent logs (Claude Code, Codex, Copilot, HAR) into ATIF v1.7 trajectories.
License: MIT
Requires-Python: >=3.12
Provides-Extra: parquet
Requires-Dist: pyarrow>=15; extra == 'parquet'
Provides-Extra: spec
Requires-Dist: harbor>=0.21.0; extra == 'spec'
Description-Content-Type: text/markdown

# atif-make

Make [ATIF v1.7](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md)
trajectories from agent logs — Claude Code, Codex, Copilot CLI, HAR captures,
the SLEIGHT-Bench, ATBench and METR MALT datasets, Inspect AI eval logs, and
transcripts published as Markdown rather than as the log they were made from.

Zero runtime dependencies, bar one optional extra for reading Parquet.
Python 3.12+.

## Install

```sh
uv tool install atif-make          # puts `atif-make` on your PATH
```

Or as a library in a project:

```sh
uv add atif-make
```

For a browser view of what you convert, see the companion
[`transcript-viewer`](https://github.com/jammastergirish/transcript-viewer), which depends on this package.

`uv tool install` builds an isolated environment, so nothing lands in your
project or system Python. To follow local edits instead, use
`uv tool install --editable .`; to remove it, `uv tool uninstall atif-make`.

Running from a checkout without installing works too: `uv run atif-make ...`.

```sh
atif-make ~/.claude/projects/my-project/session.jsonl   # convert one log
```

## Commands

```
atif-make <file>                 convert (shorthand for `atif-make convert`)
atif-make convert <file>         convert one log
atif-make convert <dir|archive>  convert every log inside
atif-make convert <dataset>      convert every transcript in one file
atif-make formats                list supported input formats
```

| Flag | Command | Meaning |
|---|---|---|
| `-o, --output` | convert | output path (default `<input>.trajectory.json`) |
| `-f, --format` | convert | force the input format instead of detecting it |
| `--json` | convert | write one self-contained document to stdout |
| `--bundle OUT.zip` | convert | zip the trajectory with its images and subagents |
| `--split-subagents` | convert | write subagents as sibling files, not embedded |
| `--indent N` | convert | JSON indent (default 2) |
| `-q, --quiet` | convert | suppress progress output |

## Supported inputs

Most agents write *two* unrelated log shapes — what the CLI streams, and what it
persists on disk — and they are not interchangeable. atif-make reads both.

| Format | Source |
|---|---|
| `claude-code-transcript` | `~/.claude/projects/<project>/<session>.jsonl` |
| `claude-code-stream` | `claude -p --output-format stream-json` |
| `codex-rollout` | `~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl` |
| `codex-exec` | `codex exec --json` |
| `copilot-cli` | Copilot CLI session logs |
| `sleight-bench` | SLEIGHT-Bench transcripts — one content block per line |
| `atbench` | ATBench — labelled agent trajectories, many to a file |
| `metr-malt` | METR MALT — agent runs recorded as a tree of nodes |
| `transcript-markdown` | A transcript rendered as Markdown for a person to read |
| `inspect-eval` | An Inspect AI `.eval` log — a zip of JSON, one run per sample |
| `har` | Anthropic Messages, OpenAI Chat Completions, OpenAI Responses |
| `atif` | An ATIF trajectory that is already converted |

`sleight-bench` reads the
[SLEIGHT-Bench](https://huggingface.co/datasets/sleightbench/SLEIGHT-Bench)
benchmark, where each line holds a single Anthropic content block rather than a
whole message, so one assistant turn spans several lines. Its transcripts open
with a canary object asking that the data be kept out of training corpora; that
line is skipped as a message and carried into the trajectory's `extra`, so the
opt-out travels with the converted file instead of being dropped at the door.

`atbench` reads [ATBench](https://huggingface.co/datasets/AI45Research/ATBench),
a safety benchmark of tool-using agent trajectories where each one is labelled
for whether the agent did something unsafe. Its two configs differ only in
spelling — `contents`/`id` against `content`/`conv_id` — and both are read.

A row's parts carry three roles: a `user` request, an `agent` turn as a thought
plus an action, and an `environment` reply holding what a tool returned. An
action is either a tool call or the word `Complete` followed by the agent's
final answer, so the ending is a message rather than a call to a tool named
Complete. Thoughts and final answers frequently arrive JSON-encoded a second
time, as a quoted string inside a string; those are unwrapped, but only when
they really do decode, so a thought that merely opens with a quotation mark
keeps it.

The labelling has no home in ATIF's own fields, so it travels in the
trajectory's `extra`: `label`, `risk_source`, `failure_mode`, `real_world_harm`
and `reason`. A tool whose description was tampered with keeps the original
under `_original_description` in the agent's tool definitions — that is the
whole mechanism of an indirect prompt injection, and it is preserved as it
ships.

`metr-malt` reads METR's
[MALT transcripts](https://huggingface.co/datasets/metr-evals/malt-transcripts-public),
where a run is a tree rather than a list: each node carries one message and
points at its parent, and a `branch_id` says which line of the run it belongs
to. Messages are in OpenAI's older chat shape, with `function_call` rather than
`tool_calls`, because these runs date from that scaffold.

Almost all of the tool use here is written *into the message*. The scaffolding
has no structured call: it tells the model in its system prompt to write
`<bash>ls</bash>`, runs what it finds, and hands the output back as an ordinary
user turn wrapped in `<bash-output>`. Across one shard that is 6,961 calls
against 78 in the structured `function_call` field, so reading only the
structured field leaves a transcript showing markup where it should show a
command, and the tool's output looking like something the user said.

The tools are read from the system prompt, which lists them — `bash`, `python`,
`submit`, `timeout`, and rarer ones like `describe_image` — rather than from a
list kept here. These transcripts are full of `<i>`, `<b>` and `<h1>` in
ordinary prose, and a run may use a tool no such list would have known about; a
run whose prompt is missing falls back to the names its own results reveal. A
call left unclosed because the run stopped mid-write is still read as the call
it was reaching for.

Most records hold one conversation with short offshoots hanging off it — an
alternative continuation sampled at some step, sometimes followed by a rater
scoring what it was worth. Those offshoots are the reason the dataset exists:
MALT publishes the same runs five times over, each with the chain of thought
altered a different way, so what branches off a step is what is being compared
against it. Each is kept on the step it branches from, in that step's `extra`,
with the rater's scores where there are any — they arrive in a call's arguments
with nothing in the message at all, so reading only the text would lose them.

A few records hold several *whole* conversations at once: an advisor briefing an
agent, the agent working, and raters scoring it, each with its own system prompt
and its own root, interleaved in one list of nodes. Splicing those into one
would invent a run that never happened, so they are separated by the root each
node climbs to, the longest becomes the trajectory, and the rest travel as
subagent trajectories.

The dataset ships as Parquet. Reading it needs the `parquet` extra
(`uv tool install "atif-make[parquet]"`); nothing else here does, so the package
stays dependency-free for every other format. Splitting a shard is the only step
that needs it — what comes out is ordinary JSON.

`transcript-markdown` reads a transcript that was rendered for a person rather
than kept as a log. Some corpora publish only that: a bucket of 136 of these
prompted the parser, each a half-megabyte document with no JSON anywhere near
it. Two renderings exist and both are read — one writes the assistant at level
three with each argument as a bullet, the other at level two with the arguments
as a JSON object and the tool's answer arriving as a user turn that opens
`**tool_result:**`. Reading that second one naively puts the machine's words in
the person's mouth in every transcript that uses it.

Only three headings are structure. Everything else that looks like one is
content, because these documents are full of writing *about* documents:
`## Hypothesis`, `## TL;DR`, a file being written that has headings of its own.
One held 1,032 headings inside fenced code alone.

Two things in the real corpus cost more than the rest of the format put
together, and both are now fixtures:

* **A fence closes only with one at least as long.** A document quoting a fenced
  block wraps it in four backticks — 1,881 times across 121 of the 136 files.
  Treating every fence alike walks the tracking out of step, and headings inside
  quoted text start being read as structure.
* **A rule drawn with tildes is not a fence.** Markdown allows a tilde fence;
  these documents never use one, and command output is full of `~~~~~~~~` as a
  separator. One such line opened a fence that swallowed the next 1,507 lines of
  its transcript.

Reasoning folded into `<details><summary>Thinking</summary>` is read as
reasoning rather than as speech — 3,260 blocks in that corpus — including where
the rendering never closed the disclosure.

What is lost is what the rendering lost: token counts, timestamps, model names
and tool call ids were never written down. Ids are assigned so a result can be
attached to the call it answers.

`inspect-eval` reads [Inspect AI](https://inspect.aisi.org.uk/) logs, which is
where most agent transcripts in control and evaluation research actually live:
BashArena, control-arena and MonitoringBench all run on it. A `.eval` is a zip
of JSON — `header.json` saying what was run against what model, and a
`samples/` directory holding one run of one task each, with its messages, its
events and its scores.

The messages are the transcript. Inspect gives every call an id and every
result the id it answers, so a call and its result are joined by what the log
says rather than by where they sit. Large content is stored out of line and
referenced as `attachment://<hash>`; those are resolved, because a transcript
that keeps them reads as a list of hashes.

Two things are kept beyond the conversation, because for a control corpus they
are the point of the run: what the scorers said, and what the task was — the
mode (`attack` or `honest`), the main and side task descriptions, the setting.
Token usage is recorded per model for the whole sample rather than per message,
so the totals come from what the log counted, and name the models that did the
work.

A `.eval` is a zip, which makes it look like an archive to anything sniffing by
content. It is not: extracting one gives JSON files that mean nothing on their
own, so it is excluded from archive handling by name.

### A file that holds many transcripts

ATBench is published as one JSON array of a thousand trajectories rather than a
thousand files. That is a container, like a zip, so it is treated as one: the
file is split into a transcript per file and each is converted separately, into
an output directory.

```sh
atif-make convert test.json -o converted/     # 1000 transcripts -> 1000 documents
```

Names carry the dataset's own identifier (`test-00007-unsafe_003923_aa62fc1a`),
so a converted document can be traced back to the row it came from. Asking to
convert such a file as though it were a single transcript is refused rather than
answered with the first of a thousand.

The parser was built against all 86 transcripts in the dataset, not the
published spec alone, which is how the three places they disagree came to light:
a tool result can be a list of content blocks rather than a string, `cwd` is
documented as required but is sometimes absent, and a transcript can end on a
tool call that never got a result. All 86 convert with no loss and pass the
reference validator.

Directories and archives — `.zip`, `.tar`, `.tar.gz`, `.tgz`, `.tar.bz2`,
`.tbz2`, `.tar.xz`, `.txz` — are read
as containers: every log inside is found and converted. That closes the loop on
`--bundle` — the zip atif-make hands you to send someone opens again in atif-make,
images and all.

Archives are extracted to a temporary directory, once per run. Members naming
absolute paths or climbing out with `..` are refused rather than quietly
sanitised, and an archive that expands past 8 GB or 20,000 entries is rejected
outright.

Format is detected from content, never from the extension. `atif` exists so a
trajectory someone sends you opens like anything else — it is loaded, not
reparsed, and unknown fields from a newer ATIF minor version are dropped rather
than rejected.

## What it gets right

These are the things that are easy to get wrong, and that silently corrupt a
trajectory when you do:

**Split messages.** Claude Code writes one API response as several JSONL lines
that share a `message.id` — thinking, text, and each parallel `tool_use` arrive
separately, with the *same* `usage` object repeated on every line. Treating those
as separate turns inflates step counts and multiplies token totals. atif-make
coalesces them and counts usage once.

**Out-of-order tool results.** Parallel calls come back interleaved, and a slow
call can return several turns after it was issued. Pairing results to calls by
*position* drops some and misattributes others. atif-make pairs by `tool_use_id`.

**Byte-capped detection.** A JSONL preamble (hook events, rate-limit notices) can
push the identifying line kilobytes into a file. atif-make scans whole lines.

**Subagent structure.** Claude Code links a delegated agent through a `.meta.json`
sidecar (`toolUseId`) and an `agentId` field on the result line — not through
anything in the result *text*. atif-make links by call id, so refs actually resolve
instead of leaving orphaned subagents.

**Delegation is in the sidecar, not the directory.** Every subagent of a session
is written into one flat `<session>/subagents/` directory whatever its depth; the
tree is `parentAgentId`, which names the subagent that spawned this one and is
absent when the main thread did. Reading the directory is not reading the tree: a
session where a subagent delegated in turn comes out as one wide row of
subagents that nothing refers to, because their spawning call is in another
subagent's log rather than the session's. Measured on a real three-level session
of fourteen subagents, that is eleven of the fourteen.

So atif-make nests them by `parentAgentId` and links each parent to its own
children by the sidecar's `toolUseId`, which is the only link that works below
the main thread. A subagent is named for its whole chain —
`<session>.<agent-id>.<agent-id>` — and the name is final before any ref to it is
written, because a subagent writes refs to its own children before its parent
ever sees it; a parent that renamed it afterwards would leave every ref below the
first level pointing at nothing. The spec does not catch either fault — it asks
that ids be present and unique among siblings, not that a ref resolves — so the
guards are parser tests.

Claude Code stops handing down the Agent tool below depth two, so three levels of
subagents is as deep as a session of its own goes; nothing here assumes that.

**Codex threads point the other way.** The Codex app has a multi-agent surface —
`create_thread`, `send_message_to_thread`, `read_thread` and the rest — declared
in each rollout's `session_meta.dynamic_tools` with a `codex_app` namespace and a
full JSON `inputSchema`. Most are `deferLoading`, so a model that reached one
through a tool search calls it `codex_app__create_thread` while a preloaded one
is called bare; reading only one spelling misses every session that had to go
looking.

The link between a thread and whoever spawned it is written only on the child.
A thread the app spawns gets a rollout of its own alongside the parent's, and its
`session_meta` carries `parent_thread_id`, `thread_source: "subagent"` and a
`session_id` that is the *root* session rather than its own — while the parent's
log says nothing about it at all. Read one file at a time, a run with subagents
comes out as unrelated sessions filed on the same day.

A thread is a peer rather than a child: the app lists every thread the user owns
in one sidebar whoever asked for it. So a thread hangs off the trajectory that
first spoke to it and everyone else refers to it there, which is what makes a
message from one agent to another agent's thread read as the two of them talking
rather than as one containing the other. Copying the thread under each sender
instead would count one agent twice and split what it was told across the copies.

What a thread tool *returns* is not declared anywhere, and on the machines this
was written against no rollout has ever called one — the names appear only in the
declaration list. So the request side is read from the schemas and the reply side
is not invented: a new thread's id is taken from the `::created-thread{threadId="…"}`
directive the app instructs the model to echo in its next message, and the single
inference — reading an id out of the result itself — is isolated in one function
that fails closed. A result it cannot read costs the thread its id, which is
honest; a looser reader picking a uuid out of prose would file one thread's work
under another. A thread whose id never surfaces is still shown, because the
delegation certainly happened.

**Images.** Codex embeds screenshots as base64 data URLs and Claude Code as
base64 content blocks — 24 of the sessions on one test machine carried them, and
a single Codex session held 65 images totalling 14 MB. Dropping them loses the
thing the agent was actually looking at, and inlining them makes an unreadable
document. atif-make writes them to `images/` and references them by relative path,
which is what the spec asks for.

**Malformed timestamps.** ATIF requires ISO 8601. A truncated or hand-edited log
can carry something else, and passing it through would make the whole trajectory
fail validation, so an unparseable timestamp is dropped rather than emitted.

**HAR tool results.** In a HAR capture a tool's output is not in the response that
called it — it appears in the *next* request's message history. atif-make harvests
results across entries and pairs them back by id, while emitting the shared
conversation prefix only once.

## Output — a trajectory is a directory, not a file

ATIF references images and split subagents by path *relative to the trajectory
file*, so anything with attachments is inherently multi-file:

```
session.trajectory.json                 the document
session.trajectory.<agent-id>.json      subagents, with --split-subagents
images/<sha>.png                        images, referenced as "images/<sha>.png"
```

Images are de-duplicated by content hash, so the same screenshot pasted five
times is stored once.

Three ways out, depending on where it's going:

```sh
atif-make session.jsonl -o out/t.json      # directory form: t.json + images/
atif-make session.jsonl --json             # one self-contained doc; images inlined as data: URIs
atif-make session.jsonl --bundle send.zip  # zip of the whole directory — for sending someone
```

`--json` is the exception that proves the rule: stdout has no directory to put
siblings in, so images become data URIs to keep the document standalone.

Output carries `timestamp`, `reasoning_content`, per-step `metrics`, multimodal
`ContentPart` message content, and subagents either embedded
(`subagent_trajectories`) or split into sibling files with resolvable
`trajectory_path` refs.

## What this does not do

It converts, and stops there. No index, no library, no memory of what you have
already looked at — those belong to whatever is doing the looking. In practice
that is [`transcript-viewer`](https://github.com/jammastergirish/transcript-viewer), which
depends on this package and keeps its own `~/.transcript-viewer/index.json`.

The line is worth stating because it moved: the index used to live here, which
meant the converter knew the viewer's directory (`~/.transcript-viewer/opened`) and its
vocabulary for where a session came from, for the sake of a command conversion
never needed. A converter should take one log — or an archive of them — and
produce ATIF.

Discovery survives only as far as conversion requires it: `convert.find` walks a
directory or an archive and reports what is convertible, with the format of
each. Nothing is remembered between runs.

## Tests

```sh
uv run pytest                      # unit tests
uv run --extra parquet pytest      # + the Parquet a published dataset ships as
uv sync --extra spec               # pulls harbor (large)
uv run --extra spec pytest         # + validate against the reference ATIF models
```

Fixtures for the agent formats are written by hand, small enough to read and
reason about. The dataset fixtures are not: ATBench and MALT are real rows from
the published files with only the prose shortened, because their shapes — a
tampered tool description, a rater with empty content, a call written into a
message, a record holding several conversations — are exactly what an invented
fixture would have smoothed away. Each of those was found in the real data after
a first parser had already been written against a summary of it.

The `spec` suite validates every fixture against harbor's own pydantic models —
ground truth for whether the output is really ATIF, rather than what atif-make
believes ATIF to be.
