Metadata-Version: 2.4
Name: eqty-lineage-deepagents
Version: 0.2.1
Summary: EQTY lineage callback handler for DeepAgents.
License-Expression: Apache-2.0
License-File: LICENSE
Requires-Dist: eqty-sdk>=2.4.2
Requires-Dist: eqty-lineage-langchain>=0.1.0
Requires-Dist: langchain-core>=0.3
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# eqty-lineage-deepagents

EQTY lineage callback handler for [DeepAgents](https://github.com/langchain-ai/deepagents). Records a deep agent run —
its virtual filesystem, its plan, its subagents and its skills — as EQTY data assets and computation statements.

`EqtyDeepAgentsHandler` subclasses `EqtyCallbackHandler` from
[`eqty-lineage-langchain`](../eqty-lineage-langchain/README.md), so everything that handler records is recorded here
too: graph nodes, model calls, tool calls, retrievals, subagent boundaries, failures, credential redaction. This
package adds the parts of a deep agent that live in its **state** rather than in its callbacks.

## Usage

```python
from eqty_lineage.deepagents import EqtyDeepAgentsHandler

agent.invoke({"messages": [...]}, config={"callbacks": [EqtyDeepAgentsHandler()]})
```

Use one handler per conversation — see [Handler lifetime](#handler-lifetime). `deepagents` itself is never imported,
and is **not** a dependency of this package — everything is read from the callback stream and from graph state, so the handler works against whatever
version of DeepAgents produced the run, and installing it does not pin your `langchain`/`langgraph` versions.
`langchain-core` is the only framework requirement.

```bash
just deepagents-demo          # scripted model, no API key; writes manifests/deep-agent.json
```

For a worked survey of the fifteen examples upstream DeepAgents ships — which eight can be instrumented, the
exact edit each needs, and why the other seven cannot — see [EXAMPLES.md](EXAMPLES.md).

## Handler lifetime

**One handler per conversation, never one shared between conversations running at once.**

Reusing a handler *sequentially* is deliberate and useful: across the turns of one checkpointed thread it is what
chains a file written in the first turn to an edit in the third, rather than leaving two unrelated entities.

Sharing one between **concurrent** runs is a bug. File and plan versions are keyed by path — or, for the plan, by
nothing at all — because they describe a single run's filesystem. Two runs in flight on one handler collide on those
keys, and the second run's write of `/report.md` is recorded as a revision of the first run's file, between runs that
share nothing but the handler. The graph still looks well-formed; it is simply wrong.

Because that failure is quiet, the handler warns when it sees a second run open while another is still running:

```
EqtyDeepAgentsHandler is observing 2 runs at once. File and plan versions are keyed per run, so
concurrent runs will be linked to each other's assets. Use one handler per invocation ...
```

A server that fans out concurrent runs over one shared agent should build the handler per run — the agent holds no
lineage state and can stay shared:

```python
async def execute_run(user_message: str) -> None:
    with graph_context(ctx):
        await agent.ainvoke(
            {"messages": [HumanMessage(user_message)]},
            config={"callbacks": [EqtyDeepAgentsHandler()]},  # one per run
        )
```

`eqty_sdk.init()` is process-global, and a second call does not raise — it logs `Config already initialized` and
returns, leaving the first store in place. So initialise it once at startup: a per-run `init()` is silently ignored
and every run writes into the store the first one created. `graph_context` is backed by a `ContextVar`, so entering it inside each task is safe under `asyncio`.

## What it records

| DeepAgents concept | asset | where it lands in the lineage |
| --- | --- | --- |
| the deep agent | `Agent` | input to the root `graph` computation |
| a subagent | `Agent` | input to its own `agent` computation **and** to the `task` call that spawned it |
| a subagent's execution | — | an `agent` computation (from the base handler) |
| a virtual file | `Document` | output of the call that wrote it; input to every call that reads it |
| a file the run only reads | `Document` | input to the `read_file` call — the tool's rendering, not the file's bytes |
| a revision of the todo list | `Dataset` | output of the `write_todos` call that wrote it, chained to the one it replaced |
| a loaded skill | `Skill` | input to the model turn whose state held the catalogue |
| a model turn's system prompt | `SystemPrompt` | input to that `chat_model` computation |

Everything above maps onto an asset type the SDK already ships; this package introduces no new primitives.

## The virtual filesystem

A deep agent carries its whole filesystem in graph state, so without an extractor every node's state asset embeds
every file — once per node — and no file is ever an entity in its own right. `VirtualFileExtractor` claims the `files`
state key and replaces it with a map of path → content CID, so the state asset says *which* files the node had
without containing them.

**A file asset's CID is its content's CID.** The payload handed to the `Document` constructor is the file's
content and nothing else, so the CID is exactly what `get_cid_for_bytes` returns for those bytes — the same
identity any other tool computes for that file, and what Lineage Explorer renders when you open the node. The
path is metadata (`file_path`), attached unconditionally rather than only in verbose mode. Two consequences
worth knowing: identical content at two paths is *one* asset, and a file's identity is comparable across runs,
across capture paths, and against a CID computed outside this package.

Versions are keyed on `(path, content)`, the same rule `PathExtractor` applies to real paths:

- **new content at a known path** — a new `Document`, recorded as an *output* of the call that wrote it, with the
  version it replaced linked as an *input*. Successive edits form a chain.
- **content already registered** — carried through as an *input*, never re-emitted as an output. A *write* that
  restores earlier bytes is the exception: it mints no new asset, but it is still that call's output, because the
  version it replaced has stopped being current.

Paths are normalized the way DeepAgents normalizes them — mirroring `validate_path` step for step, not
approximating it. It forces a leading slash and collapses `.` and interior `//`, so a model that asks to write
`report.md` creates `/report.md` in state; keying the raw argument would put the write on one entity and every later
read on another, and live models omit the leading slash routinely.

*Near* agreement would be worse than none: two paths the backend keeps apart but this folds together are two real
files recorded as one asset, so a write to one is attested as a rewrite of the other. A **leading** `//` is exactly
that case — POSIX gives it a meaning of its own and `normpath` preserves exactly two slashes, so `//report.md` is a
different file from `/report.md`. A path `validate_path` *refuses* (a `..` component, a leading `~`, a Windows drive
letter) never reaches the backend, so nothing is recorded for it rather than it being rewritten into a plausible
path the call never touched. A test asserts this agreement against the real `validate_path` rather than against a
table of expected strings, so it fails rather than drifts if upstream changes its rules.

The path is part of the asset's payload, so the same bytes written to two paths are two files rather than one entity
wearing whichever name happened to be registered first.

### Why the write is attributed from the tool's arguments

DeepAgents' `StateBackend` applies its writes through LangGraph's channel API, not through the tool's return value.
The new content therefore appears in neither the tool's result nor the enclosing node's output state — it simply
shows up in `files` at the next model turn. Left at that, a file the agent wrote would be an *input* to everything
that later read it and an output of nothing, which in a provenance graph says the run found it already there.

So the handler reconstructs the write from the call's own arguments:

- `write_file` carries the exact content, so the new version is registered directly.
- `edit_file` reports only that it succeeded, so the result is derived from the version the edit was made against —
  a plain string replacement, with the backend's own occurrence rules re-checked before it is trusted. If any check
  fails, **nothing is registered**: the file then appears at its next sighting in state, as an input, which
  understates its provenance rather than misstating it. Those re-checked rules are a copy of the backend's, so the
  tests assert them against the real `perform_string_replacement` over a table of edits it applies *and* edits it
  refuses — a copy that drifts is worse than no copy, in either direction.
- `delete` drops the path's current version, so a later write chains to nothing rather than to content that no
  longer existed, and a later read is not linked to a version it could not have read.
- a call that failed registers nothing. Failure is read from the result's `status` rather than its wording: only
  some backends say "Error", while the store, LangSmith and sandbox backends report "Failed to write file …" or the
  remote's own message verbatim, and a prefix match would take those for successes.

### Files the run only reads

The above covers files the agent *writes*, and the extractor covers files carried in state. Neither reaches a file
that already existed and is never written — which is every source file an agent reads under a **filesystem, store or
sandbox backend**, since those keep the filesystem out of graph state entirely. Left alone, the model's answer
derives from a `read_file` computation that consumed nothing.

So a successful `read_file` on a path with no known version registers what the tool returned, as an **input** to that
call. It is labelled as a rendering rather than as the file, because that is what it is: the result is line-numbered,
chunked at long lines and truncated when large, and the loss cannot be undone — a file ending in a newline renders
identically to one that does not, so the bytes cannot be recovered. Recording it under the file's own identity would
assert a content hash the file never had.

That asset is therefore kept apart from the version chain: a rendering never becomes the version a later `edit_file`
is reconstructed against, never satisfies a later read, and is only registered when the path has no real version
already — so a file the run wrote is never shadowed by how it was read.

### When a shell runs

`execute` is in the default tool belt, and a shell can create, rewrite or remove files without naming a
path. What it did cannot be recovered from the command, so it is not recorded. What is avoided is
*asserting* the filesystem it left behind: a successful `execute` drops the reconstruction caches, so a
later `edit_file` reconstructs against nothing and registers nothing, and a later `read_file` registers the
rendering it actually got rather than linking a version that may no longer exist. A gap where a shell ran,
rather than a version the file never held.

Only a sandbox or local-shell backend implements `execute`; every other backend errors the call, and an
error changes nothing. Under the state backend the extractor re-registers the filesystem from the next
state it sees, so nothing is lost there either.

## The plan

`TodoListMiddleware` is **not** part of the default deep agent stack — it comes from `langchain` and has to be passed
explicitly:

```python
from langchain.agents.middleware import TodoListMiddleware

create_deep_agent(..., middleware=[TodoListMiddleware()])
```

With it, each `write_todos` call produces a `Dataset` holding that revision of the plan, chained to the revision it
replaced. A plan written once and never revised is one asset; a plan revised four times is four, in a chain. Without
the middleware the state key does not exist and nothing is recorded — no phantom plan appears.

## Skills

`SkillsMiddleware` parses each `SKILL.md` and puts the catalogue in the `skills_metadata` state key, so the skills a
model turn could draw on are exactly the ones in the state entering it. Each is registered as a `Skill` asset,
content-addressed to that metadata, and **carried** as an input — a skill is something the run was given, never
something it produced. The `SKILL.md` file itself is an ordinary virtual file, registered when it is read.

## State keys, not tool arguments

The extractors claim a state key at the top of a state and inside the `update` of a LangGraph `Command` (which is how
a tool applies a state update, and how `task` hands a subagent's work back). They deliberately do **not** claim the
same name anywhere else, because the arguments of a call and the state it produces are serialized by the same code:
`write_todos(todos=[...])` would otherwise register the new plan as an *input* to the call that wrote it, reversing
the one edge the plan's lineage exists to record.

## Writing your own extractor

`StateExtractor` is re-exported here, so a custom state key needs no separate import:

```python
from eqty_lineage.deepagents import UNCLAIMED, EqtyDeepAgentsHandler, StateExtractor


class BudgetExtractor(StateExtractor):
    def extract(self, key_path, value, sink):
        if key_path != ("budget",):
            return UNCLAIMED
        ...


handler = EqtyDeepAgentsHandler()
handler.add_extractor(BudgetExtractor())
```

Extractors are consulted newest-first, so one registered here beats this package's own. See the
[LangChain package README](../eqty-lineage-langchain/README.md) for the full contract.

## Limitation: compaction is recorded, but not *as* a compaction

`SummarizationMiddleware` is in the default deep agent stack. On a long run it compacts the conversation: the new
context is derived from the old, but **lossily**, with most of it discarded.

**It is observable, and it is recorded.** An earlier version of this note claimed otherwise, on the belief that
compaction happens inside `wrap_model_call`. It does not: `SummarizationMiddleware` overrides `before_model`, which
LangGraph runs as a node of its own, so the handler records it like any other. Verified against a run that compacted
4 of its 7 model turns:

- the node appears as a `graph_node` computation named `SummarizationMiddleware.before_model`, on every turn;
- a turn that compacted has an output state carrying the `RemoveMessage` sentinel, the summary message and whatever
  `keep` preserved, where a pass-through turn carries `state_update: null`;
- the summarization model call is its own `chat_model` computation, with the summary prompt as a `Prompt` asset and
  the generated summary as `Reasoning`.

**What is missing is the edge.** The SDK's vocabulary is `add_computation_statement(inputs, outputs)` —
`prov:used` and `prov:wasGeneratedBy` — plus `CERTIFIES`, `INCLUDES` and `IS_INSTANCE_OF` associations. The
compaction is therefore recorded as an ordinary derivation, which implies the output carries the input when in fact
most of it was dropped. No asset type fixes this: the missing thing is an *edge*, not an entity. PROV, OpenLineage
and in-toto do not model lossy derivation either.

**What this means for a reader.** You can see *that* a compaction happened, which turn it happened on, and what
survived it. You cannot tell from the edges alone that the step was lossy rather than an ordinary transformation.
An `eqty:wasCompactedFrom` edge is the one thing here that cannot be expressed today and cannot be worked around by
choosing a different asset type; it is being raised with the SDK team separately.

Compaction only fires on long runs, so shorter runs are unaffected.

## `@eqty_tool` and the built-in tool belt

`eqty_tool` is re-exported here and works as it does in the LangChain package: decorate your own tool and
its `Tool` asset is content-addressed to its source rather than to a name/description stub.

The tools that do a deep agent's most interesting work, though, are not yours to decorate — `write_file`,
`edit_file` and `task` are built by DeepAgents' own middleware. Without their source, the manifest records
*that* a file was written but not by what code. Their source is perfectly readable, so the same function the
decorator calls can be applied to the belt the compiled agent assembled:

```python
agent = create_deep_agent(...)

for tool in agent.nodes["tools"].bound.tools_by_name.values():
    eqty_tool(tool)
```

Upgrade DeepAgents and those assets change, which is the point. `examples/deepagents/research_agent.py`
does exactly this; its manifest carries six Tool assets, each holding the source of the function that ran.
The handler does not do it for you — reaching into a compiled graph for the belt is a caller's liberty, not
something a callback handler should assume — and a tool whose source cannot be read simply falls back to
its stub.

## Verbose metadata

```python
EqtyDeepAgentsHandler(verbose=True)
```

Works exactly as in the LangChain package; see
[its README](../eqty-lineage-langchain/README.md#verbose--extra-metadata-on-assets).
