# Logging

<!-- BEGIN GENERATED: guide-map -->
## Guide map

Read only the section needed for the next decision. Line ranges are exact
for this generated runtime target.

| Section | What it covers | Lines |
| --- | --- | --- |
| [Purpose](#purpose) | Scope, ownership, and the central operating question. | 29-34 |
| [Where To View Progress](#where-to-view-progress) | The local **Verbiflow daemon dashboard** is the canonical place to watch live job progress. | 35-49 |
| [Use This When](#use-this-when) | Tasks and conditions that belong in this guide. | 50-56 |
| [Do Not Use This For](#do-not-use-this-for) | Boundaries and the correct routes for excluded work. | 57-72 |
| [Mental Model](#mental-model) | Concepts and state boundaries needed for correct operation. | 73-92 |
| [Job Status Values](#job-status-values) | The dashboard's **Status** column (and `vf.log.status(job_id)`) shows one of these. | 93-109 |
| [Preflight](#preflight) | State, authority, inputs, and readiness to verify before execution. | 110-114 |
| [Risk And Side Effects](#risk-and-side-effects) | Effects, approvals, cost, privacy, and retry boundaries. | 115-126 |
| [Canonical Operating Pattern](#canonical-operating-pattern) | The normal end-to-end execution sequence. | 127-187 |
| [Core SDK Surfaces](#core-sdk-surfaces) | Exact source-backed callable contracts owned by this guide. | 188-199 |
| [Data Contracts](#data-contracts) | Required identities, fields, evidence, handles, and durable shapes. | 200-250 |
| [Examples](#examples) | Representative usage patterns and code. | 251-310 |
| [Verification](#verification) | Evidence required before reporting success. | 311-323 |
| [Troubleshooting](#troubleshooting) | Failure diagnosis and corrective routes. | 324-334 |
| [Dashboard](#dashboard) | Run the local job-progress dashboard: | 335-351 |
| [Related References](#related-references) | Adjacent boundaries and when to cross them. | 352-360 |
<!-- END GENERATED: guide-map -->

## Purpose

Per-job logging and progress reporting for same-machine long-running SDK or worker jobs. Use it when a caller needs to see lifecycle, counters, and the latest unit of work while an instrumented job is still running.

The wire format is a JSONL file at `~/.verbiflow/state/job-logs/<job_id>.jsonl` on the producer machine. The SDK provides an ergonomic writer (`vf.log.get(job_id)`) and reader (`vf.log.status`, `vf.log.progress`, `vf.log.recent`, `vf.log.tail`) helpers. Durable audit, billing, artifacts, workflow snapshots, and remote production job state still belong to backend/workflow tables and artifacts.

## Where To View Progress

```bash
verbiflow daemon start     # idempotent; safe to call repeatedly
open http://127.0.0.1:7777 # auto-refreshes every 2s
```

The local **Verbiflow daemon dashboard** is the canonical place to watch live job progress. It reads the same JSONL files this reference documents and shows every job on the machine with status badge + progress bar + last unit completed + last activity. Click a job row for the full event timeline.

JSON API for programmatic / scripted readers:

- `GET http://127.0.0.1:7777/api/jobs` — list of all jobs with summary
- `GET http://127.0.0.1:7777/api/jobs/<job_id>` — one job's full event history
- `verbiflow daemon stop` when you're done.

## Use This When

- You are writing a worker, activity, or SDK function that takes longer than a couple seconds and want the caller to be able to see what's happening.
- You are writing a script that calls a long job and want to print live progress while it runs.
- You are debugging "is this job stuck or just slow" — read the log file (or `vf.log.tail`) and the answer is in the timestamps.
- You are building a local dashboard or monitoring view that aggregates job state across running jobs on the same machine.

## Do Not Use This For

- High-frequency per-millisecond telemetry. The file is opened+appended on every call; a tight inner loop emitting 100k events/sec will slow down. Keep emission per logical unit of work (per URL, per row, per page) — not per CPU cycle.
- Persistent audit logs that must survive across machines or weeks. The file lives on the worker machine and is meant for live progress. Artifacts, workflow snapshots, and Postgres tables are the durable audit trail.
- Remote production job state unless the remote worker's log directory is explicitly bridged back to the operator. A local `verbiflow daemon` cannot read files from another machine.
- Replacing stdout for human-readable scripts. Use `print(...)` for one-shot scripts that only need terminal output. `vf.log` is for jobs whose progress needs to be readable from a separate process.
- Proving row, artifact, workflow output, report, or sequence-preview
  correctness. Use `testing.md` after rows/artifacts exist; `vf.log` only says
  whether the producer is alive and what unit of work last moved.
- Reporting a user-meaningful Workflow Play stage. Use `ctx.milestone(...)` for
  semantic stages and cite durable `run`, `output`, `job`, or `artifact` ids
  when a stage completes. The SDK resolves every cited id from the current
  run's workflow store before recording the milestone. Keep
  `vf.log.*` and `ctx.progress(...)` for inspectable low-level activity and
  numeric counters.

## Mental Model

Two parties:

- **Producer** (worker / activity / SDK function executing the job) — knows the job id, appends events as work progresses.
- **Consumer** (a separate process, a CLI, a dashboard, an LLM checking on a long job) — reads the file at the well-known path, or uses the SDK reader to get the latest progress.

They communicate through one JSONL file per job. The producer doesn't need to know the consumer exists; the consumer doesn't need a live connection to the producer. The file is the contract.

Four event types:

| `type` | Fields | Meaning |
| --- | --- | --- |
| `start` | `pid` (auto), plus arbitrary kv | Marks the beginning of work. Includes the producer's process id so a reader can detect a crashed worker that never wrote `end`. Emit once at the top, before any progress. |
| `log` | `level`, `msg`, plus arbitrary kv | A message. Like `print(...)` but structured + persistent. |
| `progress` | `done`, `total`, optional `unit`, optional `msg`, plus arbitrary kv | Numerator/denominator counter. Latest event wins for "how far along are we." `unit` is the identifier of whatever was just completed (url, row id, batch name, etc.) so the consumer knows *which* one. |
| `end` | `status` (`succeeded`, `failed`, or `cancelled`), plus arbitrary summary kv | Terminal event. After this is written the job's status is determined regardless of further events. Emit once in a finally block so it fires on both success and exception paths. |

A consumer reading `vf.log.progress(job_id)` gets back the latest `progress` event, with a derived `pct = round(100 * done / total, 1)`. A consumer reading `vf.log.tail(job_id)` gets every event of every type as it's appended.

## Job Status Values

The dashboard's **Status** column (and `vf.log.status(job_id)`) shows one of these. Each value is a strict consequence of which events are present in the log file.

| Status | Meaning | When you'll see it | What to do |
| --- | --- | --- | --- |
| `running` | `start` event seen, no `end` event yet, worker PID is alive. | The job is in flight right now. | Wait or check the dashboard for progress. |
| `succeeded` | `end` event with `status: "succeeded"` is the most recent terminal event. | The job finished cleanly. | Read the `end_summary` for counts/elapsed. |
| `failed` | `end` event with `status: "failed"`. | The worker caught an exception and emitted end inside a finally block. | The `end_summary.error` field carries the exception message. |
| `cancelled` | `end` event with `status: "cancelled"`. | The job was deliberately cancelled. | None — expected. |
| `crashed` | `start` event seen, no `end` event, worker PID is dead. | The worker process died (segfault, OOM, kill -9) before it could emit end. | Re-run the job; check system logs for the cause. |
| `unknown` | No `start` event in the file (and no `end` event). | The producer did not emit a `start` event — usually because the file was created before the start/end convention was wired in, or because a hand-written producer only emits `progress`. The reader genuinely cannot tell what state this job is in. | Either wire the producer to emit `log.start(...)` at the top, or delete the stale file: `rm ~/.verbiflow/state/job-logs/<job_id>.jsonl`. |

### Why `unknown` is common right after a producer upgrade

When a tool is upgraded from "only emits progress" to "also emits start/end", every log file written by the **old** version stays `unknown` forever — the new reader logic looks for events that the old producer never wrote. This is intentional: the reader refuses to guess. The fix is to delete the stale files (they're debug history, not durable state) once the new producer is shipped. From then on, every new job will resolve to a real status.

## Preflight

- Generate or get the job id before emitting anything. Without a job id the writer is a no-op (silently — no exceptions). Job ids should be globally unique within the machine; use the same id the artifact / workflow tables use.
- Decide what "1 unit of progress" means for this job before you start emitting. For a 200-URL scrape, 1 unit = 1 URL completed. For a 1000-row enrichment, 1 unit = 1 row enriched. For a multi-batch job, 1 unit can be 1 batch. Once chosen, stay consistent — `done` going down or `total` changing mid-job confuses every consumer.

## Risk And Side Effects

| Operation | Risk | Rule |
| --- | --- | --- |
| `vf.log.get(...).start(...)` | local file append | Required once at the top of an instrumented producer. Records PID for same-machine crash detection. |
| `vf.log.get(...).print(msg)` | local file append (~50 bytes per call) | Safe. Emit freely for meaningful events. Avoid in tight inner loops. |
| `vf.log.get(...).update_progress(done, total)` | local file append | Safe. Call exactly once per unit-of-work completion. |
| `vf.log.get(...).end(status, ...)` | local file append | Required in a `finally` block for instrumented producers so the dashboard can terminalize the job. |
| `vf.log.status(job_id)` | local file read | Safe. Derives `running`, `succeeded`, `failed`, `cancelled`, `crashed`, or `unknown` from lifecycle events and PID liveness. |
| `vf.log.progress(job_id)` | local file read | Safe. Returns `None` if the file doesn't exist yet. |
| `vf.log.tail(job_id)` | blocking iterator | Caller must break out of the loop or set a timeout — `tail` waits forever for new events by default. |

## Canonical Operating Pattern

**Producer side.** Bind the logger once at the top of the worker. Emit `start` first (records the worker's PID for crash detection), `update_progress` per unit, and `end` in a finally block.

```python
import verbiflow as vf

def run_scrape_job(job_id: str, urls: list[str]) -> None:
    log = vf.log.get(job_id)
    log.start(tool="scrape", url_count=len(urls))          # required: marks job alive + pid
    log.print("starting", n_urls=len(urls))
    log.update_progress(0, len(urls))
    ok = 0
    failed = 0
    status = "succeeded"
    error: str | None = None
    try:
        for i, url in enumerate(urls, start=1):
            try:
                # Perform exactly one logical unit of work for this URL here.
                ok += 1
            except Exception as exc:
                failed += 1
                log.print("fetch failed", level="warn", url=url, error=str(exc))
            # ALWAYS emit progress after each unit — even on failure — so the
            # consumer can tell something is happening. Stalled progress is the
            # signal that the job is actually stuck.
            log.update_progress(i, len(urls), unit=url)
    except BaseException as exc:
        status = "failed"
        error = str(exc)
        raise
    finally:
        summary = {"ok": ok, "failed": failed}
        if error:
            summary["error"] = error
        log.end(status, **summary)                         # required: marks job terminal
```

**Consumer side.** Choose based on what you need.

```python
# One-shot pull of the latest progress (e.g. for a status check).
p = vf.log.progress(job_id)
if p:
    print(f"{p['done']}/{p['total']} ({p['pct']}%)")

# Live follow (e.g. printing progress in a long-running script).
for evt in vf.log.tail(job_id):
    if evt["type"] == "progress":
        print(f"  {evt['done']}/{evt['total']} {evt.get('msg', '')}")
    elif evt["type"] == "log":
        print(f"  [{evt['level']}] {evt['msg']}")
    if evt.get("type") == "end":
        break

# Last N events for context (e.g. showing the tail in a dashboard).
for evt in vf.log.recent(job_id, n=20):
    print(evt)
```

## Core SDK Surfaces

<!-- BEGIN GENERATED: core-sdk-surfaces -->
| Function | Use When | Inputs | Output And Handles To Preserve | Side Effect | Source |
| --- | --- | --- | --- | --- | --- |
| `vf.log.get` | Create a per-job local logger handle for lifecycle, message, and progress events. | <strong>Required</strong>: `job_id` (str; required): Stable job identifier whose JSONL file should receive lifecycle and progress events.<br><strong>Optional</strong>: none | `JobLogger handle; preserve it for .start(...), .print(...), .update_progress(...), and .end(...) calls.` payload returned by the SDK operation. Inspect the documented function reference and `raw`/dictionary fields when debugging provider-specific data.<br><strong>Common next calls</strong>: `vf.log.status`, `vf.log.progress`, `vf.log.recent` | none | installed SDK: `verbiflow/log.py:184-214` |
| `vf.log.progress` | Read the latest progress event from a local per-job JSONL file. | <strong>Required</strong>: `job_id` (str; required): Job id whose latest progress event should be read.<br><strong>Optional</strong>: none | `latest progress dict with done, total, derived pct, ts, and optional unit/msg; None when no progress event exists.` payload returned by the SDK operation. Inspect the documented function reference and `raw`/dictionary fields when debugging provider-specific data.<br><strong>Common next calls</strong>: `vf.log.status`, `vf.log.recent` | local_file_read | installed SDK: `verbiflow/log.py:329-378` |
| `vf.log.recent` | Read the last N events from a local per-job JSONL file. | <strong>Required</strong>: `job_id` (str; required): Job id whose tail events should be read.<br><strong>Optional</strong>: `n` (int; optional; default `20`): Maximum number of recent JSONL events to return. | `chronological list of recent event dictionaries.` payload returned by the SDK operation. Inspect the documented function reference and `raw`/dictionary fields when debugging provider-specific data.<br><strong>Common next calls</strong>: `vf.log.status`, `vf.log.progress` | local_file_read | installed SDK: `verbiflow/log.py:381-427` |
| `vf.log.status` | Derive the current local job status from start/end events and same-machine PID liveness. | <strong>Required</strong>: `job_id` (str; required): Job id whose JSONL log file should be summarized.<br><strong>Optional</strong>: none | `dict with status, pid, started_at, ended_at, end_summary, and last_activity_at.` payload returned by the SDK operation. Inspect the documented function reference and `raw`/dictionary fields when debugging provider-specific data.<br><strong>Common next calls</strong>: `vf.log.progress`, `vf.log.recent` | local_file_read | installed SDK: `verbiflow/log.py:232-326` |
| `vf.log.tail` | Follow a local per-job JSONL file and yield events as producers append them. | <strong>Required</strong>: `job_id` (str; required): Job id whose JSONL file should be followed.<br><strong>Optional</strong>: `poll_interval_seconds` (float; optional; default `0.5`): Positive polling delay between file checks. | `blocking iterator of event dictionaries; caller owns timeout/break conditions.` payload returned by the SDK operation. Inspect the documented function reference and `raw`/dictionary fields when debugging provider-specific data.<br><strong>Common next calls</strong>: `vf.log.status` | local_file_read | installed SDK: `verbiflow/log.py:430-484` |
<!-- END GENERATED: core-sdk-surfaces -->

## Data Contracts

### File location

```
~/.verbiflow/state/job-logs/<job_id>.jsonl
```

The directory is created lazily on first write. One file per job. Files are append-only.

### Event shapes

```jsonl
{"ts": 1718712345.0, "type": "start",    "pid": 12345, "tool": "page_extract", "url_count": 173}
{"ts": 1718712345.1, "type": "log",      "level": "info", "msg": "starting", "n_urls": 173}
{"ts": 1718712345.2, "type": "progress", "done": 0,       "total": 173}
{"ts": 1718712347.3, "type": "log",      "level": "warn", "msg": "rate limited", "url": "a.com", "attempt": 2}
{"ts": 1718712348.9, "type": "progress", "done": 1,       "total": 173, "unit": "https://a.com"}
{"ts": 1718712349.4, "type": "progress", "done": 2,       "total": 173, "unit": "https://b.com"}
{"ts": 1718712380.0, "type": "end",      "status": "succeeded", "ok": 156, "failed": 4}
```

Required fields:

- Every event: `ts` (float, unix seconds), `type` (`"start"`, `"log"`, `"progress"`, or `"end"`).
- `start` events: `pid` (int) is added by `JobLogger.start(...)`.
- `log` events: `level` (`"debug"` | `"info"` | `"warn"` | `"error"`), `msg` (str).
- `progress` events: `done` (int), `total` (int).
- `end` events: `status` (`"succeeded"`, `"failed"`, or `"cancelled"`).

Recommended fields:

- `progress.unit` — the identifier of whatever was just completed (URL, row id, batch name). Including this turns `done: 25, total: 30` from "25 of 30 finished" into "just finished `<unit>`, 25 of 30." Without it the consumer can see the counter advance but can't tell which item is currently slow or which one just failed.

Anything else is optional structured context. Consumers should tolerate unknown keys.

### The "one progress event per unit done" rule

Any tool that does N units of work (N URLs, N rows, N pages, N batches) must call `update_progress(done, total)` exactly once per completed unit, including failures. This is what makes "is it stuck or just slow" answerable from outside the worker — if `done` is moving, the worker is alive; if it's been stuck on the same value for >N seconds, the worker is genuinely stalled. Without per-unit emission, every consumer has to guess.

Acceptable variations:

- Batched units: emit once per batch (`done += batch_size`) when batches are small (≤ 100ms each).
- Sub-second units: emit every K units instead of every unit (`if i % K == 0: update_progress(...)`). Pick K so emission is at most every ~500ms.

Not acceptable:

- Emitting only `total` once at start with no further progress.
- Emitting only at the end. That's just a status, not progress.
- Skipping progress on the failure path. Progress = "this many units have completed", success or fail.

## Examples

### Producer: wired into a Bridge scraping activity

```python
log = vf.log.get(job_id)
log.start(tool="page_extract", url_count=len(urls), formats=list(formats))
log.print("page_extract start", url_count=len(urls), formats=list(formats))
log.update_progress(0, len(urls))

done = 0
ok_count = 0
fail_count = 0
status = "succeeded"
error = None
try:
    async for row in iter_bounded_results(urls, concurrency=3, worker=extract_one):
        done += 1
        log.update_progress(done, len(urls), unit=row.get("url"))
        if row.get("error"):
            fail_count += 1
            log.print("extract failed", level="warn", url=row["url"], error=row["error"])
        else:
            ok_count += 1
except BaseException as exc:
    status = "failed"
    error = str(exc)
    raise
finally:
    summary = {"ok": ok_count, "failed": fail_count}
    if error:
        summary["error"] = error
    log.end(status, **summary)
```

### Consumer: status check from another script

```python
import time
import verbiflow as vf

while True:
    p = vf.log.progress(job_id)
    if p is None:
        print("no progress yet")
    else:
        print(f"{p['done']}/{p['total']} ({p['pct']}%)  msg={p.get('msg', '')}")
        if p["done"] >= p["total"]:
            break
    time.sleep(2)
```

### Consumer: rendering a dashboard widget

```python
events = vf.log.recent(job_id, n=10)
latest_progress = vf.log.progress(job_id)
# Send `events` + `latest_progress` to the dashboard template.
```

## Verification

After integrating logging into a new tool, confirm by:

1. Running the tool with a small input (5–10 units).
2. `cat ~/.verbiflow/state/job-logs/<job_id>.jsonl` and verifying:
   - `start` event at the top with a `pid`.
   - One `progress` event per unit completed (count should equal `len(input)`).
   - `end` event at the bottom with the terminal status and result summary.
3. `vf.log.progress(job_id)` returns `done == total` after the tool finishes.
4. `vf.log.status(job_id)` returns `succeeded`, `failed`, or `cancelled` after the tool finishes.
5. Mid-run, `vf.log.progress(job_id)` returns a partial value (run a longer job and check while it's running).

## Troubleshooting

| You see | Likely cause | Fix |
| --- | --- | --- |
| `vf.log.progress(job_id)` returns `None` after the job ran | Job emitted no `progress` events (only `log`). Or you have the wrong `job_id`. | Verify the producer is actually calling `update_progress(...)`, not just `print(...)`. Check `~/.verbiflow/state/job-logs/` for the right file. |
| Dashboard shows a job as `unknown` even though it clearly finished | The job's log file has no `start` (and no `end`) event — usually written by an older producer before the start/end convention was wired in. The reader refuses to guess at a missing-state. | Either delete the stale file (`rm ~/.verbiflow/state/job-logs/<job_id>.jsonl`) or update the producer to emit `log.start(...)` at the top + `log.end(status, ...)` in a finally block. See "Job Status Values" above for the full state table. |
| Progress jumps directly from 0 to total at the end | Producer is only emitting progress at start and end, not per unit. | Move the `update_progress` call inside the per-unit loop. |
| `vf.log.tail` never yields | The producer hasn't started yet (file doesn't exist) — tail waits politely. Or the file exists but is empty. | If you want a non-blocking probe, use `vf.log.recent(job_id, n=N)` and handle the empty case. |
| Multiple writers conflict / events out of order | Multiple processes are calling `vf.log.get(same_job_id).print(...)` concurrently. POSIX `O_APPEND` is atomic for small writes (<4KB), so this is usually safe — but events may interleave by timestamp, not by source order. | Pick one writer per job. If you need multi-writer, accept the interleaving and rely on `ts` for ordering. |
| File grows unbounded | The producer emits in a tight loop with no rate limit. | Batch-emit (every K units) or hoist out of the inner loop. Add explicit retention/rotation before using this for high-volume producers. |

## Dashboard

Run the local job-progress dashboard:

```bash
verbiflow daemon start               # one-shot, idempotent
open http://127.0.0.1:7777           # auto-refreshes every 2s
verbiflow daemon stop                # when done
```

The daemon reads the same JSONL files this reference documents. Every job that emits a `start` event shows up; status transitions to `running` while the PID is alive, then to the `end` event's status once it fires (`succeeded` / `failed` / `cancelled`). Crashed workers (start emitted, no end, PID dead) show as `crashed`.

JSON API for programmatic readers:

- `GET /api/jobs` — all jobs with summary fields.
- `GET /api/jobs/<job_id>` — one job's full event list.

## Related References

| If you are doing this | Read next |
| --- | --- |
| Instrumenting page extraction or another long-running producer | `web.md` for the producer's row/data contract, then this file only if your wrapper emits `vf.log.*` JSONL progress. Plain `vf.host.web.capture_pages(...)` uses host action history and manifest files, not daemon job-progress logs. |
| Waiting on Bridge jobs, backend events, committed batches, or durable artifacts | `artifacts_jobs.md`; those are backend job handles, not local `vf.log.*` files. |
| Running or inspecting a generated workflow root | `workflow_operations.md`; workflow-local low-level progress uses `ctx.progress(...)`, semantic stages use `ctx.milestone(...)`, and both persist as SQLite events/outputs rather than `vf.log.*` JSONL files. |
| Proving produced rows/artifacts/previews are correct before staging, publishing, or showing them | `testing.md`; `vf.log.*` only proves the producer is alive and moving. |
| Building a standalone row visualization with `vf.viz.*` | `dashboard.md`; this is unrelated to the local daemon job-progress dashboard. |
