# Parallelism

<!-- BEGIN GENERATED: guide-map -->
## Guide map

Read only the section needed for the next decision. Line ranges are exact
for this generated runtime target.

| Section | What it covers | Lines |
| --- | --- | --- |
| [Purpose](#purpose) | Scope, ownership, and the central operating question. | 25-37 |
| [Use This When](#use-this-when) | Tasks and conditions that belong in this guide. | 38-51 |
| [Do Not Use This For](#do-not-use-this-for) | Boundaries and the correct routes for excluded work. | 52-64 |
| [Mental Model](#mental-model) | Concepts and state boundaries needed for correct operation. | 65-90 |
| [Preflight](#preflight) | State, authority, inputs, and readiness to verify before execution. | 91-108 |
| [Risk And Side Effects](#risk-and-side-effects) | Effects, approvals, cost, privacy, and retry boundaries. | 109-123 |
| [Canonical Operating Pattern](#canonical-operating-pattern) | The normal end-to-end execution sequence. | 124-143 |
| [Data Contracts](#data-contracts) | Required identities, fields, evidence, handles, and durable shapes. | 144-162 |
| [Examples](#examples) | Representative usage patterns and code. | 163-287 |
| [Verification](#verification) | Evidence required before reporting success. | 288-302 |
| [Troubleshooting](#troubleshooting) | Failure diagnosis and corrective routes. | 303-314 |
| [Related References](#related-references) | Adjacent boundaries and when to cross them. | 315-327 |
<!-- END GENERATED: guide-map -->

## Purpose

This reference owns concurrency choices for SDK, provider, artifact, row, and
workflow work. It answers: should this fan-out use async SDK calls, a built-in
bulk helper, artifact streaming, workflow step concurrency, or a local worker?

The default for SDK/provider fan-out is one `asyncio` runtime with
`vf.AsyncClient`, bounded by explicit concurrency. Do not parallelize sync
`vf.*` calls with `ThreadPoolExecutor`, `asyncio.to_thread(lambda: vf.*...)`, or
`asyncio.run(...)` from worker threads. Those patterns create multiple event
loops around async SDK objects and can fail with errors such as `Event bound to
a different event loop`.

## Use This When

- A workflow or script starts, waits on, streams, exports, enriches, qualifies,
  or reads many SDK jobs.
- A step has a serial `for chunk: start provider job; wait; read rows` loop and
  should instead start or wait bounded chunks concurrently.
- An agent is tempted to use `ThreadPoolExecutor`, `asyncio.to_thread(...)`,
  `asyncio.run(...)`, or sync `vf.*` calls to speed up SDK reads.
- You need to choose between `vf.jobs.wait_many(...)`, `vf.jobs.stream_rows(...)`,
  `vf.rows.ai_analyze(mode="enrich", ...)`, `vf.rows.ai_analyze(mode="qualify", ...)`, or a hand-written
  `AsyncClient` fan-out.
- A generated workflow uses `pipelined=True`, step concurrency, or row streaming
  and you need to confirm what actually runs in parallel.

## Do Not Use This For

- Pure local CSV/row shaping. Use `row_operations.md` unless the local transform
  itself is CPU-bound enough to need a process/thread worker.
- Job and artifact lifecycle details such as `JobResult`, `artifact_id`,
  `next_offset`, or batch cursors. Use `artifacts_jobs.md` after choosing the
  concurrency primitive.
- Workflow checkpointing and source freshness semantics. Use
  `workflow_sources.md` for source cursors and `workflow_operations.md` for
  replay/reset behavior.
- Sequence staging, publish, send, or billing approvals. Use `sequences.md` and
  `safety.md`; parallelism does not lower approval requirements.

## Mental Model

There are three different execution classes. Keep them separate:

| Class | Correct Primitive | Never Do |
| --- | --- | --- |
| SDK/provider IO | `vf.AsyncClient` plus `asyncio.gather(...)`, `asyncio.Semaphore`, or SDK bulk helpers. | Do not run sync `vf.*` calls in `ThreadPoolExecutor` or `asyncio.to_thread(...)`; do not call `asyncio.run(...)` inside worker threads. |
| Already-started jobs/artifacts | `vf.jobs.wait_many(...)`, `vf.jobs.stream_rows(...)`, `JobResult.iter_rows(...)`, artifact pages, and preserved cursors. | Do not poll one job to completion before starting or waiting on the next one. |
| Local CPU/blocking work | `asyncio.to_thread(...)` for small blocking local work, or `ProcessPoolExecutor` for real CPU isolation. | Do not share SDK clients, `JobResult` handles, HTTP clients, or workflow stores across event loops/threads. |

The sync SDK facade is for top-level one-off scripts and notebooks. Most sync
namespace methods bridge through `Client.run(...)`; some sync streaming/listing
helpers use blocking iterators directly. Treat all sync SDK surfaces as
single-threaded convenience APIs. `Client.run(...)` uses `asyncio.run(...)` when
no event loop is active and raises `Use AsyncClient inside an active event
loop.` when a loop is active. That is useful guardrail behavior, not a
parallelism primitive.

Workflow sync steps are allowed for local transforms, but SDK IO inside
workflow steps should use an async step and `ctx.async_client` so it stays in
the workflow event loop. `pipelined=True` in workflows is not magic parallelism
for arbitrary Python loops. It allows eligible workflow steps to advance based
on workflow runtime rules. A loop inside one step remains serial until you use
a built-in `_all` helper, `vf.jobs.wait_many(...)`, or an explicit async
gather/semaphore pattern.

## Preflight

Before adding concurrency:

1. Confirm the current execution authority and relevant provider readiness
   through the owning reference.
2. Identify whether the work is local, provider/job-backed, sequence/source
   staging, or customer-visible mutation.
3. Preserve every `job_id`, `artifact_id`, `batch_id`, `after_batch_id`,
   `source_run_id`, and output path before fan-out.
4. Choose a concurrency cap before starting work. Default to small provider
   caps (`5..10`) unless the reference or backend contract says otherwise.
5. Prefer a built-in bulk helper when it exists. It already encodes chunking,
   row caps, start/wait separation, and handle preservation.
6. If an async runtime already exists, use `await`, `ctx.async_client`, or
   `vf.AsyncClient`; do not bridge through sync `vf.*`, `asyncio.to_thread(...)`,
   or nested `asyncio.run(...)`.

## Risk And Side Effects

Parallelism changes blast radius. Starting ten provider jobs concurrently can
spend credits or hit provider limits ten times faster. It does not change the
approval model:

- Local transforms are safe once input shape is verified.
- Platform reads and job polling are read-only but can still overload a backend
  if unbounded.
- Source, enrichment, qualification, web, LinkedIn, CRM, and host jobs can
  spend credits or call providers. Preview counts and limits first.
- Sequence/source staging remains a mutation even when done concurrently.
- Publish/send/purchase operations still require explicit user confirmation
  immediately before the call.

## Canonical Operating Pattern

1. Use the narrow reference first: CRM, enrichment, web, rows, artifacts/jobs,
   workflow operations, or sequence docs.
2. If the narrow reference exposes a bulk helper, use it:
   `vf.rows.ai_analyze(mode="enrich", ...)`, `vf.rows.ai_analyze(mode="qualify", ...)`,
   `vf.crm.fetch_associations_all(...)`, or `vf.jobs.wait_many(...)`.
3. If no bulk helper exists, write one async function that creates one
   `vf.AsyncClient`, one `asyncio.Semaphore`, and one list of coroutines.
4. Start jobs first, preserve handles, then wait or stream with bounded
   concurrency. Do not start and wait one chunk at a time unless the provider
   contract requires serial behavior.
5. Collect results with enough isolation for the task. Use normal
   `asyncio.gather(...)` when one failure should stop the batch; use
   `return_exceptions=True` only when partial success is intentional and errors
   are printed/saved.
6. Materialize outputs incrementally for large artifacts: stream batches/rows,
   flush to workflow state or files, and preserve cursors for restart.
7. Verify counts and failures before moving to the next side-effect boundary.

## Data Contracts

Preserve these handles and counters in logs, workflow outputs, or report
sidecars:

| Handle | Why It Matters |
| --- | --- |
| `job_id` | Required to poll, stream, export, inspect, or retry Bridge-backed work. |
| `artifact_id` | Required for durable row paging and complete output reads; preview rows are not complete artifacts. |
| `batch_id` / `after_batch_id` | Required for restart-safe streaming of committed artifact batches. |
| `next_offset` | Required when paging artifact rows; non-advancing offsets are a data integrity error. |
| returned chunk-job count / `concurrency` | Required to explain runtime, cost, and provider pressure. The helper owns chunk sizing. |
| row counts | Required to catch silent truncation or one-chunk-only reads. |
| error list | Required when using partial success patterns. Do not hide per-chunk failures. |

For row-level helpers, remember `rows.ai_analyze(mode="enrich", ...)` and
`rows.ai_analyze(mode="qualify", ...)` return one `PendingJob` per chunk. Inspect each returned
job and its artifact before claiming the merged result is correct.

## Examples

### Correct: one async runtime, one async client

```python
import asyncio
import verbiflow as vf


async def wait_for_jobs(job_ids: list[str]) -> list[vf.PendingJob]:
    async with vf.AsyncClient() as client:
        return await client.jobs.wait_many(job_ids, concurrency=8, timeout=600)


jobs = asyncio.run(wait_for_jobs(["job_...", "job_..."]))
```

Use `asyncio.run(...)` only once at the top-level entrypoint for an async
script. Inside an existing async workflow, notebook cell, tool runtime, or async
function, call `await wait_for_jobs(...)` instead.

### Correct: hand-written bounded SDK fan-out over existing jobs

```python
import asyncio
import verbiflow as vf


async def read_job_rows(job_ids: list[str]) -> list[dict[str, object]]:
    async with vf.AsyncClient() as client:
        sem = asyncio.Semaphore(8)

        async def one(job_id: str) -> dict[str, object]:
            async with sem:
                result = await client.jobs.result(job_id)
                await result.wait(timeout=600)
                rows: list[dict] = []
                async for row in result.iter_rows(page_size=200):
                    rows.append(row)
                return {
                    "job_id": result.job_id,
                    "artifact_id": result.artifact_id,
                    "rows": rows,
                }

        return await asyncio.gather(*(one(job_id) for job_id in job_ids))
```

This reads independent job artifacts concurrently without creating a new event
loop per job. When you have `PendingJob` handles and only need status polling,
prefer `client.jobs.wait_many(...)`.

### Correct: use SDK bulk helpers instead of reimplementing chunking

```python
import verbiflow as vf

jobs = vf.rows.ai_analyze(
    rows_or_artifact=rows,
    mode="enrich",
    prompt="Return concise product launch signal fields for each row.",
    output_schema={
        "type": "object",
        "properties": {
            "signal_summary": {"type": "string"},
            "signal_date": {"type": "string"},
        },
        "required": ["signal_summary"],
    },
    concurrency=6,
    wait=True,
)
```

The sync wrapper is acceptable here because the whole call is one top-level SDK
operation. Do not put this call inside a worker pool or inside an async
function. If you are already async, call `await client.rows.ai_analyze(mode="enrich", ...)`
on `vf.AsyncClient`.

### Wrong: sync SDK calls inside a thread pool

```python
# Do not do this.
from concurrent.futures import ThreadPoolExecutor
import verbiflow as vf


def read_rows(job_id: str) -> list[dict]:
    return vf.jobs.result(job_id).wait().rows(limit=200)


with ThreadPoolExecutor(max_workers=8) as pool:
    rows_by_job = list(pool.map(read_rows, job_ids))
```

This can create one event loop per worker around sync SDK wrappers, reuse or
close shared async HTTP clients from different loops, and collide with async SDK
objects. Rebuild it as one `async def` using `vf.AsyncClient`,
`client.jobs.wait_many(...)`, `client.jobs.result(...)`, and
`asyncio.gather(...)`.

### Wrong: `asyncio.run(...)` per worker

```python
# Do not do this.
def worker(job_id: str) -> list[dict]:
    return asyncio.run(read_job_rows(job_id))
```

`asyncio.run(...)` belongs at the top-level process entrypoint, not inside each
thread, chunk, row, or callback.

### Wrong: hiding sync SDK calls in `asyncio.to_thread(...)`

```python
# Do not do this.
rows = await asyncio.gather(
    *(asyncio.to_thread(lambda job_id=job_id: vf.jobs.result(job_id).wait().rows())
      for job_id in job_ids)
)
```

`asyncio.to_thread(...)` is fine for local CPU or blocking file work. It is not
a way to make sync SDK calls safe inside async code.

## Verification

After converting a serial loop to parallel SDK work:

1. Preserve the helper's input-row/chunk summary, configured `concurrency`, and started `job_id`s.
2. Assert that every returned chunk job was started, waited, and inspected.
3. Inspect every returned job for failed/canceled/unfinished status before
   reading artifacts.
4. Compare expected row counts against artifact row counts, not just preview
   rows.
5. For workflow steps, verify the Play/report output includes the same job ids,
   artifacts, metrics, and final row counts.
6. If the work spends credits, verify the credit/cost summary through the
   owning billing or workflow reference.

## Troubleshooting

| Symptom | Likely Cause | Fix |
| --- | --- | --- |
| `Event bound to a different event loop` | Sync SDK wrappers or async SDK handles were used across threads/event loops. | Remove `ThreadPoolExecutor` and `asyncio.to_thread(...)` around SDK calls. Use one `async def`, one `vf.AsyncClient`, `asyncio.gather(...)`, and `asyncio.Semaphore`. |
| `Use AsyncClient inside an active event loop.` | A sync `vf.*` function was called from an active async runtime. | Replace the call with `await vf.AsyncClient().<namespace>.<method>(...)` or pass the existing async client down. |
| Fan-out is still serial | Code starts one job and waits before starting the next, or assumes `pipelined=True` parallelizes inner loops. | Start bounded jobs first, then use `vf.jobs.wait_many(...)` or the relevant `_all` helper. |
| 429s or provider throttling | Concurrency is too high or rate limits require pacing, not only a semaphore. | Lower concurrency; for strict per-second provider caps, add explicit async pacing around the SDK call. |
| Browser capture flakes or loses pages | Browser capture is bounded by the runtime's managed Chromium/session capacity, not generic SDK job fan-out. | Read `web.md` or `host_browser.md` and honor the documented capture concurrency rather than increasing generic job fan-out. |
| Only 20/100 preview rows show up | Code read job status preview rows instead of artifact rows. | Preserve `artifact_id` and page artifacts or use `JobResult.iter_rows(...)` / `jobs.stream_rows(...)`. |
| One chunk failed but the step claimed success | `return_exceptions=True` was used without promoting errors to output state. | Save the error list, mark the step diagnostic/failed, or rerun failed chunks deliberately. |

## Related References

- `artifacts_jobs.md` for job handles, events, streaming rows, artifact paging,
  CSV export, and `JobResult` methods.
- `row_operations.md` for `RowSet`, row qualification, and row-level AI
  enrichment helpers.
- `workflow_operations.md` for operating existing generated workflows.
- `workflow_sources.md` for checkpointed source acquisition and source
  freshness.
- `workflow_composition.md` for step design, effects, and `pipelined=True`.
- `web.md` and `host_browser.md` for runtime browser/session concurrency and
  capture limits.
- `safety.md` for credit spend, staging, publish/send, and purchase approvals.
