# Web Research

<!-- BEGIN GENERATED: guide-map -->
## Guide map

Read only the section needed for the next decision. Line ranges are exact
for this generated runtime target.

| Section | What it covers | Lines |
| --- | --- | --- |
| [Purpose](#purpose) | Scope, ownership, and the central operating question. | 27-31 |
| [Use This When](#use-this-when) | Tasks and conditions that belong in this guide. | 32-37 |
| [Do Not Use This For](#do-not-use-this-for) | Boundaries and the correct routes for excluded work. | 38-47 |
| [Choose A Path](#choose-a-path) | Decision hierarchy for selecting the right capability or provider. | 48-55 |
| [Primary SDK Paths](#primary-sdk-paths) | Operating guidance and contracts for this section. | 56-64 |
| [Supporting References](#supporting-references) | Capability, provider, policy, and runtime guides for this outcome. | 65-74 |
| [Mental Model](#mental-model) | Concepts and state boundaries needed for correct operation. | 75-91 |
| [Preflight](#preflight) | State, authority, inputs, and readiness to verify before execution. | 92-104 |
| [Risk And Side Effects](#risk-and-side-effects) | Effects, approvals, cost, privacy, and retry boundaries. | 105-114 |
| [Canonical Operating Pattern](#canonical-operating-pattern) | The normal end-to-end execution sequence. | 115-127 |
| [Data Contracts](#data-contracts) | Required identities, fields, evidence, handles, and durable shapes. | 128-141 |
| [Verification](#verification) | Evidence required before reporting success. | 142-149 |
| [Troubleshooting](#troubleshooting) | Failure diagnosis and corrective routes. | 150-160 |
| [Related References](#related-references) | Adjacent boundaries and when to cross them. | 161-170 |
<!-- END GENERATED: guide-map -->

## Purpose

Research public or rendered web sources while retaining enough evidence to
reproduce and review the result.

## Use This When

Use this guide for explicit URLs, web search, rendered pages, screenshots,
structured page data, or provider-session evidence. Prefer structured SDK
sources when they directly own the requested fact.

## Do Not Use This For

| Need | Owning Guide |
| --- | --- |
| A fact owned by a structured company, contact, funding, job, CRM, or provider source | The matching outcome or provider guide |
| Generic browser or app testing unrelated to GTM evidence | The host's browser automation tools |
| Transform or publish rows already acquired | `row_operations.md` and `artifacts.md` |
| Store raw credentials, cookies, or session secrets in artifacts or model context | Never do this |
| Turn research directly into send, publish, purchase, or other activation | The relevant activation and safety guides |

## Choose A Path

1. Use public web operations for accessible pages and search results.
2. Use managed browser capture for rendered, interactive, or session-bound
   evidence.
3. Save large HTML, Markdown, screenshots, and network captures as artifacts.
   Keep only bounded facts and paths in model context.

## Primary SDK Paths

<!-- BEGIN GENERATED: primary-sdk-paths -->
| Primary function | Choose it when | Required inputs | Returns | Exact contract |
| --- | --- | --- | --- | --- |
| `vf.web.search_start` | Use to discover relevant public pages from a query under bounded provider work and credit tracking. | `queries` | `pending_artifact_job` | [`web.md`](web.md) · installed SDK: `verbiflow/web.py:16-85` |
| `vf.host.web.capture_pages` | Use when URLs are already known and rendered HTML, Markdown, screenshots, network evidence, or structured page data must be captured. | `output_dir`, `urls` | `host_page_capture` | [`web.md`](web.md) · installed SDK: `verbiflow/host.py:1488-2012` |
<!-- END GENERATED: primary-sdk-paths -->

## Supporting References

Open only the child that resolves the next material decision.

| Reference | Use it for | Kind |
| --- | --- | --- |
| [`web.md`](web.md) | Public web search, page extraction, screenshots, Markdown, and browser evidence. | capability |
| [`parallelism.md`](parallelism.md) | Bounded consolidation, concurrency, retries, deadlines, and no-progress guards. | policy |
| [`host_browser.md`](host_browser.md) | Managed browser capture and provider-session evidence across Local and Hosted runtimes. | runtime |

## Mental Model

Web research has two products. The evidence capture preserves what the source
showed. The normalized finding states what was extracted or inferred from that
capture. Keep them linked but distinct.

| Object | Meaning |
| --- | --- |
| Requested URL or query | The user or program intent before redirects and discovery. |
| Final source | The canonical URL, response state, and observation time actually inspected. |
| Capture | Saved HTML, Markdown, screenshot, structured data, or bounded network evidence. |
| Finding | A normalized fact or inference citing the capture and relevant range. |

A rendered page is not automatically more authoritative than a structured
source. Choose browser capture because rendering or session state is necessary,
not because it is available.

## Preflight

1. Define the claim, entity, source authority, freshness requirement, expected
   output fields, and evidence needed to review it.
2. Prefer the narrowest structured owner. Use web research only when the web is
   the source of truth or fills a named evidence gap.
3. Classify the source as public/static, rendered, interactive, or session-bound
   and choose the matching child guide.
4. Bound URLs, pages, redirects, retries, concurrency, capture size, deadline,
   and no-progress behavior.
5. Decide which raw evidence must be saved and which bounded facts may enter
   model context.

## Risk And Side Effects

| Stage | Risk | Rule |
| --- | --- | --- |
| Public search or page read | Network/provider work | Bound results and retain final URLs, status, and observation time. |
| Rendered or session-bound capture | Managed browser capacity and provider-session access | Use the narrowest managed profile, preserve handles, and never expose raw credentials. |
| Save HTML, screenshots, or network evidence | Potentially large or sensitive artifacts | Save exact bytes durably; keep large bodies and secrets out of chat and logs. |
| Parse or summarize evidence | Extraction error or unsupported inference | Cite the saved source/range and label inference separately. |
| Act on a finding | Customer-visible mutation | Leave this guide for the owning activation and safety reference. |

## Canonical Operating Pattern

1. Record the research claim, source authority, freshness rule, and output schema.
2. Choose public web or managed browser capture through the existing hierarchy.
3. Inspect one representative source and confirm final URL, status, rendered
   shape, and evidence availability.
4. Run the bounded capture program, saving raw evidence and structured metadata
   outside model context.
5. Normalize facts with source/range lineage, preserve contradictions and errors,
   and label inference.
6. Verify representative captures and materialize the final findings and evidence
   artifacts.

## Data Contracts

Each capture should retain, when available:

- requested URL or query, final canonical URL, redirects, status, and capture time;
- capture mode, source title, content type, and durable artifact/path identity;
- structured data and the exact saved range used for each extracted claim;
- error, blocked, partial, login-required, and stale states;
- no raw cookies, access tokens, passwords, or unbounded payloads.

Each finding should retain entity identity, normalized value, fact or inference
classification, source/capture identity, relevant range, observation time,
confidence, contradiction, and limitation.

## Verification

Verify the final URL, capture time, source status, structured evidence, and the
specific saved range used for each claim.

Re-open representative captures, confirm the cited ranges support the findings,
inspect errors and contradictions, and verify the exact final artifact bytes.

## Troubleshooting

| Symptom | Likely Cause | Next Step |
| --- | --- | --- |
| The page is empty or missing client-rendered content | A static path was used for a rendered source. | Move only that source to the managed browser path. |
| The capture requires login | The source is session-bound or the profile is not ready. | Use the managed session flow in `host_browser.md`; never request raw cookies. |
| Context fills with repeated HTML or tool output | Raw captures were printed instead of saved. | Persist the capture and inspect only the relevant saved range. |
| A finding cannot be traced to a source | Normalization discarded capture lineage. | Rebuild the finding with final URL, capture ID/path, and exact range. |
| Repeated requests stop making progress | Pagination, redirects, or retries have no guard. | Preserve partial evidence and stop at the deadline/no-progress boundary. |
| Two sources disagree | Entity identity, time, or source authority differs. | Preserve both findings, resolve the conflict explicitly, and label uncertainty. |

## Related References

| Next Decision | Read |
| --- | --- |
| Public search, extraction, screenshots, or structured web rows | `web.md` |
| Rendered, interactive, or session-bound browser evidence | `host_browser.md` |
| Discover or enrich companies and contacts | The matching search or enrichment outcome guide |
| Transform findings and publish artifacts | `row_operations.md` and `artifacts.md` |
| Use web evidence inside a repeatable Play | `workflows.md` and `workflow_sources.md` |
| Apply approval boundaries before activation | `safety.md` |
