Metadata-Version: 2.5
Name: outcrawl
Version: 0.2.0
Summary: The Python client for Outcrawl: scrape, crawl, search and run agent browser tasks on one credit balance.
Project-URL: Homepage, https://outcrawl.ai
Project-URL: Repository, https://github.com/aeonmindai/outcrawl-browser
Project-URL: Issues, https://github.com/aeonmindai/outcrawl-browser/issues
License-Expression: MIT
License-File: LICENSE
Keywords: agent,browser-automation,crawler,outcrawl,scraping,stealth,web-search
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: httpx<1,>=0.27
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Description-Content-Type: text/markdown

# outcrawl

The Python client for [Outcrawl](https://api.outcrawl.ai): scrape, crawl, search and run agent browser
tasks, on one credit balance. Pure Python — one `py3-none-any` wheel for every operating system,
and no compiler on any of them.

```sh
pip install outcrawl
```

Requires Python 3.10 or newer. One runtime dependency, [`httpx`](https://www.python-httpx.org/),
which is pure Python, as is everything it pulls.

## Usage

```python
import asyncio
from outcrawl import Outcrawl

async def main() -> None:
    async with Outcrawl() as oc:                       # reads OUTCRAWL_API_KEY
        doc = await oc.scrape("https://example.com", formats=["markdown"])
        print(doc.markdown)                            # or doc["markdown"] — it is the wire object
        print(doc.usage)                               # every call returns what it cost

        job = oc.agent(
            task="Report the title of the top story on Hacker News.",
            caps={"budget": "3.00"},                   # the one ceiling a submit must carry
        )
        async for record in job:                       # the event cursor, while it runs
            print(record["kind"], record.get("text"))
        run = await job                                # and the settled run
        print(run["status"], run["data"])

asyncio.run(main())
```

`await` and `async for` on **one** job is one run and not two: they are separate requests against
one durable row, so watching a run and having its answer is not a choice.

## Configuration

| Variable | Meaning | Default |
| --- | --- | --- |
| `OUTCRAWL_API_KEY` | your key | required |
| `OUTCRAWL_API_URL` | base url | `https://api.outcrawl.ai` |

The same two names `@outcrawl/sdk`, the `outcrawl` CLI and the Outcrawl MCP server read. Pass
`Outcrawl(api_key=..., base_url=...)` to override them per client.

## Surface

Every one of the 34 capabilities in the registry is reachable, and `tests/test_registry.py` fails
when one is not.

- `oc.scrape(url, **options)` / `oc.crawl(url, **options)` / `oc.search(query, **options)` —
  `crawl` is async-iterable and single-pass.
- `oc.crawl.job(id)` — the same crawl by id, with `status()`, `results()`, `cancel()`, and
  `oc.crawl.get(id)` / `.results(id)` / `.cancel(id)` for an id on its own. A streaming crawl
  belongs to its connection, so hanging up cancels it: the row settles `cancelled` with every
  page it had already delivered, and these routes are how you read the pages you paid for. The
  id is on `handle.job["id"]` from the first progress frame — store it before you need it.
- `oc.agent(task=..., caps=...)` — `caps` is required at runtime and `caps.budget` is required
  inside it; `caps.steps` and `caps.duration` are optional and unset unless you name them, and
  all three are hard stops when present. The job is awaitable, async-iterable, and carries
  everything a handle does: `status()`, `results()`, `events()`, `cancel()`, `control()`,
  `answer()`, `add_file()`. `oc.agent.get(id)` and friends reach a run by id — an id from a
  webhook needs no submit. `schema` accepts a JSON Schema object and constrains `run["data"]` to
  it, the same typed submit `@outcrawl/sdk` documents.
- `oc.profiles` / `oc.secrets` / `oc.rules` / `oc.integrations` / `oc.sessions` / `oc.monitors`.
  `oc.secrets` has deliberately **no `get`**: there is no route to widen. A run reaches a value
  only by naming its HANDLE in the submit, and the substitution happens below the model.
- `oc.usage(**query)` and `oc.credits()` — every credit figure is a decimal **string**; use
  `decimal.Decimal`, never `float()`.
- `CAPABILITY_AVAILABILITY` — whether anything is SERVED behind a declared route. A capability
  nothing serves is refused in your own process, with what is missing, rather than as a 503.

## Errors

Identical to the TypeScript SDK's: same classes, same fields, same sentences. Branch on
`error.code`, never on `str(error)`.

```python
from outcrawl import ProfileInUseError, QuotaExceededError

try:
    ...
except ProfileInUseError as leased:
    retry_at = leased.held_until          # a retry that knows when to retry
except QuotaExceededError as over:
    await asyncio.sleep(over.retry_after_seconds)
```

`tests/error_parity.json` is emitted from the TypeScript SDK's own error reconstruction and read by
both test suites, so neither surface can drift from the other without turning its own suite red.

## What is not here

`oc.browser()`. The live browser needs a CDP connection driven in the caller's own process, which
is why it is deliberately not a registry capability and why the TypeScript SDK is the one place it
exists. The same page capabilities are reachable through `agent` and `scrape`, which run the loop
on our side. That is a difference in kind, not an omission.

The same reason takes `Session.take_control` with it: the TypeScript SDK's `takeControl()` opens a
CDP connection to a running session's own page so a person can drive it directly, and that
connection has to live in the caller's process for the same reason `oc.browser()` does. Everything
else on a session — `get`, `list`, `export`, `live()` — is a row read or a held stream and is here.

## Development

```sh
pip install -e '.[dev]'                  # pytest + pytest-asyncio, never runtime dependencies
python3 scripts/gen_registry.py          # regenerate registry.py from packages/core/src/registry.ts
python3 scripts/gen_registry.py --check  # or just fail if it is stale
bun ../sdk-python/scripts/gen_error_parity.ts   # regenerate the shared error contract
python3 -m pytest tests -q               # the drift, parity and ergonomics tests
```

From the repo root, `npm run test:python` runs the staleness check and the suite together on
the SUPPORTED FLOOR — `uv run --extra dev --python 3.10 pytest` — so the gate exercises 3.10
rather than whatever `python3` happens to be. Without `uv`: `pip install -e '.[dev]'` and
`python3 -m pytest tests -q`.

The two registry-versus-TypeScript tests skip when run from an unpacked sdist, which has no
TypeScript to compare against, and run whenever the monorepo is present. The reachability test —
the one that fails when a registry row has no method — has no such dependency and always runs.
