Metadata-Version: 2.4
Name: analog-sdk
Version: 0.12.0
Summary: Python SDK for Analog — the perception layer for LLMs. Understand any website, in a format built for AIs.
Project-URL: Homepage, https://getanalog.io
Author-email: Marcus Campbell <marcus@getanalog.io>
License: MIT
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: click>=8.1
Requires-Dist: httpx>=0.27
Requires-Dist: lxml>=5.2
Requires-Dist: playwright>=1.49
Requires-Dist: protego>=0.4
Requires-Dist: pydantic>=2.6
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13
Requires-Dist: typing-extensions>=4.4
Provides-Extra: dataframe
Requires-Dist: pandas>=2.0; extra == 'dataframe'
Description-Content-Type: text/markdown

# analog-sdk

Python SDK for [Analog](https://getanalog.io) — the perception layer for
LLMs. Understand any website, in a format built for AIs.

## Installation

One line, macOS / Linux / WSL:

```bash
curl -LsSf https://getanalog.io/install.sh | sh
```

Windows PowerShell:

```powershell
powershell -ExecutionPolicy ByPass -c "irm https://getanalog.io/install.ps1 | iex"
```

The script installs [`uv`](https://docs.astral.sh/uv/) if you don't have
it, then installs Analog as an isolated `uv` tool with the `analog`
command on your PATH. Already have uv? It's just:

```bash
uv tool install analog-sdk
```

**Why uv, not pip?** Analog needs Python 3.10+, and system Pythons are
often older (macOS ships 3.9) — so a `pip`/`pipx` install against the
system interpreter can fail before you start. uv brings its own managed
Python, sidestepping the problem entirely. If you'd rather not use uv:
`pipx install analog-sdk` works when pipx has a 3.10+ interpreter to
offer, `pip install analog-sdk` works inside any Python 3.10+ virtualenv
you manage yourself, and Homebrew users can
`brew install getanalog/tap/analog`.

Analog renders every page in a real headless browser (running its JS)
before extraction, so client-side-built content — single-page apps,
infinite scroll, content that hydrates client-side — is captured by
default. The browser is built in (`analog.Browser`); its binaries
download automatically on first use (~150 MB, one-time). To fetch them
ahead of time — in CI or an agent harness, so the first call doesn't
pause — run:

```bash
analog browser install
```

Prefer a plain HTTP GET (no JS)? Pass `fetcher=HttpFetcher()`.

## Authenticate

New to Analog? `analog signup` opens the account-creation page
(invite code required during the private alpha). Then:

```bash
analog login
```

This opens your browser to sign in. The credential is minted straight
into a local store and never shown on screen, so the model driving your
session never sees it. Subsequent SDK and CLI usage picks it up
automatically; sign out any time with `analog logout`.

Signing in requires a browser on the machine running `analog login`.

Try it straight from the shell — point it at a JS-rendered page that's
tedious to scrape by hand:

```bash
analog get https://greylock.com/portfolio/
```

One call turns the rendered portfolio into structured records — every
company, with sectors, founders, and links — ready to query and export.

Run `analog --help` to see all commands (`signup`, `login`,
`logout`, `whoami`, `status`, `get`).

## Usage

```python
from analog import analog

result = analog("https://example.com")
```

Same one-liner whether or not the JS-aware fetcher is installed. When
it is, JS-heavy pages just work.

See https://getanalog.io for full documentation.

## When Analog pays for itself

Analog's economics depend on your task, not just on the page — so here
is the honest map. The fit check (`analog assess`, `result.fit`)
applies it per page; this is the reasoning behind it.

- **One quick question, once.** Your harness's fetch tool likely wins.
  Most fetch tools summarize a page through a smaller model before your
  model reads it, so their effective cost usually lands below the
  page's full markdown size — and Analog adds a browser render plus a
  backend round-trip. The fit check will often tell you to skip Analog
  here; believe it.
- **Working with the page as data.** Building a dataset by reading
  pages costs at least the full markdown of every page, plus your
  model's attention while it hand-copies values — and it leaves nothing
  behind for next time. One extraction produces typed, queryable
  records: filter, sort, export, and pull narrow slices for a small
  fraction of a full read.
- **Coming back.** The economics invert on the second touch. Every
  result is saved locally: re-opening costs no fetch and no extraction,
  a narrow query costs tokens proportional to the slice rather than the
  page, and `analog diff` compares saves without touching the network.
  A page worth returning to is worth extracting, at any size.

The numbers Analog shows are measured (the page's own markdown size).
It does not estimate other tools' costs or claim a savings figure —
the task-shaped judgment above is yours to make.

That honesty extends to other tools entirely: when a page belongs to a
platform that serves its own data feed (a Shopify store's
`/products.json`), `analog assess` verifies the feed with one
robots-respecting request and tells you what it returned just now. The
disclosure states the feed's existence, never its authority — a
platform-controlled feed can skew, and the rendered page stays your
check against it. Skip the request with `--no-probe`
(`assess(url, probe_feed=False)` in Python).

## Data flow

Extraction runs on Analog's servers. Here is what happens to a page's
content when you call `analog(url)` (or `analog get <url>`):

- **Sent to Analog.** The SDK fetches the page — by default rendering it in
  a local headless browser so client-side JavaScript runs — and sends that
  page content, plus the URL, to Analog's hosted extraction API, which
  returns the structured records.
- **Stored on your machine.** The structured records and the full-page
  markdown, saved under a handle in your local cache (pass `save=False` to
  skip; see **Saved results** below).
- **Never written to disk.** The raw HTML — it is used only for the
  extraction request and to render markdown locally, then dropped.
- **`mode="local"` sends nothing.** The page is fetched and converted to
  markdown entirely on your machine — no extraction, no backend call, no
  account needed.

By default Analog fetches as an unauthenticated visitor: it sees only what
any visitor to that URL sees, so public pages are straightforward. If you
hand it authenticated or otherwise sensitive content — by passing your own
`html=`, or a logged-in custom `fetcher` — that content is sent to the
extraction API just the same. Point Analog at content that is acceptable to
send, or use `mode="local"` to keep a page on your machine.

## Saved results

Every `analog(url)` call saves its result locally, so you can re-open or
re-export it later — without re-fetching or re-rendering the page:

```python
from analog import analog, history, latest

result = analog("https://example.com")
print(result.handle)        # "20260618-k7m2p9"

# Later — rehydrated from disk, no network:
again = latest()            # the most recent result
print(again.markdown)       # full-page markdown, stored alongside the data
for meta in history():      # everything saved, newest first
    print(meta.handle, meta.url)
```

Re-open a specific result by handle through the `results` module — a
separate import, because `from analog import analog` above rebinds the name
`analog` to the function:

```python
from analog import results

same = results.open(result.handle)   # re-open by handle (results.latest() / .history() too)
```

Artifacts live
under your per-user cache directory (`~/.cache/analog/results`;
`~/Library/Caches/analog/results` on macOS; `ANALOG_CACHE_DIR` overrides)
and store only safe derivatives — the structured result and full-page
markdown, never raw HTML. Pass `analog(url, save=False)` to skip saving;
the store is size-bounded and drops least-recently-opened results.

From the shell, `analog get <url>` saves and prints the handle, and
`analog history` / `analog open <handle>` / `analog export <handle> -f csv`
/ `analog rm <handle>` manage saved results (`latest` works anywhere a
handle does).

Exit codes are uniform across commands, so scripts can branch on `$?` by
category instead of grepping stderr: `0` success, `1` command error, `2`
usage error, `3` auth, `4` backend unreachable, `5` page fetch refused or
failed (including robots.txt refusals), `6` extraction failed. The same
table is printed in `analog --help` and on each command that can exit
non-zero with a semantic code.

### Inspect a result's fields

`analog describe <handle>` prints field-level statistics for a saved
result — per field: coverage (how many records have the field), cardinality
(how many distinct values), where it was extracted from, semantic traits,
and a sample value. It's the quick way to tell real data from decoration
before pulling records:

```bash
analog get https://greylock.com/portfolio/   # prints a handle
analog describe <handle>                       # field-stats table
analog describe <handle> -f markdown           # format: rich (default) | markdown | plain
```

Field names are **best-effort deterministic heuristics — a starting point, not
ground truth.** A semantically rich page like this one names cleanly; a thin,
div-soup page falls back to positional names like `text_2`. A positional name
means the page gave the field no usable label — not that extraction failed — so
read it as "what's here", then rename to taste in one batch, persisted to the
saved result:

```bash
analog rename-fields <handle> text_2=title text_4=site
```

Subsequent `describe` / `export` / `distinct` use the new names.

Field *names* are heuristic; field *values* are byte-faithful. **Analog
never truncates extracted text** — a value ending in `…` is the page's own
visible truncation (or the author's ellipsis), preserved exactly as
rendered. `describe` flags such fields with a `visually_truncated` trait;
the full string often sits in a `title` attribute on the page. The only
ellipsis Analog itself introduces is in *display* surfaces (`preview`'s
per-value cap), and that one is explicitly marked as a display cap with
the elided character count.

Narrow an export without writing any Python — pick columns with `--fields`,
filter/sort/limit rows with `--where` / `--sort` / `--limit`:

```bash
analog export <handle> -f csv --fields name,sector --where sector=Fintech --sort name --limit 20
```

`--where` supports `field=value`, `field!=value`, `field~value` (contains), and
the numeric comparisons `field<value`, `field<=value`, `field>value`,
`field>=value`; repeat `--where` to AND conditions.

`--sort` and the numeric `--where` are value-aware. A price field keeps its
honest display value (`"from $5.41"`, `"$1,299.00"`) yet sorts and compares by
the real number — so the ten cheapest is just a sort and a limit, and "under
$20" is a filter, no parsing:

```bash
analog export <handle> -f csv --fields name,price --sort price --limit 10
analog export <handle> -f csv --fields name,price --where "price < 20" --sort price
```

Values that carry no number (`"Call for price"`) sort last and never match a
numeric comparison. In Python, `section.numeric("price")` gives the parallel
numbers aligned with the records.

**Sponsored placements are data too.** Analog shows you the page as shipped,
and pages ship with ads — so a catalog's sponsored cards come through as
records, never silently removed. Platforms label them ("Sponsored",
"Promoted"), and those labels come through as ordinary field values
(`describe` typically shows them in a `badge`-style field). Whether ads count
as data is your call, and either call is one line:

```bash
analog export <handle> -f csv --where badge!=Sponsored   # organic only
```

`=` and `!=` are list-aware: a record whose `badge` carries several labels
matches on any one of them, and records with no badge at all pass through
untouched. The same filter in Python:

```python
organic = [r for r in section.records if "Sponsored" not in (r.get("badge") or [])]
```

For pandas work, `result.to_dataframe()` (or per-section
`section.to_dataframe()`) hands over a DataFrame with numeric fields as
real `float64` columns built from those numbers — `NaN` where a value
carried none — and everything else exactly as the records hold it. The
display strings stay untouched in `records` and every other export.
pandas is not installed by default; it ships behind an extra:

```bash
pip install "analog-sdk[dataframe]"
```

To read a single field's distinct values:

```bash
analog distinct <handle> sector      # value-counts, most frequent first
```

Compare two saved results to see what changed — added/removed fields, record
counts, and per-field coverage/cardinality shifts:

```bash
analog diff <handleA> <handleB>
```

### Pages with several areas

A page often has more than one repeating area — a storefront splits its catalog
across several product grids plus a reviews carousel; a reference page carries
several tables. Analog extracts **each** as its own section, in page order, and
`describe` lists them all (one stats table per section, with its heading).

Scope any tool to the sections you mean — `--section` (a heading label or a
0-based index) or `--kind` (every section of a kind):

```bash
analog describe <handle> --kind product             # just the product areas
analog export  <handle> -f csv --section "Solids"   # one grid, by its heading
```

This matters because pooling unlike sections conflates them — a product grid's
`name` isn't a reviews carousel's reviewer name. `distinct` and `export` refuse
to silently pool across sections of different shapes; scope them, or project
shared columns with `--fields`.

**"How many distinct products?"** is the question multiple areas exist to answer.
The catalog is spread across grids, and some products reappear in the reviews
carousel — so the honest answer is the **deduped** distinct count of an identity
field (a product URL) across the product sections:

```bash
analog distinct <handle> products_url --count --kind product
# 21 distinct products_url across 3 sections (24 total values)
```

`--count` treats the field as an identity key and counts it across sections,
deduping a value that appears in more than one — the count you'd otherwise
assemble by hand. `analog diff` is section-aware too: it matches sections across
the two results by heading / kind and diffs them like-for-like.

## Feedback

Hit a page that extracted poorly? Report it, tagged with why:

```bash
analog feedback quality https://example.com/page \
  --reason under_rendered --reason poor_field_naming \
  --note "prices render client-side"
```

The reasons come from a fixed label set (see `analog feedback quality
--help`), so reports aggregate into a prioritizable backlog — a labeled
report from whoever just watched the extraction is directly actionable.
Feature requests are free-form: `analog feedback request "your idea"`.

Submitting a quality report shares that URL with Analog — and that is
the **only** time Analog sees one. Extraction URLs are never logged, so
a report is a page volunteered, one at a time, because you want it
fixed. No page content is ever sent: just the URL, the labels, and your
note. In Python: `Client().submit_feedback(FeedbackRequest(...))`.

## License

MIT — see [LICENSE](./LICENSE).
