Metadata-Version: 2.4
Name: wondersearch
Version: 0.3.1
Summary: Typed synchronous and asynchronous clients for WonderSearch
License-Expression: Apache-2.0
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: httpx<1,>=0.27
Requires-Dist: packaging<27,>=24
Description-Content-Type: text/markdown

# WonderSearch for Python

Upload files and retrieve relevant passages with matching synchronous and asynchronous clients. Install the SDK from PyPI with `pip install wondersearch`.

```sh
pip install wondersearch
```

Configure `WONDERSEARCH_API_KEY` with a workspace API key from Console. The SDK connects to `https://api.wondersearch.ai` unless you set `WONDERSEARCH_BASE_URL` or pass `base_url`. `POLYGRES_API_KEY` and `POLYGRES_BASE_URL` remain supported and take precedence when both names are set. A `read_write` key can upload and change the workspace default; a `search` key retains its existing read/search permissions.

```python
from wondersearch import WonderSearch

with WonderSearch() as client:
    batch = client.upload("handbook.pdf")
    results = client.search("What is our refund policy?")
    for passage in results.results:
        print(passage.text, passage.document_id, passage.document_revision)
```

```python
from wondersearch import AsyncWonderSearch


async def search_handbook():
    async with AsyncWonderSearch() as client:
        await client.upload(["handbook.pdf", "policies.docx"])
        return await client.search("What is our refund policy?")
```

`search()` returns ranked passages and citations, not a generated answer. Empty `results` is a successful search. Every effort returns at most ten results, or fewer when fewer eligible matches exist. It defaults to the `medium` effort. Advanced options include `effort`, `folder_id`, `group_by_document`, `timeout_ms` (server search budget), and `timeout` (whole SDK operation budget). The server enforces query length, quotas and enabled efforts.

`ask()` is a deprecated alias of `search()` with the same arguments and response. It emits one `SDKDeprecationWarning` per process and will be removed in the next major version. Only `ask()` accepts the legacy `limit` argument, which is ignored and not sent to the API.

The SDK has no model-selection argument, and the API ignores a `model` request field. The response's `model` field names the WonderSearch version that served the search. `group_by_document=False` permits several matching passages from the same document.

Compare `effort="small"` and `effort="medium"` on representative questions from your collection.

## Drives

New workspaces have a selected **Default drive**. Change it on the Storage page or through the SDK:

```python
client.drives.list()
client.drives.get_default()
client.drives.set_default("DRIVE_UUID")
client.upload("handbook.pdf", drive_id="DRIVE_UUID")
client.search("Refund policy?", drive_id="DRIVE_UUID")
```

Create a drive with either client:

```python
drive = client.drives.create("Knowledge Base")
print(drive.id, drive.status)
# Async: drive = await client.drives.create("Knowledge Base")
```

`create(name, *, timeout=30, idempotency_key=None)` returns a typed `Drive`.
Names are trimmed and must contain 1-120 characters. Creation requires content
write permission. The index may still be provisioning when this call returns;
creation does not change the shared default. Use `drives.set_default(drive.id)`
to select it explicitly. Reuse an `idempotency_key` and the same name to retry
a creation whose response was lost.

Selection order is the method's `drive_id`, the client's `drive_id`, then the shared workspace default. The default is resolved once per operation and remains pinned through retries and polling. Explicit invalid drives never fall back. Changing the shared default affects subsequent calls by all workspace users. You must choose another default before deleting the selected default drive.

## Uploads and recovery

`upload()` accepts a path or a list of paths for PDF, DOCX, TXT and Markdown, each from 1 through 1,000,000,000 bytes. It hashes and streams original bytes, follows proxy or direct transfer instructions, then optionally waits for indexing. Transfer concurrency defaults to three and is configurable with `concurrency=1..50`. Processing polling starts after the transfer phase, so indexing never occupies a transfer slot. Directories are not traversed. `wait=False` returns after queuing processing; the transfer phase has a `timeout` budget of 600 seconds by default. The subsequent processing phase has a separate `processing_timeout` budget of 600 seconds. Each budget covers its whole batch, including waiting for a slot.

Uploads return an `UploadBatch`, including for one file. Each receipt includes upload, drive and workspace IDs. Searchable receipts also contain document, operation and snapshot IDs. A fresh document identity is generated by default. Use a stable `external_id` on single-file uploads to deliberately replace that document with a server-assigned revision.

```python
from wondersearch import UploadBatchError

try:
    batch = client.upload("handbook.pdf", idempotency_key="handbook-update-42")
except UploadBatchError as exc:
    print(exc.batch.files)  # Successful files are preserved.
    print(exc.idempotency_key, exc.drive_id)  # Reuse these for a retry.
    for filename, failure in exc.failures.items():
        print(filename, failure)
```

Reuse the same paths in the same order, content, metadata, drive and `idempotency_key` to retry an interrupted upload. Do not change the workspace default between those retries without passing the original explicit drive. Unfinished transfers expire after 24 hours. Durably queued processing and its storage reservation survive that deadline. Creation idempotency still expires after 24 hours; retain receipts and use `uploads.resume()` for queued work instead of recreating an old batch. SDK recovery is not a persistent upload queue.

For an upload already queued for extraction/indexing:

```python
receipt = client.uploads.resume("UPLOAD_UUID", drive_id="DRIVE_UUID")
```

If bytes have not been sealed, `resume()` raises `upload_transfer_incomplete` and instructs you to repeat `upload()` with the original operation key. It does not pretend that polling can finish transferring bytes. Cancellation stops local waiting; already accepted server work may continue. A synchronous network request already in flight may finish cleanup after its caller's deadline; outstanding transport work is bounded.

## Bulk imports

`client.imports.run()` loads a large collection of TXT and Markdown files into one drive. It requires bulk imports to be enabled for your organization; otherwise it raises `BulkImportError` with code `not_found`.

```python
from wondersearch import WonderSearch

with WonderSearch(drive_id="DRIVE_UUID") as client:
    status = client.imports.run(
        "library/",                      # a directory, a file, or a list of paths and ImportFile
        journal="library-import.jsonl",  # run again with the same journal to resume
        on_progress=lambda p: print(p.phase, p.acknowledged_bytes, p.searchable_documents),
    )
```

The SDK plans chunks from file sizes (`client.imports.plan()` shows the plan without uploading), checks the declared totals against `limits()`, creates the import, uploads and acknowledges chunks with bounded `concurrency` (default 8, at most 32), seals it and waits until it is searchable. A chunk is a 4-byte big-endian manifest length, a `wondersearch-import-chunk.v1` JSON manifest and the original file bytes; each file keeps its own manifest entry and `external_id` (its path relative to the directory, or as given, or set with `ImportFile`). Files must be non-empty UTF-8 and fit in one chunk (`chunk_bytes`, at most 32,000,000 bytes); use `upload()` for others.

- **Combining.** `combine_bytes` (0 to 1,000,000, off by default) joins neighbouring small files without metadata into one document, separated by a blank line. The combined document's `external_id` is `<first external_id>+<count>`, and its `source_files` metadata is a JSON list of `[external_id, start_byte, end_byte]`, so a passage can be traced to its file.
- **Retries.** Backpressure (`request_in_progress`) waits for the service's `Retry-After`; outages, failed transfers and missing objects retry with a fresh upload URL. The service stays authoritative for drive limits (`drive_limit_exceeded`).
- **Resume.** The journal is append-only JSON lines with IDs, sizes and checksums, never credentials, URLs or text. A rerun with the same sources, options and journal skips acknowledged chunks; a sealed import only waits. Without a journal, pass the same `import_id`. Changed sources raise `source_changed` or `journal_mismatch`.
- **Errors.** `BulkImportError` carries `import_id`, `drive_id`, `journal` and, when known, `import_status`. Exceptions raised by `on_progress` propagate unchanged.
- **Other actions.** `create`, `get`, `seal`, `wait`, `close`, `recover`, `failed_operations`, `retry_operation` and `retry_failed` expose each step. After `import_failed`, call `retry_failed()` then `wait()`. `import_revision_conflict` means a document with that `external_id` already exists or was deleted: SDK imports only create documents, so retrying cannot help.

Memory use peaks at about `concurrency` times `chunk_bytes`. UTF-8 is checked when each chunk is prepared, so an invalid file stops the import at that point (`invalid_import_file`); other file and metadata problems are rejected before anything is uploaded.

While part of an import is applied, other writes to the same drive return `temporarily_unavailable`. `upload()` defaults to `route="auto"`: a batch of at least 1,000 files or 1,000,000,000 bytes, all TXT or Markdown that fit in a chunk, with no `folder_id` or `external_id`, goes through an import when imports are enabled, and returns an `UploadBatch` with `import_status` and no receipts. Other batches upload as before. `route="imports"` always imports and `route="uploads"` never does. `upload()` rejects directories on the import route; pass them to `imports.run()`. On the import route, failures raise `BulkImportError`, the upload timeouts apply, `idempotency_key` or `journal` makes a rerun resume, a failed open import without either is closed, and each document's `external_id` is `<import_id>/<path>` (or `<import_id>/sha256:<hex>` when that exceeds 1,024 bytes), so uploading the same files again creates new documents.

## Limits

Each drive holds at most 1,000,000 documents and 100 GB of files; a single file can be up to 1 GB. Read the values the service currently enforces with `client.limits()` (or `await client.limits()`); each client fetches them at most once an hour. Treat them as guidance only: the service enforces its current limits and raises a `PolygresError` with code `drive_limit_exceeded` (HTTP 409) when a drive would grow past them. Call `client.limits(refresh=True)` after such an error; after a limit changes, cached values can be up to about two hours old.

```python
limits = client.limits()
print(limits.max_drive_documents, limits.max_drive_source_bytes, limits.max_file_bytes)
```

## Errors and citations

All API errors derive from `PolygresError` and expose `status`, `code`, `request_id`, `details` and `retry_after`. Unknown server codes remain inspectable. Upload failures retain receipts when a session exists. `UploadBatchError` retains successes and per-file errors.

Search and mutation retries use the same payload and idempotency key, with at most two retries and bounded backoff. When a retried search finds the first attempt still running (HTTP 409 `search_request_in_progress`), the SDK keeps polling with the same key until the call's `timeout`, without using up those retries. Polls wait at least the service's `Retry-After`, backing off from 0.5 s to 4 s with jitter; the last wait is shortened so a final poll lands about 0.5 s before the deadline. Transport failures while polling still count against the two retries. If the search is still running at the deadline, the SDK raises `SearchInProgressTimeoutError` (an `SDKTimeoutError`, code `search_in_progress_timeout`) with the `idempotency_key`, `request_id` and `status` of the last reply. Call `search()` again with the same question, options and `idempotency_key` to fetch the result without running the search again. Upload admission receiving `upload_limit` instead honors `Retry-After`, adds jitter and waits until the transfer deadline. A search whose replay has expired is never silently re-executed. Supplying your own `idempotency_key` lets you identify a logical search across invocations.

Search results include passage text and citation identifiers directly. New search responses include the selected `model`; historical replay responses may omit it. Snapshot IDs, cache counts and timings remain internal. `passages.get()` requires a separately known snapshot ID, such as one retained from an upload receipt. A current drive snapshot may differ from the snapshot used by a concurrent search, so it is not a substitute for that exact snapshot.

An expired snapshot raises the server error instead of returning newer content. `SearchResponse.raw` preserves additive response fields, and usage amounts retain the server's exact decimal representation.

## Versions

Package versions use semantic versioning and PEP 440 release candidates. `0.3.0` targets API `/v1` and is prepared for release. During the 0.x series, minor releases may introduce breaking changes; patch releases contain compatible fixes. From stable v1 onward, removals receive at least 90 days' notice and a migration guide.

Earlier `1.0.0rc*` versions were TestPyPI previews. Version `0.1.0` begins the production release series and sorts below those previews. When migrating, after publication explicitly install `wondersearch==0.3.0` from PyPI and change imports from `polygres_search` to `wondersearch`. Version 0.1.1 corrects the client names to `WonderSearch` and `AsyncWonderSearch`. Update imports and constructors from `PolygresSearch` and `AsyncPolygresSearch`; the old names are not exported. This initial branding correction is an exception to the patch compatibility policy.

Version 0.2.0 stops sending a model selector; the service chooses the search model.
Search returns at most ten results, and the legacy `limit` argument is ignored.
The server enforces the active model's query limit. Retry an uncertain search
with the same SDK version, request options, and idempotency key.

Version 0.1.2 makes an additional exception to the patch compatibility policy: `upload(timeout=...)` now covers transfers rather than the complete upload-and-processing operation. Set `processing_timeout` explicitly and allocate both budgets when enforcing an overall deadline.

The SDK sends its version on API requests (`X-Polygres-SDK-Version` and `X-WonderSearch-SDK-Version`) and processes compatibility metadata from the API. Unsupported API majors or explicitly unsupported SDK versions raise `SDKCompatibilityError`. Supported older versions receive `SDKUpgradeWarning`; deprecations use `SDKDeprecationWarning`.

From 0.3.0 the SDK also reads notices from every API response:

- `Deprecation`, `Sunset` and `Link: <...>; rel="deprecation"` headers, and entries in a response's top-level `deprecations` list (also on `SearchResponse.deprecations`), emit `SDKDeprecationWarning` naming the replacement, retirement date and migration guide.
- `WonderSearch-Client-Notice: upgrade; latest=<v>` emits `SDKUpgradeWarning`; `deprecation; minimum=<v>` emits `SDKDeprecationWarning`. Both include the exact `pip install --upgrade` command.
- HTTP 426 `client_upgrade_required` raises `ClientUpgradeRequiredError` (an `SDKCompatibilityError`) with `minimum_version`, `latest_version` and `upgrade_command`. HTTP 410 `api_version_retired` raises `APIVersionRetiredError` with `replacement`, `docs_url` and `sunset_at`.

Warnings occur at most once per notice per process and can be controlled with Python warning filters or `version_warnings=False`. Structured notices remain available at `client.notices`. A library must not reinstall itself inside an application, so the SDK only notifies: there is no automatic upgrade or import-time PyPI request.

## Development and release

Use a project virtual environment. Run `pytest`, `ruff check`, and `mypy src`, then `python -m build --no-isolation`. Build tooling includes Hatchling, build, pytest, Ruff and mypy. The wheel needs only HTTPX and packaging plus their transitive dependencies.

Do not publish until package ownership, public metadata and package contents are approved. The release manifest must leave `latest_stable_sdk_version` null until a corresponding stable release actually exists. Deployment and package publication are separate actions. Python 3.10+ is the target; report the actual tested platforms and interpreter versions for each release.

The supplied `tox.ini` exercises installed wheels on Python 3.10 through 3.14.
Run this matrix on Linux, macOS and Windows before stable publication; missing
interpreters are failures rather than silently skipped. Tox is optional release
tooling.

## License

Licensed under [Apache-2.0](LICENSE), matching the Polygres CLI and SDK.

### Bulk uploads in 0.1.2

Multi-file uploads automatically use bulk requests when the API advertises support. Reservation and processing-status requests contain at most 50 files; byte transfers still obey `concurrency` (default 3, maximum 50). Capacity is enforced per drive by the server. Extraction and indexing do not consume transfer slots. `wait=False` returns only after source bytes have been verified and durably queued.

Use a stable `idempotency_key` and the same ordered file list when retrying an interrupted call. Reordering the list changes the derived per-file identities. Keep returned receipts and `UploadBatchError` failures for recovery. This SDK does not persist a local recovery manifest. Explicit single-file replacements use the existing endpoint and replacement quota rules; bulk admission reserves each new file’s full source size.
