Metadata-Version: 2.5
Name: scrub.bot
Version: 0.1.1
Summary: Clean messy data without guessing: a CLI and local engine with a change log for every edit.
License-Expression: LicenseRef-BSL-1.1
License-File: LICENSE
Requires-Python: >=3.12
Requires-Dist: alembic>=1.18.4
Requires-Dist: arrow>=1.4.0
Requires-Dist: beautifulsoup4>=4.12
Requires-Dist: boto3>=1.43.19
Requires-Dist: dateparser>=1.4.0
Requires-Dist: duckdb>=1.5.3
Requires-Dist: faker>=37.0.0
Requires-Dist: fastapi>=0.136.3
Requires-Dist: fastmcp<4,>=3.3.1
Requires-Dist: ftfy>=6.3.1
Requires-Dist: geoip2>=5.2.0
Requires-Dist: httpx>=0.28.1
Requires-Dist: jsonschema>=4.26.0
Requires-Dist: manuf>=1.1.5
Requires-Dist: mutagen>=1.47.0
Requires-Dist: num2words>=0.5.14
Requires-Dist: openpyxl>=3.1.5
Requires-Dist: phonenumbers>=9.0.31
Requires-Dist: pillow>=12.2.0
Requires-Dist: presidio-analyzer>=2.2.362
Requires-Dist: presidio-anonymizer>=2.2.362
Requires-Dist: psycopg[binary]>=3.3.4
Requires-Dist: pyarrow>=24.0.0
Requires-Dist: pydantic-settings>=2.14.1
Requires-Dist: pydantic>=2.13.4
Requires-Dist: pypdf>=6.0.0
Requires-Dist: python-docx>=1.2.0
Requires-Dist: python-multipart>=0.0.29
Requires-Dist: python-pptx>=1.0.2
Requires-Dist: python-whois>=0.9
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: redis>=5.2.0
Requires-Dist: rich>=14.0.0
Requires-Dist: rq>=2.9.0
Requires-Dist: schwifty>=2026.3.0
Requires-Dist: sentry-sdk[fastapi]>=2.61.1
Requires-Dist: spacy>=3.8.14
Requires-Dist: sqlalchemy>=2.0.50
Requires-Dist: stripe>=15.2.0
Requires-Dist: tiktoken>=0.13.0
Requires-Dist: timezonefinder>=8.2.4
Requires-Dist: typer>=0.25.1
Requires-Dist: ua-parser>=1.0.0
Requires-Dist: usaddress>=0.5.16
Requires-Dist: uvicorn[standard]>=0.48.0
Requires-Dist: validators>=0.35.0
Provides-Extra: ai
Requires-Dist: detoxify>=0.5.2; extra == 'ai'
Requires-Dist: langdetect>=1.0.9; extra == 'ai'
Requires-Dist: openai-whisper>=20240930; extra == 'ai'
Requires-Dist: pytesseract>=0.3.13; extra == 'ai'
Requires-Dist: torch>=2.12.0; extra == 'ai'
Requires-Dist: transformers>=5.9.0; extra == 'ai'
Provides-Extra: dev
Requires-Dist: httpx>=0.28.1; extra == 'dev'
Requires-Dist: mypy>=2.1.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=1.3.0; extra == 'dev'
Requires-Dist: pytest>=9.0.3; extra == 'dev'
Requires-Dist: ruff>=0.15.14; extra == 'dev'
Provides-Extra: grpc
Requires-Dist: grpcio-tools>=1.76.0; extra == 'grpc'
Requires-Dist: grpcio>=1.76.0; extra == 'grpc'
Provides-Extra: server
Description-Content-Type: text/markdown

# scrub.bot

Command-line client and Python client for ScrubBot, a data operations API that cleans, validates, transforms, classifies, and scores records. Every run returns the cleaned records, a change log of what was modified, and a list of anything that could not be resolved. Built-in handlers are deterministic: same input, same output.

The package contains the `scrub` command and the engine behind it. `scrub serve` starts the engine on your machine with SQLite and no other services, so nothing leaves your computer. The same CLI can also point at a hosted or self-hosted ScrubBot API. AI model handlers are an optional extra: `pip install "scrub.bot[ai]"`.

## Install

```bash
pip install scrub.bot
```

Installs two commands that are the same program: `scrub` (short, for daily use) and `scrubbot` (explicit, for docs and scripts).

## Start the engine

```bash
scrub serve                      # local API on http://127.0.0.1:8000, prints a key
```

In another terminal:

```bash
export SCRUBBOT_API_URL=http://127.0.0.1:8000
scrub api doctor                 # check key, endpoint, and version
curl -sO https://www.scrub.bot/demo/contacts.csv
scrub clean/text/whitespace contacts.csv -o contacts.clean.csv
```

The CLI provisions its own key against a local server. To use a hosted or self-hosted ScrubBot API instead, set `SCRUBBOT_API_URL` and `SCRUBBOT_API_KEY`. Hosted accounts are not open yet.

## Run a handler

```bash
scrub handlers pii                              # search the catalog
scrub handlers clean/pii                        # show one handler and its config
scrub clean/pii customers.csv --config strategy=mask
scrub clean/text/whitespace notes.csv
scrub runs                                      # recent run history
```

`scrub clean/pii file.csv` is shorthand for `scrub run clean/pii file.csv`. Input can be CSV, JSON, or inline text. Output goes to stdout or `--output`.

## Run a pipeline

`prep.yaml`:

```yaml
name: customer_prep
handlers:
  - handler: clean/text/whitespace
    config: {}
  - handler: clean/pii
    config: {strategy: mask}
  - handler: validate/format/email
    config: {}
```

```bash
scrub pipeline prep.yaml customers.csv
```

## Python

```python
import scrubbot

c = scrubbot.Client(api_key="sk_...", base_url="https://api.scrub.bot")

result = c.run_handler("clean/pii", data=[{"email": "a@b.com"}], config={"strategy": "mask"})
print(result["records"], result["changes"])

for event in c.stream_pipeline(
    record_chunks=[[{"text": "  hi  "}]],
    handlers=[{"handler": "clean/text/whitespace", "config": {}}],
):
    print(event)
```

Errors raise `scrubbot.ApiError` (HTTP status and payload) or `scrubbot.ApiConnectionError` (unreachable or timed out). The package also ships the generated, fully typed client classes (`scrubbot.HandlersApi`, `scrubbot.PipelineApi`, `scrubbot.JobsApi`, and so on) for callers who want a model per endpoint.

## Environment variables

| Variable | Meaning |
|----------|---------|
| `SCRUBBOT_API_KEY` | API key; overrides the saved login |
| `SCRUBBOT_API_URL` | API base URL; the default today is `http://localhost:8000`, so set this to the hosted API |
| `SCRUBBOT_DEFAULT_FORMAT` | Output format for runs |
| `SCRUBBOT_TIMEOUT` | Request timeout in seconds |

## Self-hosting and license

The server is in the same repository and runs with Docker Compose. It is source-available under the Business Source License 1.1 and converts to Apache 2.0 on 2030-01-01. Internal use is permitted; offering it as a hosted service to third parties needs a commercial license. This client package follows the same license.

Project: https://github.com/scrubbot/scrubbot. Docs: https://scrub.bot/docs.
