Metadata-Version: 2.4
Name: omnisim
Version: 0.2.0
Summary: OMNISIM Synthetic Users SDK — register your API and run persona-driven synthetic user simulations
License: MIT
Project-URL: Homepage, https://github.com/adityasai1234/OMNISIM
Project-URL: Documentation, https://github.com/adityasai1234/OMNISIM/blob/main/docs/synthetic-users-sdk.md
Keywords: omnisim,synthetic-users,api-testing,sdk
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: httpx>=0.26.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-httpx>=0.30.0; extra == "dev"
Provides-Extra: cli
Requires-Dist: python-dotenv>=1.0.0; extra == "cli"
Provides-Extra: browser
Requires-Dist: playwright>=1.40.0; extra == "browser"

# omnisim

Official **Python** SDK for [OMNISIM](https://github.com/adityasai1234/OMNISIM) Synthetic Users — register your staging API (OpenAPI) and run persona-driven tests against real HTTP endpoints.

## Install

```bash
pip install omnisim
```

### Web / hybrid runs

Web and hybrid runs auto-prepare **agent personas** before starting (no separate scrape step). After a successful run, a Swarm-style UX research PDF is saved to `./reports/` and opened in your browser. Use `--no-report` and `--no-open-report` to opt out.

```bash
omnisim-run run --agents 3 --scenarios explore
```

## Quick start

Set required environment variables:

```bash
export OMNISIM_API_KEY="omni_..."      # Required
export OMNISIM_BASE_URL="..."          # Required (e.g., http://localhost:8000)
```

```python
from omnisim import Omnisim

client = Omnisim.from_env()

product = client.products.register(
    name="Acme API",
    base_url="https://staging.api.acme.dev",
    openapi=open_api_dict,
    auth={"type": "bearer", "token": staging_token},
)

run = client.synthetic_users.run(
    product_id=product["id"],
    personas=50,
    scenarios=["signup"],
    mode="behavioral_only",
)

result = run.wait_for_complete()
print(result["friction_points"], result.get("recommendations"))
```

## Two layers: short map + detailed code index

**1. `app_context` (short)** — You write a brief map: routes, auth, how to navigate.  
**2. Code index (detailed)** — SDK scans the repo, uploads chunks + graph edges; at run time OMNISIM retrieves the relevant pieces (embeddings when `HF_API_TOKEN` is set on the API host).

```bash
# After register-web / register
omnisim-run index push --product-id prod_xxx --repo .
omnisim-run run --scenarios onboarding
```

Agents get both: Mapper-discovered app knowledge **and** code summaries — fewer blind `read_screen` loops, lower token burn.

## How agents know your app (Mapper + Code Index)

**No manual app description required.** When you first run 5+ agents against a web/hybrid product, OMNISIM automatically:

1. **Runs a Mapper agent** (one time only) to explore auth, routes, and APIs
2. **Saves discoveries** to a structured map (auth works via magic link, `/billing` route exists, etc.)
3. **Injects the map** into all subsequent test agents — they start informed

```python
from omnisim import Omnisim

client = Omnisim.from_env()
product = client.products.register(
    name="Acme",
    base_url="http://127.0.0.1:3000",
    app_url="http://127.0.0.1:3000",
    interaction_mode="web",  # or "hybrid"
)

# First run with >= 5 agents triggers auto-mapping
run = client.synthetic_users.run(product_id=product.id, agents=40, scenarios=["onboarding"])
# Mapper runs (~1-2 min), then 40 agents test with the saved map

# Second run: no Mapper delay, all 40 agents start with the map
run2 = client.synthetic_users.run(product_id=product.id, agents=40, scenarios=["billing"])
```

**Pre-mapping** (optional — avoids the ~1-2 min delay on first run):

```bash
omnisim-run map --product-id prod_xxx
```

**Re-map after deploy** (if routes or auth changed):

```bash
omnisim-run map --product-id prod_xxx --force
```

**Check map status:**

```bash
omnisim-run map --product-id prod_xxx --status
```

### Layer 1: Code Index (optional, CI-powered)

For deeper knowledge, push your repo structure from CI:

```bash
omnisim-run index push --product-id prod_xxx --repo .
```

This adds file-level context (Next.js routes, component locations) that the Mapper can reference but not replicate from browser exploration alone.

## Resources

- `client.products` — register / list / get / delete products (`api`, `web`, `hybrid`)
- `client.synthetic_users` — behavioral runs with poll backoff
- `client.usage` — `GET /api/v1/usage` (daily limits, tier stats)
- `client.webhooks` — `run.complete` callbacks (Pro / Pro+)
- `verify_webhook()` — HMAC verification for webhook payloads
- `capture_browser_session()` — capture authenticated Playwright session (manual login)
- `client.products.ensure_browser_auth()` — auto-attach session via staging bypass URL
- `client.products.attach_browser_session()` — attach session to web/hybrid product

## API key tiers

| Tier | Agent LLM | Browser `read_screen` |
|------|-----------|------------------------|
| Free | Groq | ARIA accessibility tree |
| Pro / Pro+ | Moonshot Kimi K2.5 | Enhanced DOM (boxes + key styles) |

Your API key tier is returned from `client.usage.get()["tier"]`. The hosted API must have `MOONSHOT_API_KEY` configured for paid-tier agent runs.

Full docs: [docs/synthetic-users-sdk.md](../../docs/synthetic-users-sdk.md)

## Browser authentication (web/hybrid products)

### Low-friction: staging bypass (CI-friendly)

Add a test-only endpoint on your staging app that sets session cookies and redirects:

```http
GET /api/test/session?token=${STAGING_AUTH_SECRET}
→ Set-Cookie session=...; Redirect /dashboard
```

Then set the full URL (with token) in CI:

```bash
export STAGING_AUTH_BYPASS_URL="https://staging.app/api/test/session?token=..."
omnisim-run run --product-id prod_xxx --agents 3
# or explicitly:
omnisim-run ensure-browser-auth --product-id prod_xxx
```

`omnisim-run run` calls `ensure_browser_auth` automatically for web/hybrid when `STAGING_AUTH_BYPASS_URL` is set. Public apps with no bypass URL are unchanged (`--no-ensure-browser-auth` to skip).

```python
client.products.ensure_browser_auth(product_id, strategy="auto")
client.icp.ensure_ready(product_id, agent_count=3, product=product)
run = client.synthetic_users.run(product_id=product_id, personas=3, scenarios=["explore"])
```

Optional product hints in `user_flows._config.browser_auth`:

```json
{ "strategy": "dev_bypass", "bypass_url": "https://staging.app/api/test/session", "wait_for_url": "/dashboard" }
```

### Manual capture (OAuth / form login)

For staging apps behind login where bypass is not available, capture once and agents reuse the session:

```bash
# Install with browser support
pip install omnisim[browser]
playwright install chromium

# Capture session (opens browser, you login, press Enter)
omnisim-run capture-session --app-url http://127.0.0.1:3000 --product-id prod_xxx
```

Or in Python:

```python
from omnisim import Omnisim, capture_browser_session

client = Omnisim.from_env()

# Capture after manual login
state = capture_browser_session("http://127.0.0.1:3000")
client.products.attach_browser_session(product_id, state)

# All subsequent runs start authenticated
run = client.synthetic_users.run(product_id=product_id, personas=3, scenarios=["explore"])
```

## Internal QA: Agent Evals (optional)

For testing the simulation quality itself — not your product — enable post-run agent evaluations:

```bash
# Install DeepEval first
pip install -r backend/requirements-eval.txt

# Run with evals (internal testing only, adds ~2-5s per agent)
omnisim-run run --product-id prod_xxx --agents 5 --evals

# Machine-readable eval results
omnisim-run run --product-id prod_xxx --agents 5 --evals --json | jq '.agent_evals'
```

Evals score both **synthetic agent quality** (task completion, persona consistency) and **finding trustworthiness** (narrative faithfulness to traces). Results attach to `result_json.agent_evals` and appear in the CLI report.

Also works with `test-product`:
```bash
omnisim-run test-product --product-id prod_xxx --scenarios explore --evals
```

See [docs/agent-evals.md](../../docs/agent-evals.md) for full documentation including prerequisites, environment setup, and troubleshooting.

## v0.2.0 highlights

- `Omnisim.from_env()` and HTTPS base URL validation
- Typed errors (`OmnisimAuthError`, `OmnisimRateLimitError`, …)
- Automatic retries on 429 / 502 / 503 / 504
- HMAC webhook signatures + `verify_webhook()`
- `RunMode` enum and exponential backoff in `wait_for_complete()`

## Development

```bash
pip install -e "packages/sdk-python[dev]"
pytest packages/sdk-python/tests/
```

## TypeScript

Optional mirror: `@omnisim/sdk` (npm). Python is canonical.
