Metadata-Version: 2.4
Name: applied-cli
Version: 0.6.10
Summary: CLI and shared client library for Applied Labs AI support agents
Author: Applied Labs
License-Expression: MIT
Project-URL: Homepage, https://github.com/AppliedLabsAI/applied-cli
Project-URL: Repository, https://github.com/AppliedLabsAI/applied-cli
Keywords: applied-labs,ai-agents,support,mcp,claude
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: httpx>=0.27.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: typer>=0.9.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"

# Applied Labs CLI

CLI and client library for Applied Labs AI support agents.

## Installation

```bash
pip install applied-cli
```

## CLI Usage

```bash
# Authenticate
applied login

# List agents
applied agents
applied agents --format json

# Find and inspect conversations
applied conversation-find --query "refund" --limit 5
applied conversations --resolution escalated --view detail --limit 10
applied conversation <id> --messages --format json

# Query tickets
applied tickets --status open

# Knowledge base
applied knowledge --type qa --search "refund"
applied knowledge <id> --format json
applied knowledge --source "https://help.example.com" --format csv
applied knowledge-diff --source "https://help.example.com"
applied content list --source "https://help.example.com"
applied content-update <content_id> --text "Corrected source text"
applied content-resync <content_id> --wait
applied knowledge-pin <id>
applied knowledge-link <response_id> --content <content_id>
applied knowledge-protect <id>
applied knowledge-unprotect <id>

# Taxonomy
applied taxonomy --type topics
applied taxonomy-counts --start 2026-03-01 --end 2026-03-18 --format csv

# Analytics
applied analytics-report --view overview_aggregate_metrics --model conversation --start 2026-04-01 --end 2026-04-30 --format json
applied analytics --group-by topic --metrics count --start 2026-04-01 --end 2026-04-30 --format json
applied analytics --group-by intent --metrics count --start 2026-04-01 --end 2026-04-30 --format json
applied metrics --metric-name conversation.resolve --start 2026-04-01 --end 2026-04-30 --period day --format json
```

`analytics-report` returns the selected report payload, not necessarily a `{ "rows": [...] }`
object. `analytics` returns grouped rows and currently supports `--metrics count`.
Raw analytics SQL is not available through the public CLI surface.

## Benchmarks & Scenarios

A **benchmark** is a named regression suite; a **scenario** is one test conversation
(built from a real `input_conversation_id`) that can belong to one or more benchmarks.
The typical loop is: build a suite → run it → review the pass rate → fix → re-run.

```bash
# Inspect benchmarks and their scenarios
applied benchmarks --agent-id <agent_id> --format json
applied benchmark <benchmark_id> --format json
applied scenarios --benchmark-id <benchmark_id> --format json

# Build a suite
applied benchmark-create --agent-id <agent_id> --name "Cancel Regression"
applied scenario-create --input-conversation-id <conversation_id> --name "<name>" \
  --benchmark-id <benchmark_id>

# Build a suite fast from several real conversations at once. Each scenario is
# named from its conversation's title (or "<prefix> N" with --name-prefix).
applied scenario-create-bulk --conversation-ids <id1>,<id2>,<id3> \
  --benchmark-id <benchmark_id>

# Port a suite to another agent (e.g. email -> chat). Cross-agent recreates the
# scenarios under the destination agent; same-agent just tags them in.
# Dry-run by default; add --apply to write.
applied benchmark-clone <source_benchmark_id> --dest-benchmark-name "Chat Regression" \
  --target-agent-id <chat_agent_id> --apply

# Run a benchmark and wait for results in one command.
# --contact-email runs as a contact that has an email, fixing
# "Email is not present in the conversation" on test conversations.
applied scenario-bulk-run --benchmark-id <benchmark_id> \
  --contact-email test@example.com --wait
applied scenario-bulk-status <job_id> --include-runs --format json

# Kill a stuck bulk run (deletes its queued/running runs; finished runs preserved)
applied scenario-bulk-cancel <job_id> --apply

# Review pass/fail health (pass_status reflects the latest run per scenario)
applied benchmark-results <benchmark_id> --format json

# Portfolio go/no-go: pass rates across all of an agent's benchmarks at a glance
applied benchmarks --agent-id <agent_id> --with-results --format json

# Rate scenarios as you evaluate
applied scenario-update <scenario_id> --pass-status pass --feedback "<note>"

# Safe delete — refuses to wipe scenarios unless you opt in
applied benchmark-delete <benchmark_id> --detach-scenarios   # preserve scenarios
applied benchmark-delete <benchmark_id> --force              # cascade delete

# Recover deleted benchmark/scenario rows from a local PITR export
applied scenario-recover-catalog --recovery-dir <dir> --apply
```

Deleting a benchmark cascades and permanently deletes its scenarios and runs, so
`benchmark-delete` refuses a non-empty benchmark unless you pass `--detach-scenarios`
(unlink the scenarios first so they survive under their agent) or `--force`.

## Library Usage

```python
from applied_cli import AppliedClient, tools

# Create client
client = AppliedClient(token="al_xxx")

# Use tools
agents = await tools.agent_list(client, output_format="csv")
conversations = await tools.conversation_query(
    client,
    filters={"resolution": "escalated"},
    view="search",
    limit=20,
)
```

## Tools

| Tool | Description |
|------|-------------|
| `agent_list` | List all AI agents |
| `conversation_find` | Find conversations with compact agent-friendly fields |
| `conversation_get` | Get single conversation with messages |
| `conversation_query` | Search/filter conversations |
| `ticket_query` | Search/filter tickets |
| `knowledge_list` | List knowledge base items |
| `taxonomy_list` | List topics, intents, and flags |
| `taxonomy_counts` | Aggregate conversation counts by topic and intent |
| `analytics_report` | Read standard dashboard/report analytics views |
| `analytics_query` | Aggregate supported conversation dimensions with count |
| `metrics_query` | Roll up named metric events |
| `benchmark_clone` | Copy all scenarios from one benchmark into another |
| `benchmark_delete` | Delete a benchmark (guards against wiping scenarios) |
| `benchmark_results` | Pass/fail/unrated tally and pass rate for a benchmark |
| `benchmark_list` | List benchmarks (with per-benchmark pass rates via `with_results`) |
| `scenario_create_bulk` | Build scenarios from several conversations at once |
| `scenario_bulk_run` | Run scenarios (contact override + wait-to-completion) |
| `scenario_bulk_cancel` | Cancel a stuck bulk run's queued/running scenario runs |

## Examples

```python
# Find conversations using the default compact search view
await tools.conversation_find(
    client,
    query="refund",
    output_format="csv"
)

# Find escalated conversations
await tools.conversation_query(
    client,
    filters={"resolution": "escalated"},
    view="detail",
    output_format="csv"
)

# Get conversation with transcript
await tools.conversation_get(
    client,
    conversation_id="abc-123",
    include_messages=True,
    message_limit=50,
    output_format="json"
)

# Search knowledge base
await tools.knowledge_list(
    client,
    kb_type="qa",
    search="refund policy",
    output_format="json"
)
```

## Development

```bash
# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Lint
ruff check --fix .
ruff format .
```

## License

MIT
