Metadata-Version: 2.5
Name: giskard-scan
Version: 1.0.1
Summary: Agent vulnerability scanner — red teaming, prompt injection, adversarial scenario generation
Author-email: Giskard R&D <rnd@giskard.ai>
License: Apache Software License 2.0
Requires-Python: >=3.12
Requires-Dist: giskard-agents<2,>=1.0.3
Requires-Dist: giskard-checks<2,>=1.0.5
Requires-Dist: huggingface-hub<2,>=1.11.0
Requires-Dist: numpy<3,>=2.4.1
Provides-Extra: deepteam
Requires-Dist: deepteam<2,>=1.0.7; extra == 'deepteam'
Requires-Dist: standard-nntplib>=3.13; (python_version >= '3.13') and extra == 'deepteam'
Provides-Extra: garak
Requires-Dist: garak<1,>=0.15; extra == 'garak'
Description-Content-Type: text/markdown

# giskard-scan

Agent vulnerability scanner — red teaming, prompt injection, adversarial scenario generation.

**Telemetry:** This package depends on `giskard-core`, which may send optional,
aggregated scan lifecycle metrics (scan type, fixed execution modes, counts,
durations, and outcomes). Descriptions, language values, generated scenarios,
prompts, outputs, recommendations, and other user content are never included.
See [Telemetry](../giskard-core/README.md#telemetry) for details and opt-outs.

## Scan entrypoints

`quality_scan` and `vulnerability_scan` share the same explicit execution options.
Pass shared settings by keyword on either entrypoint; each scan adds scan-specific
options (`knowledge_base` for quality, `commercial_use` for vulnerability).

```python
from giskard.scan import quality_scan, vulnerability_scan


async def echo(inputs: str) -> str:
    return inputs


vulnerability_result = await vulnerability_scan(
    target=echo,
    description="Customer support chatbot for an e-commerce store.",
    languages=["en"],
    max_scenarios=20,
    seed=42,
    group_by="threat-type",
    parallel=True,
    max_concurrency=8,
    return_exception=False,
    target_mode="multiturn",
    commercial_use=False,
)

quality_result = await quality_scan(
    target=echo,
    description="Customer support chatbot for an e-commerce store.",
    languages=["en"],
    knowledge_base=["Paris is the capital of France."],
    max_scenarios=20,
    seed=42,
    group_by="component",
    parallel=True,
    max_concurrency=8,
    return_exception=False,
    target_mode="multiturn",
)
```

## Shared defaults

`DEFAULT_TARGET_MODE` (`"multiturn"`) is shared by `generate_suite`,
`vulnerability_scan`, `quality_scan`, and `third_party_scan`. Pass
`target_mode="singleturn"` to skip multi-turn generators/attacks and cap turn
budgets to 1.

Discover selectable items for any tool with `list_scan_items`:

```python
from giskard.scan import list_scan_items

list_scan_items("giskard")  # scenario generator class names
list_scan_items("garak")  # active garak probe names
list_scan_items("deepteam")  # vulnerability + attack names
```

## Third-party scanners (experimental)

`third_party_scan` runs an external security scanner against a Giskard target and
returns a `SuiteResult`. Two scanners are supported, each shipping as an optional
extra: [garak](https://github.com/NVIDIA/garak) and
[deepteam](https://github.com/confident-ai/deepteam).

```bash
pip install giskard-scan[garak]
pip install giskard-scan[deepteam]
```

### garak

```python
import asyncio

from giskard.scan import list_scan_items, third_party_scan


def target(inputs: str) -> str:
    # Your agent / model call. Structured (BaseModel) inputs also work.
    return call_my_agent(inputs)


result = asyncio.run(
    third_party_scan(
        target,
        tool="garak",
        description="A helpful assistant",  # required by the API; garak ignores it
        # probes=None -> curated default set; probes="all" -> every active probe
        probes=["probes.goodside.ThreatenJSON"],
        # target_mode defaults to "multiturn"; pass "singleturn" to drop iterative probes
    )
)

print(result)
print(list_scan_items("garak")[:5])
```

Probes run in parallel; the target is invoked concurrently, so it must be safe to
call from multiple threads (per-conversation state is tracked in the `Trace`, not on
the target).

Omitting `probes` runs a small curated default set aligned with DeepTeam themes
(Bias/Toxicity, PII, Misinformation, PromptLeakage, jailbreak/injection, data
exfil) — not the full garak catalog. Pass `probes="all"` for every active probe,
or an explicit name list.

Unknown, inactive, or unloadable probe names are logged and emitted as
``CheckResult.skip`` scenarios rather than raising.

### deepteam

Deepteam generates adversarial attacks with an LLM and judges the responses with an
LLM, so it needs a working Giskard default generator (see
`giskard.checks.get_default_generator()`) — there is no keyless mode.

```python
result = asyncio.run(
    third_party_scan(
        target,
        tool="deepteam",
        description="A helpful assistant",  # becomes deepteam's target_purpose
        vulnerabilities=["Bias", "Toxicity"],  # omit for a curated default set
        attacks=["PromptInjection", "LinearJailbreaking"],  # omit for defaults
        attacks_per_vulnerability_type=1,  # default; each vuln subtype × this many
        # target_mode defaults to "multiturn" (shared with native Giskard scans)
    )
)
```

`vulnerabilities` accepts `Bias`, `Toxicity`, `PIILeakage`, `PromptLeakage`, and
`Misinformation`. Instantiating those classes without subtypes runs **all** of
their types (for example Bias → race, gender, politics, religion), so cost scales
with types × `attacks_per_vulnerability_type` × attacks.

`attacks` accepts the single-turn `PromptInjection`, `Roleplay`, `Leetspeak`, and
`ROT13`, plus the multi-turn `LinearJailbreaking`, `CrescendoJailbreaking`,
`TreeJailbreaking`, `SequentialJailbreak`, and `BadLikertJudge`. Unrecognized
names are logged and emitted as skip scenarios (valid names in the same call still
run).

`target_mode` defaults to the shared `DEFAULT_TARGET_MODE` (`"multiturn"`).
Pass `"singleturn"` to drop multi-turn attacks (they surface as skip scenarios).
Unknown names also skip — the suite is never an empty "everything passed"
result when you asked for attacks that could not run.

### API keys and LLM-judge detectors

Some garak detectors need an LLM or a third-party API to score a probe:

- **LLM-judge detectors** (garak's `judge.*`, e.g. refusal detection) normally require
  their own OpenAI key. Instead, they are automatically backed by Giskard's default
  generator (`giskard.checks.get_default_generator()`), so they run with the same
  credentials as the rest of Giskard — no separate OpenAI key needed.
- **Detectors that need a third-party API key** you have not set (for example
  `perspective.*`, which needs `PERSPECTIVE_API_KEY`) are **skipped** rather than
  silently dropping the whole probe. Each skipped detector surfaces as a skip result
  (`CheckResult.skip`) in the returned `SuiteResult`, with the missing key named in the
  message, so the rest of the probe's detectors still run.
