Metadata-Version: 2.4
Name: super-scanner
Version: 0.0.1
Summary: Extensible, orchestration-first security scanning base package
Project-URL: Homepage, https://github.com/KarAshutosh/super-scanner
Project-URL: Repository, https://github.com/KarAshutosh/super-scanner
Project-URL: Documentation, https://github.com/KarAshutosh/super-scanner#readme
Project-URL: Issues, https://github.com/KarAshutosh/super-scanner/issues
Project-URL: Changelog, https://github.com/KarAshutosh/super-scanner/releases
Author: KarAshutosh
Maintainer: KarAshutosh
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: cspm,llm-security,sast,sbom,sca,scanner,security,vulnerability
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Security
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: anyio>=4
Requires-Dist: jinja2>=3
Requires-Dist: packaging>=23
Requires-Dist: platformdirs>=4
Requires-Dist: pydantic-settings<3,>=2
Requires-Dist: pydantic<3,>=2
Requires-Dist: pyyaml>=6
Requires-Dist: rich>=13
Requires-Dist: structlog>=24
Requires-Dist: tomli>=2; python_version < '3.11'
Provides-Extra: all-python
Requires-Dist: bandit; extra == 'all-python'
Requires-Dist: cfn-lint; extra == 'all-python'
Requires-Dist: checkov; extra == 'all-python'
Requires-Dist: cvss; extra == 'all-python'
Requires-Dist: cyclonedx-python-lib; extra == 'all-python'
Requires-Dist: detect-secrets; extra == 'all-python'
Requires-Dist: guarddog; extra == 'all-python'
Requires-Dist: modelscan; extra == 'all-python'
Requires-Dist: picklescan; extra == 'all-python'
Requires-Dist: pip-audit; extra == 'all-python'
Requires-Dist: presidio-analyzer; extra == 'all-python'
Requires-Dist: schemathesis; extra == 'all-python'
Requires-Dist: semgrep; extra == 'all-python'
Requires-Dist: sslyze; extra == 'all-python'
Requires-Dist: yara-python; extra == 'all-python'
Provides-Extra: api
Requires-Dist: schemathesis; extra == 'api'
Provides-Extra: aws
Requires-Dist: prowler; extra == 'aws'
Provides-Extra: aws-boto3
Requires-Dist: boto3; extra == 'aws-boto3'
Requires-Dist: botocore; extra == 'aws-boto3'
Provides-Extra: aws-scoutsuite
Requires-Dist: scoutsuite; extra == 'aws-scoutsuite'
Provides-Extra: binaries
Provides-Extra: ci
Requires-Dist: bandit; extra == 'ci'
Requires-Dist: cfn-lint; extra == 'ci'
Requires-Dist: checkov; extra == 'ci'
Requires-Dist: cyclonedx-python-lib; extra == 'ci'
Requires-Dist: detect-secrets; extra == 'ci'
Requires-Dist: pip-audit; extra == 'ci'
Requires-Dist: semgrep; extra == 'ci'
Provides-Extra: cloud
Requires-Dist: boto3; extra == 'cloud'
Requires-Dist: botocore; extra == 'cloud'
Requires-Dist: cloudsplaining; extra == 'cloud'
Requires-Dist: policy-sentry; extra == 'cloud'
Requires-Dist: prowler; extra == 'cloud'
Provides-Extra: code
Requires-Dist: semgrep; extra == 'code'
Provides-Extra: code-all
Requires-Dist: bandit; extra == 'code-all'
Requires-Dist: semgrep; extra == 'code-all'
Provides-Extra: code-bandit
Requires-Dist: bandit; extra == 'code-bandit'
Provides-Extra: container
Provides-Extra: cvss
Requires-Dist: cvss; extra == 'cvss'
Provides-Extra: dev
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: genai
Requires-Dist: modelscan; extra == 'genai'
Requires-Dist: picklescan; extra == 'genai'
Provides-Extra: genai-guard
Requires-Dist: llm-guard; extra == 'genai-guard'
Provides-Extra: iac
Requires-Dist: checkov; extra == 'iac'
Provides-Extra: iac-lint
Requires-Dist: cfn-lint; extra == 'iac-lint'
Provides-Extra: iam
Requires-Dist: cloudsplaining; extra == 'iam'
Requires-Dist: policy-sentry; extra == 'iam'
Provides-Extra: k8s
Requires-Dist: kubernetes; extra == 'k8s'
Provides-Extra: license
Requires-Dist: scancode-toolkit; extra == 'license'
Provides-Extra: lite
Requires-Dist: bandit; extra == 'lite'
Requires-Dist: cyclonedx-python-lib; extra == 'lite'
Requires-Dist: detect-secrets; extra == 'lite'
Requires-Dist: pip-audit; extra == 'lite'
Provides-Extra: malpkg
Requires-Dist: guarddog; extra == 'malpkg'
Provides-Extra: malware
Requires-Dist: yara-python; extra == 'malware'
Provides-Extra: mobile
Requires-Dist: mobsfscan; extra == 'mobile'
Provides-Extra: network
Requires-Dist: python-libnmap; extra == 'network'
Provides-Extra: ocsf
Requires-Dist: py-ocsf-models; extra == 'ocsf'
Provides-Extra: otel
Requires-Dist: opentelemetry-sdk; extra == 'otel'
Provides-Extra: pii
Requires-Dist: presidio-analyzer; extra == 'pii'
Provides-Extra: provenance
Requires-Dist: sigstore; extra == 'provenance'
Provides-Extra: safety
Requires-Dist: safety; extra == 'safety'
Provides-Extra: sbom
Requires-Dist: cyclonedx-python-lib; extra == 'sbom'
Provides-Extra: sca
Requires-Dist: cyclonedx-python-lib; extra == 'sca'
Provides-Extra: sca-python
Requires-Dist: cyclonedx-python-lib; extra == 'sca-python'
Requires-Dist: pip-audit; extra == 'sca-python'
Provides-Extra: secrets
Requires-Dist: detect-secrets; extra == 'secrets'
Provides-Extra: securityhub
Requires-Dist: boto3; extra == 'securityhub'
Provides-Extra: sign
Requires-Dist: sigstore; extra == 'sign'
Provides-Extra: tls
Requires-Dist: sslyze; extra == 'tls'
Provides-Extra: web
Requires-Dist: zaproxy; extra == 'web'
Description-Content-Type: text/markdown

# super-scanner

> An extensible, **orchestration-first** security scanning base package for Python. `super-scanner` wraps best-in-class open-source scanning engines (Semgrep, Bandit, Trivy, Grype, OSV-Scanner, pip-audit, Checkov, detect-secrets, gitleaks, ModelScan, Prowler, …) behind **one typed plugin API** and **one normalized findings model**, so you can run an AWS CSPM sweep, a Semgrep code scan, an SCA/dependency scan, a container image scan, or a secrets scan in **three lines of Python** — and get back deterministic, deduplicated, SARIF/OCSF/CycloneDX-ready findings. **It orchestrates; it does not reimplement.**

- **Runs everywhere, sends nothing.** No telemetry, no analytics, no "phone home." Semgrep is always invoked with `--metrics=off`, and the core makes zero outbound network calls. Fully offline/air-gapped capable.
- **Base package to build on.** A lean, permissively-licensed (Apache-2.0) pure-Python core. Add engines, cloud providers, and output formats via `entry_points` — **no core changes, no forking**.
- **Everything configurable.** Layered config (`defaults < file < env < kwargs/CLI`), per-engine settings, rules locations, offline mode, fail-on thresholds, baselines, and suppressions.

> **Naming note.** "super-scanner" is a working name; the final PyPI/import/CLI name is an open decision. See `REQUIREMENTS.md` §29.

---

## Install

```bash
pip install super-scanner                 # lean core (no scanner engines)
pip install "super-scanner[code]"          # + Semgrep (SAST)
pip install "super-scanner[code-bandit]"   # + Bandit (Python SAST)
pip install "super-scanner[secrets]"       # + detect-secrets
pip install "super-scanner[sca-python]"    # + pip-audit
pip install "super-scanner[iac]"           # + Checkov
pip install "super-scanner[aws]"           # + Prowler (AWS CSPM)
pip install "super-scanner[ci]"            # curated: code+secrets+sca+container+iac
```

Some engines are **Go/native binaries** (Trivy, Grype, Syft, OSV-Scanner, gitleaks, nuclei, cosign, Prowler CLI…). Those are never shipped as Python packages — but you don't have to hunt them down: **super-scanner can install them for you** (see below). `super-scanner doctor` tells you exactly what's missing.

**Requirements:** Python ≥ 3.10. Pure-Python universal wheel; no compiled extensions in the core.

### Auto-installing scanner tools

super-scanner ships an installer driven by per-tool configs in `install_configs/` (one `<tool>.yaml` per tool, e.g. `semgrep.yaml`, `trivy.yaml`, `prowler.yaml`). It picks the best method available on your machine (pipx/pip/brew/go/script), respecting offline mode.

```bash
super-scanner setup                       # show what's installed / missing / how to get it
super-scanner install trivy grype nuclei  # install specific tools
super-scanner install --engines semgrep,osv-scanner   # install tools backing these engines
super-scanner install --all               # install every tool with a config
super-scanner install trivy --dry-run     # print the exact command, install nothing
super-scanner scan container nginx:latest --install-missing   # install then scan

# repo bootstrap (venv + package + optional tools):
scripts/bootstrap.sh --tools "semgrep trivy grype osv-scanner"
```

Each config lists methods per platform; override the method with `--prefer brew|pip|pipx|go|script`, or point at a specific binary via `[tool.super-scanner.engines.<tool>].binary_path`. Nothing is installed during a normal scan unless you pass `--install-missing`. Add your own tools by dropping a `<tool>.yaml` into an `install_configs/` directory (or set `$SUPER_SCANNER_INSTALL_CONFIGS`).

---

## Quick start — a few lines of Python

```python
from super_scanner import scan

# Static code analysis (Semgrep + Bandit)
report = scan.code("./src")
print(report.summary_line())               # critical=0 high=2 medium=1 ... total=5
report.to_sarif("out.sarif")               # SARIF 2.1.0
report.raise_for_status(fail_on="high")    # raises if any finding >= HIGH
```

```python
from super_scanner import scan

report = scan.container("nginx:latest")    # Trivy + Grype
report = scan.sca("./")                    # OSV-Scanner / pip-audit
report = scan.secrets("./")                # detect-secrets / gitleaks
report = scan.iac("./infra")               # Checkov (+ hadolint/tfsec/kics/cfn-lint)
report = scan.model("./model.pkl")         # ModelScan (malicious model files)
report = scan.aws()                        # Prowler CSPM (raises NoCredentialsError if no creds)

# M1/M2 domains — same one-call shape:
report = scan.sbom("./")                   # Syft SBOM
report = scan.k8s("./manifests")           # Kubescape / kube-bench
report = scan.license("./")                # ScanCode license compliance
report = scan.malpkg("requirements.txt")   # GuardDog malicious-package detection
report = scan.pii("./data")                # Presidio PII discovery
report = scan.malware("./", rules=["rules.yar"])   # YARA
report = scan.provenance("img@sha256:...") # Cosign signature/SLSA verification
report = scan.scm("github.com/org/repo")   # OpenSSF Scorecard

# Active scanners (require authorization for non-local targets):
report = scan.dast("http://localhost:8080")            # nuclei
report = scan.network("10.0.0.0/24", authorized=True)  # nmap
report = scan.tls("example.com", authorized=True)      # sslyze
report = scan.api("./openapi.json", authorized=True)   # schemathesis
```

### Async & streaming

```python
import anyio
from super_scanner import scan, ascan

report = anyio.run(lambda: ascan("code", "./src"))     # async
for finding in scan.code.stream("./src"):              # stream results as they arrive
    print(finding.severity.name, finding.rule.id, finding.locations[0].display())
```

### Multi-engine / multi-target builder

```python
from super_scanner import Scan

report = (
    Scan()
    .with_engines("semgrep", "bandit", "detect-secrets")
    .on("./src")
    .config(offline=True, fail_on="high")
    .options(rules=["p/owasp-top-ten", "./rules/custom.yml"])
    .run()
)
```

### Working with findings

```python
report.summary()                 # {"critical": 0, "high": 2, ..., "total": 5}
report.dedup()                   # collapse same-finding across engines (auto on by default)
report.diff(baseline)            # new vs unchanged vs absent
report.apply_suppressions(supp)  # drop suppressed findings
report.fail_on("high")           # count of findings >= HIGH
report.exit_code(fail_on="high") # deterministic CI exit code

# every output format:
report.to_json(); report.to_sarif(); report.to_cyclonedx(); report.to_asff()
report.to_gitlab(); report.to_junit(); report.to_csv(); report.to_html(); report.to_table()
report.to_ocsf()                 # AWS/Prowler findings only in M0
```

---

## Command line

```bash
super-scanner doctor                    # which engines are available + how to install missing ones
super-scanner list-engines              # JSON: every engine, status, license, integration
super-scanner scan code ./src --engines semgrep,bandit --fail-on high --format sarif --output-dir ./reports
super-scanner scan container nginx:latest --format json
super-scanner scan secrets ./ --format table
super-scanner baseline create code ./src --out .super-scanner.baseline
super-scanner scan code ./src --baseline .super-scanner.baseline --new-only --fail-on medium
super-scanner suppress --out .super-scanner-suppressions.json
super-scanner sbom ./ --format cyclonedx
super-scanner db gc                     # clean stale caches / temp roots
```

**Exit codes** (format-independent): `0` clean/within-threshold · `1` findings exceed `--fail-on` · `2` usage/config error · `3` engine execution error · `4` partial results · `5` authorization required but not attested. `--exit-zero` forces report-only.

---

## Configuring engines

Config precedence (lowest → highest): **built-in defaults → config file → environment variables → Python kwargs / CLI flags**.

Put a `[tool.super-scanner]` table in `pyproject.toml`, or a dedicated `super-scanner.toml`:

```toml
[tool.super-scanner]
offline = false
redact = true            # redact discovered secrets (on by default)
telemetry = false        # OFF by default; nothing is ever sent anywhere
fail_on = "high"

[tool.super-scanner.engines.semgrep]
enabled = true
timeout = 900
# Rules location: registry packs, local rule files, and/or rule dirs.
# Overridable per-call via scan.code(..., rules=[...]) and $SUPER_SCANNER_SEMGREP_CONFIG.
config = ["p/security-audit", "./rules/custom.yml", "./rules/org-pack/"]
offline_rules_path = "./vendor/semgrep-rules"   # used when offline = true

[tool.super-scanner.engines.semgrep.settings]
metrics = "off"          # default; forced off when offline
jobs = 4
max_memory_mb = 4000
exclude = ["tests/", "vendor/"]

[tool.super-scanner.engines.trivy]
binary_path = "/opt/bin/trivy"
db_repository = "registry.internal/trivy-db"

[tool.super-scanner.plugins]
allowlist = ["my-org-scanner==1.4.*"]    # only load allowlisted third-party plugins
require_signed = false
```

### Semgrep rules location — 4 ways, one precedence

1. **Python:** `scan.code("./src", rules=["p/owasp-top-ten", "./rules/custom.yml"])`
2. **Env var:** `SUPER_SCANNER_SEMGREP_CONFIG="p/security-audit:./rules"` (OS path-list separated)
3. **Config file:** `[tool.super-scanner.engines.semgrep].config = [...]`
4. **Default:** `p/default` online, or the pinned/vendored ruleset when offline.

### Environment kill-switches (authoritative)

| Variable | Effect |
|---|---|
| `SUPER_SCANNER_OFFLINE=true` | No network; engines run in offline mode |
| `SUPER_SCANNER_TELEMETRY=false` | Telemetry off (default) — nothing sent anywhere |
| `SUPER_SCANNER_REDACT=true` | Redact discovered secrets from all output/logs |
| `SUPER_SCANNER_NETWORK_EGRESS=false` | Subprocess engines run network-off |
| `SUPER_SCANNER_PLUGIN_ALLOWLIST="dist-a,dist-b"` | Only load allowlisted plugins |

An env value that is *safer* always wins over a config-file value.

---

## Privacy & offline use

- **No telemetry, ever, by default.** The core performs no outbound network calls. `super-scanner scan ... --offline` (or `offline=true`) guarantees engines run without network access, and forces `semgrep --metrics=off` regardless of any override.
- **Secrets are redacted.** Detected secrets are stored as SHA-256 hashes (never plaintext) and scrubbed from logs.
- **Nothing executes scanned code.** SAST/model scanning is static; the scanned model is never loaded/imported.

This makes super-scanner safe to run inside any company's air-gapped or regulated environment.

---

## Extending it — build your own scanner on top

Add an engine in your own package with **zero changes to super-scanner**:

```python
# my_pkg/engine.py
from super_scanner.engines.base import Capability, Engine, Integration
from super_scanner.model import Finding, Rule, Severity, PhysicalLocation

class MyEngine(Engine):
    id = "my-engine"
    name = "My Engine"
    domain = "sast"

    def capabilities(self):
        return Capability(target_types=frozenset({"path"}), integration=Integration.LIBRARY)

    def scan(self, target, ctx):
        yield Finding(engine=self.id, domain="sast", rule=Rule(id="my.rule"),
                      severity=Severity.HIGH, message="found it",
                      locations=(PhysicalLocation(uri=str(target)),))
```

```toml
# my_pkg/pyproject.toml
[project.entry-points."super_scanner.engines"]
my-engine = "my_pkg.engine:MyEngine"
```

`pip install my-pkg` and it shows up in `super-scanner doctor`, `scan("sast", ...)`, and everywhere else. Reporters (`super_scanner.reporters`) and cloud providers (`super_scanner.providers`) extend the same way. See `AGENT.md` and `REQUIREMENTS.md` §21 for the full plugin contract, and `super_scanner.testing` for a compliance test harness.

---

## What's covered

30 engines across 19 domains — all behind the same one-call API, all installable via `super-scanner install`.

| Domain | Engines | Extra / tool |
|---|---|---|
| SAST (code) | Semgrep, Bandit | `[code]`, `[code-bandit]` |
| SCA (deps) | OSV-Scanner, pip-audit | `[sca]`, `[sca-python]` |
| Secrets | detect-secrets, gitleaks, trufflehog | `[secrets]` |
| Container image | Trivy, Grype, Dockle | `[container]` (binaries) |
| SBOM | Syft (+ Trivy/Grype) | binary |
| IaC | Checkov, hadolint, tfsec, KICS, cfn-lint | `[iac]`, `[iac-lint]` |
| Kubernetes / host | Kubescape, kube-bench | `[k8s]` (binaries) |
| GenAI model files | ModelScan | `[genai]` |
| Cloud CSPM | Prowler (AWS), ScoutSuite (multi-cloud) | `[aws]`, `[aws-scoutsuite]` |
| License | ScanCode | `[license]` |
| Malicious packages | GuardDog | `[malpkg]` |
| PII / data | Presidio | `[pii]` |
| Malware | YARA | `[malware]` |
| Provenance | Cosign (sigstore/SLSA) | `[provenance]` (binary) |
| SCM / CI posture | OpenSSF Scorecard | `[scm]` (binary) |
| DAST / web | nuclei | `[web]` (binary) |
| Network | nmap | `[network]` (binary) |
| TLS | sslyze | `[tls]` |
| API | schemathesis | `[api]` |

Still on the roadmap: mobile (MobSF), CIEM/attack-path graphs, full cross-engine OCSF export, and live-cluster runtime (Falco). The plugin contract already supports them. See `REQUIREMENTS.md`.

## License

Apache-2.0. Wrapped engines keep their own licenses; copyleft/commercial engines are strictly optional and invoked out-of-process. See `REQUIREMENTS.md` §27 for the license inventory.
