Metadata-Version: 2.4
Name: aegis-pentest
Version: 0.10.0
Summary: AEGIS — Adaptive Engagement & Generic Inspection Scanner
Author-email: Majd Bnat <hey@majdb.com>
License-File: LICENSE
Requires-Python: >=3.12
Requires-Dist: aiosqlite>=0.20
Requires-Dist: anthropic>=0.40
Requires-Dist: anyio>=4.0
Requires-Dist: dnspython>=2.6
Requires-Dist: fastmcp>=0.4
Requires-Dist: httpx>=0.27
Requires-Dist: jinja2>=3.1
Requires-Dist: packaging>=24.0
Requires-Dist: pydantic>=2.0
Requires-Dist: python-dateutil>=2.9
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: sqlmodel>=0.0.21
Requires-Dist: structlog>=24.0
Requires-Dist: typer>=0.12
Description-Content-Type: text/markdown

# aegis · v0.9.0

**Adaptive Engagement & Generic Inspection Scanner**

An autonomous, AI-orchestrated web penetration testing agent. Aegis handles reconnaissance, fingerprinting, vulnerability discovery, and verification so the human pentester focuses on what actually requires human judgment.

---

## What it is

Aegis runs structured, methodology-driven engagements against web targets. It profiles the host environment, fingerprints the target stack, selects relevant tools and tests from the PTES + OWASP WSTG v4.2 playbook, executes them concurrently, and produces a verified findings report with remediation guidance.

It is not a Burp Suite replacement. It is the autonomous recon and vuln-discovery layer that feeds the human pentester, eliminating roughly 70% of routine busywork.

**Hard constraints built into the runtime:**

- Every engagement requires a signed `scope.yaml` before any network egress.
- Every outbound request is matched against in-scope and out-of-scope rules before the socket opens.
- Scope violations abort the current task and are written to the audit log.
- There is no `--force` flag that bypasses scope.

---

## What's new in v0.9.0

| Feature | Description |
|---|---|
| **Dockerfile** | Kali-rolling base image. Installs all 84 catalog tools in layers: apt (Kali packages), Go, binary releases, pipx, git-cloned. One `aegis docker build` → fully armed container. |
| **`aegis docker build/run/shell/pull/status`** | Full Docker sub-app. `run` mounts the engagement dir, wires `ANTHROPIC_API_KEY`, adds `NET_RAW` cap for nmap/masscan. `status` runs a live tool check inside the container. |
| **`docker-compose.yml`** | `aegis`, `aegis-shell`, `aegis-serve` services with `network_mode: host`, `NET_ADMIN`/`NET_RAW` caps, volume mounts for engagements + config. |
| **`scripts/install.sh`** | Universal one-liner installer. Auto-detects Arch, Kali, Ubuntu/Debian, Fedora, macOS. Installs via native package manager, then Go tools, pipx tools, binaries, and git-cloned tools. `--docker` flag builds the image instead. |
| **`.dockerignore`** | Excludes engagements/, dist/, secrets, DBs from build context. |

## What's new in v0.8.0

| Feature | Description |
|---|---|
| **Tool catalog overhaul** | 84-entry `TOOL_CATALOG` with typed `InstallSpec` for every tool (Go, pipx, pacman, AUR, binary, git). `install_command(name)` returns the exact shell command for any tool. |
| **`aegis env install`** | New CLI command that detects missing tools, groups them by install method, and installs in one shot. Supports `--dry-run`, `--missing`, `--category` filters. |
| **`aegis env show / tools / refresh`** | Proper `env` sub-app replacing the stub. `aegis env tools --missing` shows exactly what to install with the command. |
| **Profiler auto-sync** | `PENTEST_TOOL_BINARIES` is now built from `TOOL_CATALOG` at import time — catalog and profiler can never diverge. |
| **74 tests** | +53 new tests covering PTT graph (11 tests), hallucination guard (10), PoC generator (8), tool catalog (11), env profiler (8). Zero failures, zero warnings. |
| **Dependency fix** | `datetime.utcnow()` → `datetime.now(timezone.utc)` throughout server.py. |
| **51 tools installed** | dalfox, alterx, shuffledns, puredns, subjack, subzy, crlfuzz, gau, waybackurls, wafw00f, sslyze, dnstwist, dnsrecon, commix, xsstrike, dirsearch, paramspider, bbot, trivy, grype, syft, smbmap, massdns, theharvester, sstimap, findomain, trufflehog, semgrep + more. |

## What's new in v0.7.0

| Feature | Description |
|---|---|
| **Pentesting Task Tree (PTT)** | Persistent aiosqlite-backed directed graph of the full attack surface. Nodes: domain, IP, port, endpoint, hypothesis, finding, task, PoC. Edges carry semantic type (hosts, runs, exposes, leads\_to, proves, spawned). Auto-spawns exploitation follow-up tasks when a finding is confirmed. |
| **Hallucination guard** | `verify_tool_output` checks tool exit code against stdout validity before Claude acts on results. Catches silent failures, exit-0-but-empty output, and scan tools returning < 20 bytes. |
| **Asset hotlist scoring** | `aegis_hotlist` risk-scores every discovered asset by port sensitivity, path type, parameter presence, CVE count, and recency. Drives work order for vuln analysis so Claude attacks highest-value targets first. |
| **Context compression** | `aegis_compress_context` calls the NANO model (Haiku) to summarize old mission log entries, preventing context overflow on long multi-phase engagements. |
| **PoC generation** | `generate_poc` dispatches to 14 vuln-specific helpers and produces structured, runnable proof-of-concept artifacts (curl commands, Python scripts, JS payloads) tied to confirmed findings. |
| **153 MCP tools** | +11 tools: ptt\_add\_node, ptt\_update\_node, ptt\_get\_graph, ptt\_next\_targets, ptt\_spawn\_tasks, ptt\_summary, aegis\_hotlist, aegis\_compress\_context, generate\_poc, verify\_tool\_output, aegis\_engagement\_status. |

---

## Installation

### Option A — Docker (recommended, all tools pre-installed)

```bash
# Pull pre-built image (~4 GB) or build locally (~15-30 min)
aegis docker pull                          # from GitHub Container Registry
# OR
aegis docker build                         # build locally from Dockerfile

# Run an engagement
export ANTHROPIC_API_KEY=sk-ant-...
aegis docker run engagements/2026-acme

# Interactive shell with all 80+ tools on PATH
aegis docker shell

# Check image status and tool inventory
aegis docker status
```

The Docker image is based on **Kali rolling** and includes every tool in the catalog — nmap, nuclei, httpx, ffuf, gobuster, dalfox, sqlmap, masscan, hydra, wfuzz, theharvester, bbot, semgrep, trivy, and 65+ more.

### Option B — Native install (one-liner)

```bash
# Auto-detects Arch, Kali, Ubuntu/Debian, Fedora, macOS
curl -fsSL https://raw.githubusercontent.com/glorybnat/aegis-pentest/main/scripts/install.sh | bash

# Or from the repo
bash scripts/install.sh

# Build Docker image instead
bash scripts/install.sh --docker
```

### Option C — Manual

```bash
# Install AEGIS
pipx install aegis-pentest

# Install missing pentest tools
aegis init                          # detect what's installed
aegis env tools --missing           # show what's missing + install commands
aegis env install --missing         # install everything (no-sudo tools)
aegis env install --missing --dry-run   # preview first
```

**Requirements:**

- Python 3.12+
- `ANTHROPIC_API_KEY` in environment
- Docker (Option A) or pentest toolchain (Option B/C)

---

## Quickstart

```bash
# Docker quickstart (zero host setup)
export ANTHROPIC_API_KEY=sk-ant-...
aegis docker build                          # or: aegis docker pull
aegis docker run engagements/2026-acme

# Native quickstart
# First run: profiles host, lists missing tools
aegis init

# Sync the knowledge base (NVD, GHSA, nuclei-templates)
aegis kb sync

# Create a new engagement
aegis engagement new --client "Acme Corp" --domain "acme.com"

# Fill in the authorization details
vim engagements/2026-05-acme/scope.yaml

# Run
aegis run engagements/2026-05-acme

# Generate report
aegis report engagements/2026-05-acme --format html
```

---

## scope.yaml

Aegis refuses to run without this file. No exceptions.

```yaml
engagement_id: "BL-2026-007"
client: "Acme Corp"
operator: "Majd Bnat <hey@majdb.com>"
authorization:
  document_ref: "SOW-2026-007.pdf"
  signed_date: "2026-05-12"
  expiry: "2026-06-12"
in_scope:
  domains:
    - "*.acme.com"
    - "api-staging.acme.io"
  ips:
    - "203.0.113.0/24"
out_of_scope:
  - "admin.acme.com"
  - "*.internal.acme.com"
rules_of_engagement:
  rate_limit_rps: 10
  business_hours_only: false
  destructive_tests: false
  no_credential_stuffing: true
  no_dos_tests: true
```

---

## Architecture

```
                          aegis CLI
                              |
                      Engagement Manager
           (scope validation, lifecycle, audit log)
                              |
            +-----------------+-----------------+
            |                 |                 |
        Environment         Target          Methodology
         Profiler          Profiler           Engine
       (host info)      (fingerprint)      (PTES phases)
            |                 |                 |
            +--------+--------+--------+--------+
                              |
                       LLM Orchestrator
                   (Haiku / Sonnet / Opus)
                              |
         +----------+---------+----------+----------+
         |          |         |          |          |
        Tool    Knowledge  Findings    PTT       PoC
      Registry    Base       DB      Graph    Generator
     (153 tools) (NVD/GHSA) (SQLite) (aiosqlite) (14 types)
         |                              |
   Tool Executor               Hallucination Guard
  (async, sandboxed)          (output verification)
         |
      Reporter
    (md/html/json)
```

The orchestrator runs a bounded loop per phase:

```
plan(phase_context) -> execute(action) -> observe(result) -> update(state)
  ^                                                               |
  +---------------------------------------------------------------+
                   until phase complete OR budget exceeded
```

Three independent budgets bound each phase: token budget, wall-clock time, and action count. Whichever trips first ends the phase and triggers finalize mode.

---

## Token model

Aegis is designed to complete a full medium-scope engagement for under $2 in LLM tokens. This is achieved through several layered tactics:

| Tactic | Impact |
|---|---|
| Tiered model routing (Haiku handles ~70% of calls) | -60% cost |
| Prompt caching on system prompt + engagement context | -40% on input tokens |
| Parsed tool output, never raw stdout to the LLM | -80% on tool-heavy phases |
| Structured tool-use schema, no prose planning | -30% output tokens |
| Methodology-driven action space pruning | -50% wasted calls |
| SQLite-cached recon reused across phases | variable |
| Finding deduplication before LLM sees results | -10-30% |

Model tiers:

| Tier | Model | Used for |
|---|---|---|
| NANO | claude-haiku-4-5 | Parsing, classification, summarization |
| MAIN | claude-sonnet-4-6 | Planning, hypothesis generation, verification probes |
| DEEP | claude-opus-4-7 | Attack chain analysis, hard reasoning |
| LOCAL | ollama (optional) | Offline pre-classification |

The live cost meter runs in the terminal throughout each phase:

```
Phase: VULN_ANALYSIS  [>>>>>>>>--] 80%
Budget: $0.74 / $5.00   Tokens: 41.2k / 200k   Time: 12m / 60m
Tier breakdown: NANO 24% . MAIN 71% . DEEP 5%   Cache hit: 82%
```

---

## Tool catalog

Aegis wraps **153 tools** exposed as native MCP tools. Raw output is never passed to the LLM — each tool has a typed parser that produces structured `Finding` or `Observation` models. A nmap scan returning 47 open ports becomes 47 `OpenPort` observations of ~80 bytes each, not 200 KB of XML.

All tools degrade gracefully: if a binary is not installed, the tool returns a structured error with the exact install command.

| Category | Tools |
|---|---|
| **Subdomain enumeration** | subfinder, amass, assetfinder, findomain, dnsx, alterx, massdns, shuffledns, puredns, dnstwist, subdomain\_brute\_massive |
| **DNS recon** | dnsrecon, dnsx, massdns, fierce, crt.sh (cert transparency) |
| **OSINT / passive** | uncover (Shodan/Fofa/Censys/ZoomEye), shodan CLI, censys CLI, asnmap, passive\_intel, github\_discover, pwndb\_search, theharvester, spiderfoot, recon\_ng, google\_dorks |
| **Live host detection** | httpx, httpx\_batch, httprobe, naabu |
| **Port scanning** | nmap, naabu, masscan, rustscan |
| **Web crawling** | katana, gospider, hakrawler, getjs |
| **URL / archive recon** | gau, waybackurls, wayback\_recon, meg, unfurl, qsreplace |
| **Content discovery** | ffuf, feroxbuster, gobuster, dirsearch, wfuzz, content\_discovery |
| **Vhost fuzzing** | ffuf\_vhost |
| **Tech fingerprinting** | whatweb, httpx -tech-detect, cdncheck, tlsx |
| **WAF detection** | wafw00f, whatwaf |
| **TLS auditing** | sslscan, sslyze, tlsx |
| **Parameter discovery** | arjun, paramspider, gf (pattern filtering) |
| **XSS** | dalfox, kxss, crlfuzz, xsstrike, nuclei, aegis\_verify (xss probe) |
| **SQLi** | sqlmap, nosqlmap, nuclei, aegis\_verify (sqli / timing\_sqli probes) |
| **SSTI** | sstimap, aegis\_verify (ssti probe) |
| **Command injection** | commix, aegis\_verify (cmdi\_oob probe) |
| **HTTP smuggling** | smuggler (CL/TE, TE/CL), h2csmuggler (HTTP/2 cleartext) |
| **CORS** | corsy |
| **SSRF / OOB** | oob\_init + oob\_poll (interactsh), aegis\_verify (ssrf\_oob / xxe probes) |
| **Race conditions** | race\_condition\_scan, aegis\_verify (race probe) |
| **OAuth** | oauth\_audit (open redirect, state bypass, PKCE) |
| **JWT attacks** | jwt\_audit (alg:none, RS256→HS256, weak secret) |
| **GraphQL** | graphql\_audit, graphql\_cop |
| **WebSocket** | websocket\_test (injection, origin validation, auth) |
| **API discovery** | api\_discover, kiterunner, openapi\_audit |
| **IDOR** | idor\_check (cross-user object access) |
| **JS analysis** | js\_recon, linkfinder, secretfinder |
| **Header injection** | header\_injection\_scan (Host header, cache poisoning) |
| **403 bypass** | bypass\_403 (path tricks + header overrides) |
| **Prototype pollution** | aegis\_verify (prototype\_pollution probe) |
| **Secrets / SAST** | trufflehog, semgrep, bandit, safety, npm\_audit, retire\_js, secret\_scan |
| **Vulnerability scanning** | nuclei, nuclei\_fuzz, nuclei\_ai\_generate, wapiti, nikto |
| **CMS scanning** | wpscan\_deep, cmseek, joomscan, droopescan |
| **Network / SMB** | enum4linux, smbmap, snmpcheck |
| **Cloud storage** | cloud\_enum, s3scanner, bucket\_finder, gitdumper, cloudfox |
| **Container / SBOM** | trivy, grype, syft |
| **Auth / brute force** | hydra |
| **Subdomain takeover** | subjack, subzy, subdomain\_takeover\_scan |
| **Recon orchestration** | bbot (multi-module OSINT), interlace (parallel execution) |
| **Reporting** | aegis\_report (md / html / json) |
| **Methodology** | wstg\_check (OWASP WSTG v4.2, 80+ checks) |
| **Attack graph (PTT)** | ptt\_add\_node, ptt\_update\_node, ptt\_get\_graph, ptt\_next\_targets, ptt\_spawn\_tasks, ptt\_summary |
| **Orchestration** | aegis\_hotlist (asset risk scoring), aegis\_compress\_context (context compression), aegis\_engagement\_status |
| **PoC generation** | generate\_poc (14 vuln classes: XSS, SQLi, SSRF, LFI, RCE, IDOR, open redirect, JWT, CORS, CSRF, HTTP smuggling, SSTI, XXE, prototype pollution) |
| **Output verification** | verify\_tool\_output (hallucination guard — validates tool output before Claude acts on it) |

---

## Environment profiling

On first run, `aegis init` profiles the host and derives auto-tuned concurrency settings:

```
Host: arch-workstation
  OS        Arch Linux (rolling, kernel 6.9.3-arch1-1)
  CPU       AMD Ryzen 7 5800X . 8 cores / 16 threads . 4.7 GHz
  Memory    32 GB total . 24 GB available
  Repos     core, extra, multilib, blackarch

Pentest toolchain: 28/35 detected
  nmap 7.95       nuclei 3.2.9      httpx 1.6.6
  ffuf 2.1.0      subfinder 2.6.6   katana 1.1.0
  sqlmap 1.8.5    wpscan 3.8.27     nikto 2.5.0
  gobuster 3.6.0  amass 4.2.0       gowitness 3.0.3

Missing: testssl.sh  feroxbuster  dnsrecon  arjun  paramspider  trufflehog
  -> Run: aegis env install --missing

Auto-tuned concurrency:
  nmap_parallelism=16   nuclei_concurrency=32   ffuf_threads=64
  httpx_concurrency=80  max_parallel_tools=4
```

---

## CLI reference

```
aegis init                                    First-run setup and env profile
aegis env show                                Display host profile
aegis env tools                               Tool inventory
aegis env install --missing                   Generate install commands for missing tools
aegis env refresh                             Re-detect host profile

aegis kb sync [--source nvd|ghsa|nuclei]      Sync knowledge base
aegis kb stats                                Knowledge base summary
aegis kb query --product nginx --min-cvss 7   Query CVEs

aegis engagement new --client X --domain Y   Scaffold engagement dir and scope.yaml
aegis engagement list                         List engagements

aegis run <dir> [--phase PHASE]              Run engagement
aegis run <dir> --dry-run                    Preview planned actions
aegis run <dir> --budget-usd 2.00           Cap spend

aegis report <dir> [--format md|html|json]   Generate report
aegis findings list <dir> [--severity high]  List findings
aegis findings verify <finding-id>           Re-run verification probe
aegis findings suppress <finding-id> --reason "..."

aegis cost <dir>                             Detailed cost breakdown
aegis audit <dir>                            Full audit log
```

All commands support `--json` for scripting, `-v/-vv/-vvv` for verbosity, `--quiet` for CI.

---

## Configuration

Global config lives at `~/.config/aegis/config.toml`. Any key can be overridden per engagement in `engagement_dir/config.toml`.

```toml
[api]
anthropic_api_key_env = "ANTHROPIC_API_KEY"

[models]
nano  = "claude-haiku-4-5-20251001"
main  = "claude-sonnet-4-6"
deep  = "claude-opus-4-7"

[models.local]
enabled  = false
endpoint = "http://localhost:11434"
model    = "qwen2.5:7b"

[budgets]
tokens_per_phase       = 30000
tokens_per_engagement  = 200000
usd_per_engagement     = 5.00
wall_time_per_phase_sec = 1800

[caching]
prompt_cache          = true
kb_cache_dir          = "~/.cache/aegis/kb"
fingerprint_cache_ttl_hours = 168

[tooling]
docker_isolate         = false
default_rate_limit_rps = 10
respect_robots_txt     = false

[reporting]
default_format   = "html"
include_audit_log = true
```

---

## Tech stack

| Layer | Choice |
|---|---|
| Language | Python 3.12+ |
| CLI | Typer + Rich |
| Async | asyncio + anyio |
| HTTP | httpx (async, HTTP/2) |
| Models | Pydantic v2 |
| Storage | SQLite + SQLModel |
| LLM | Anthropic SDK (Claude) |
| Templating | Jinja2 |
| Logging | structlog + rich |
| Packaging | uv (dev), hatch (build) |
| Testing | pytest + pytest-asyncio + respx |

---

## License

MIT. Use responsibly and only against systems you are authorized to test.

---

Built by [Majd Bnat](https://majdb.com)
