Metadata-Version: 2.4
Name: assetforge-recon
Version: 0.3.0
Summary: Forge raw asset lists into actionable DNS, CDN, and HTTP intelligence
Author: Wation
License-Expression: MIT
Project-URL: Homepage, https://github.com/TheWation/AssetForge
Project-URL: Repository, https://github.com/TheWation/AssetForge
Project-URL: Issues, https://github.com/TheWation/AssetForge/issues
Keywords: asset-discovery,cdn,dns,http,reconnaissance,security,waf
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# AssetForge

```text
    ___                    __  ______
   /   |  __________  ___  / /_/ ____/___  _________ ____
  / /| | / ___/ ___/ / _ \/ __/ /_  / __ \/ ___/ __ `/ _ \
 / ___ |(__  |__  ) /  __/ /_/ __/ / /_/ / /  / /_/ /  __/
/_/  |_/____/____/  \___/\__/_/    \____/_/   \__, /\___/
                                               /____/
                   Asset Intelligence Pipeline
```

AssetForge turns IP, domain, and Subfinder inputs into actionable DNS, CDN,
WAF, and HTTP intelligence. It can filter CDN addresses, attribute providers,
probe web services, calculate Shodan-compatible favicon hashes, and export
self-contained HTML dashboards, JSONL, or CSV.

AssetForge is dependency-free at runtime and supports IPv4 and IPv6.

## Highlights

- Resolve, normalize, and deduplicate domains, URLs, and IP addresses
- Identify CDN and cloud ranges with provider attribution
- Filter CDN addresses from mixed IP lists
- Import plain text or Subfinder JSONL, including discovery sources
- Optionally collect redirects, status codes, titles, headers, timing, and WAF signals
- Optionally calculate favicon MurmurHash3 values with direct Shodan links
- Export standalone interactive HTML, JSONL, CSV, or plain text
- Refresh the range database automatically with retry and partial-failure handling

## Installation

Python 3.10 or newer is required.

Install from PyPI:

```powershell
python -m pip install assetforge-recon
assetforge --version
```

Run directly from a cloned or extracted source tree without installing:

```powershell
python assetforge.py help
python assetforge.py check 104.16.1.1
python assetforge.py scan -f subdomains.txt -o report.html
```

The root launcher handles `src/assetforge` automatically; there is no need to
change directories or set `PYTHONPATH`.

For an editable development installation:

```powershell
python -m pip install -e .
```

## Quick start

Check an IP or hostname and show its provider attribution:

```powershell
assetforge check 104.16.1.1
assetforge check example.com --format json
```

Filter CDN addresses from an input list. The default output contains only
addresses that are not in a known CDN range:

```powershell
assetforge filter ips.txt
assetforge filter ips.txt -o clean-ips.txt --stats
Get-Content ips.txt | assetforge filter -
```

Inspect providers or their ranges:

```powershell
assetforge providers
assetforge providers cloudflare
assetforge providers cloudflare --family 6
```

Use `assetforge help` or `assetforge help <command>` for the complete command
reference.

## Scan assets

By default, `scan` normalizes inputs, resolves current DNS records, identifies
private addresses, and classifies CDN ranges. It does not make HTTP requests
unless `--http` or `--favhash` is enabled.

```powershell
assetforge scan -f subdomains.txt -o report.html
```

Plain-text input accepts domains, subdomains, IPs, and URLs. `.jsonl` and
`.ndjson` files are detected automatically as Subfinder JSONL. AssetForge reads
both `source` and `sources`, merges discovery sources for duplicate hosts, and
performs a fresh DNS lookup.

```powershell
subfinder -d example.com -oJ -cs |
  assetforge scan - --input-format subfinder-jsonl --html report.html
```

Write multiple report formats from one scan:

```powershell
assetforge scan -f subdomains.txt --http `
  --jsonl results.jsonl `
  --csv results.csv `
  --html report.html
```

The HTML report is fully self-contained. It includes summary cards, global and
per-column filters, multi-select quick filters, sorting, pagination, dark and
light themes, and export of the currently filtered rows. Selecting one column
exports TXT; selecting multiple columns exports CSV. Columns for stages or data
that are not present are omitted automatically.

### HTTP probing

Enable HTTP metadata collection with `--http`:

```powershell
assetforge scan -f subdomains.txt --http -o http-report.html
```

HTTP results can include status code, title, final URL, redirect chain, Server,
X-Powered-By, content type, elapsed time, and passive WAF signals. WAF detection
is heuristic and can produce false positives.

Redirects are not followed by default. Use `--follow-redirect` to follow up to
10 redirects while preserving the complete status chain, such as `307 -> 200`:

```powershell
assetforge scan -f subdomains.txt --http --follow-redirect -o redirects.html
```

Each asset uses an isolated cookie session. Cookies received during redirects
are reused for subsequent eligible requests and favicon retrieval. HTTP probes
use browser-style headers and a Chrome-compatible User-Agent.

HTTPS is attempted before HTTP by default. Override this with `--scheme https`,
`--scheme http`, or `--scheme both`.

### Favicon hashes

`--favhash` enables HTTP probing, discovers a favicon from the page or
`/favicon.ico`, and calculates the signed MurmurHash3 value used by Shodan:

```powershell
assetforge scan -f subdomains.txt --favhash --follow-redirect -o favicons.html
```

Favicon hashes in the HTML report link directly to the corresponding
`http.favicon.hash` search on Shodan.

### Scan safety and output

Requests to private, loopback, link-local, reserved, multicast, and other
non-public targets are skipped by default. Use `--probe-private` only on
networks you are authorized to assess.

Without an output option, scan results are written as JSONL to stdout. Progress
and summaries use stderr, keeping structured output safe for pipelines. Use
`--quiet` to suppress progress.

## CDN range database

AssetForge includes an offline seed database. Before `check`, `filter`, `scan`,
or `providers`, it refreshes the local database when the file is missing or
more than 15 days old.

Refresh it manually at any time:

```powershell
assetforge update
```

Updates run concurrently, retry transient failures with exponential backoff,
and retain last-known ranges for providers whose feeds fail. Files are written
atomically so a failed refresh does not destroy the existing database.

The default database path is `%LOCALAPPDATA%\AssetForge\ranges.json` on Windows
and `~/.cache/AssetForge/ranges.json` elsewhere. Override it with `--database`
or `ASSETFORGE_DATABASE`.

Disable automatic updates for offline or controlled CI runs:

```powershell
assetforge --no-auto-update scan -f subdomains.txt -o report.html
$env:ASSETFORGE_NO_AUTO_UPDATE = "1"
```

Add a feed for a single update:

```powershell
assetforge update --source "My CDN=https://example.test/ranges.json"
```

The built-in catalog combines dedicated CDN feeds with broader public cloud
ranges. Provider attribution indicates an IP-range match; it does not prove
that every matched address currently serves CDN traffic.

## Custom ranges

Custom files accept either `CIDR` or `Provider,CIDR` per line. Blank lines and
comments beginning with `#` are ignored.

```text
Internal edge,192.0.2.0/24
2001:db8:1234::/48
```

```powershell
assetforge filter ips.txt --custom-ranges custom-ranges.txt
```

## Development

Run the test suite from the repository root:

```powershell
$env:PYTHONPATH = "src"
python -m unittest discover -s tests -v
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for the development workflow and provider
feed checklist.

## Responsible use

Only enable active HTTP or private-network probing for assets you are authorized
to assess. CDN ownership, HTTP metadata, WAF patterns, and favicon hashes are
signals, not proof that a host is safe, vulnerable, or an origin server.

## License

AssetForge is available under the [MIT License](LICENSE).
