Metadata-Version: 2.4
Name: traceforge-osint
Version: 1.0.0
Summary: TraceForge — Open-Source Intelligence & Digital Forensics Platform
Author-email: Aman Kumar Pandey <paman7647@proton.me>
License: MIT
Project-URL: Homepage, https://github.com/paman7647/TraceForge
Project-URL: Documentation, https://traceforge.readthedocs.io
Project-URL: Repository, https://github.com/paman7647/TraceForge.git
Project-URL: Issues, https://github.com/paman7647/TraceForge/issues
Project-URL: Changelog, https://github.com/paman7647/TraceForge/blob/master/CHANGELOG.md
Keywords: osint,dfir,digital-forensics,incident-response,security,threat-intelligence,pcap,stix,misp,ioc-extractor
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: System :: Networking
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Provides-Extra: reporting
Requires-Dist: openpyxl>=3.1.0; extra == "reporting"
Requires-Dist: python-docx>=1.0.0; extra == "reporting"
Provides-Extra: docs
Requires-Dist: sphinx>=7.0.0; extra == "docs"
Requires-Dist: myst-parser>=3.0.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "docs"
Provides-Extra: dev
Requires-Dist: build>=1.0.0; extra == "dev"
Requires-Dist: twine>=5.0.0; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Provides-Extra: all
Requires-Dist: openpyxl>=3.1.0; extra == "all"
Requires-Dist: python-docx>=1.0.0; extra == "all"
Requires-Dist: sphinx>=7.0.0; extra == "all"
Requires-Dist: myst-parser>=3.0.0; extra == "all"
Requires-Dist: sphinx-rtd-theme>=2.0.0; extra == "all"
Requires-Dist: build>=1.0.0; extra == "all"
Requires-Dist: twine>=5.0.0; extra == "all"
Dynamic: license-file

# TraceForge

**Open-source OSINT, DFIR and security investigation toolkit.**

[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![Platform](https://img.shields.io/badge/Platform-macOS%20%7C%20Linux%20(Debian%2C%20Ubuntu%2C%20Kali)-informational.svg)](docs/INSTALLATION.md)
[![Shell](https://img.shields.io/badge/Shell-Bash%203.2%2B-4EAA25.svg)](https://www.gnu.org/software/bash/)
[![Catalog](https://img.shields.io/badge/Catalog-152%20Tools-blueviolet.svg)](catalog/TOOLS.md)
[![Python](https://img.shields.io/badge/Python-%3E%3D3.9-3776AB.svg)](pyproject.toml)
[![CI](https://github.com/paman7647/TraceForge/actions/workflows/ci.yml/badge.svg)](.github/workflows/ci.yml)
[![Version](https://img.shields.io/badge/Version-1.0.0-green.svg)](VERSION)

---

## What It Does

TraceForge is an open-source command-line toolkit for OSINT, digital forensics, security investigation, evidence handling, and first-party investigation tools:

* **First-Party Native Tools (`traceforge` / `go/`)**: High-performance Go-native and Python utilities providing asset relationship graphing, snapshot diffing, streaming IOC extraction/defanging, forensic evidence indexing, UTC timeline normalization, PCAP network flow analysis, automated log triage, and filesystem baselining (see [docs/FIRST_PARTY_TOOLS.md](docs/FIRST_PARTY_TOOLS.md)).
* **Tool Catalog & Installer**: A searchable catalog of 152 tools and an automated installer that handles dependencies across Homebrew (macOS), APT (Linux), `pipx` (Python virtual environments), Go, Cargo, and RubyGems.
* **Investigation Modules**: 7 built-in analysis scripts covering image metadata, network PCAPs, usernames, email breach records, domain DNS records, document metadata, and local OPSEC hygiene.
* **Case Management**: Workspaces stored under `workspace/` with unique case IDs, SHA-256 evidence hashing, and append-only chain-of-custody logging.
* **Export Pipeline**: Generates reports in Markdown, standalone dark-mode HTML, CSV (with formula injection protection), TSV, JSON, JSONL streams, STIX 2.1, MISP, GeoJSON, KML, Excel XLSX, Word DOCX, and signed ZIP packages.
* **Redaction**: Built-in `--redact` flag masks sensitive IP addresses and email addresses in generated reports.

---

## Architecture

TraceForge does not ship binary copies of third-party tools inside the repository. It installs and launches external tools via your system's package managers:

```text
+-------------------------------------------------------------------+
|                            TraceForge                             |
|      (CLI Menu, Tool Catalog, Case Workspaces, Report Exporter)   |
|                         [ MIT License ]                           |
+---------------------------------+---------------------------------+
                                  |
                                  | Installs / Launches / Collects Output
                                  v
+-------------------------------------------------------------------+
|                 Third-Party Command-Line Tools                    |
|   (ExifTool, TShark, Binwalk, Nmap, Sherlock, Maigret, etc.)     |
|         [ Retain Respective Upstream Copyright & Licenses ]        |
+-------------------------------------------------------------------+
```

All third-party tools remain the property of their respective creators and are licensed under their own open-source terms. See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for details.

---

## Supported Systems

| Platform | Tier | Architecture | Package Manager | Guide & Notes |
|---|---|---|---|---|
| **macOS Apple Silicon** | Supported | `arm64` (M1/M2/M3/M4) | Homebrew | Full native support |
| **macOS Intel** | Supported | `x86_64` | Homebrew | Full native support |
| **Debian / Ubuntu** | Supported | `x86_64`, `arm64` | APT | Full native support |
| **Kali Linux** | Supported | `x86_64`, `arm64` | APT | Full native support |
| **Termux / Android** | Supported with limitations | `arm64`, `armv7`, `x86_64` | `pkg` | Non-root userland. Offline forensics & PCAP analysis. [Termux Guide](docs/platforms/termux.md) |

---

## Installation & Quick Start

### 1. Clone the Repository
```bash
git clone https://github.com/paman7647/TraceForge.git
cd TraceForge
chmod +x install_all.sh main.sh modules/*.sh scripts/*.sh tests/*.sh
```

### 2. Run the Installer
```bash
# Preview packages without installing (dry-run)
./install_all.sh --profile recommended --dry-run

# Core Profile: Minimal tools for the 7 modules (~15 packages)
./install_all.sh --profile core

# Recommended Profile (Default): Standard investigative toolset (~50 packages)
./install_all.sh --profile recommended

# Full Profile: All 152 installable catalog tools
./install_all.sh --profile full
```

### 3. Run TraceForge
```bash
# Interactive TTY Menu
./main.sh
# or via Python CLI
python3 -m traceforge

# Run system diagnostics
./main.sh doctor

# List existing cases
./main.sh list-cases

# Search the catalog
./main.sh search "steganography"

# Run first-party tools
traceforge ioc extract /path/to/intel.txt
traceforge asset graph /path/to/entities.jsonl --html graph.html

# Run a module directly
./main.sh module 1 /path/to/evidence.png CASE-20260825-ABC123

# Export a case
./main.sh export CASE-20260825-ABC123 --all --package zip
```

---

## Investigation Modules

| Module | Script | Target & Tools Used |
|---|---|---|
| **01. Media & Image Forensics** | `modules/01_image_forensics.sh` | Images and videos: EXIF/IPTC metadata, GPS coordinates, strings, steganography (`exiftool`, `binwalk`, `zsteg`, `xxd`). |
| **02. Network & PCAP Forensics** | `modules/02_network_recon.sh` | Packet captures: DNS queries, HTTP URIs, TLS SNI headers, conversations (`tshark`, `capinfos`, `aircrack-ng`). |
| **03. Identity & Social Research** | `modules/03_identity_social.sh` | Usernames and aliases: account discovery across web platforms (`sherlock`, `maigret`, `blackbird`, `socialscan`). |
| **04. Email & Breach Intelligence** | `modules/04_email_breach.sh` | Email addresses: account registrations, breach dumps, SPF/DMARC posture (`holehe`, `h8mail`, `emailrep`, `theHarvester`, `checkdmarc`). |
| **05. Domain & DNS Intelligence** | `modules/05_domain_dns.sh` | Domains: DNS records, passive subdomains, HTTP probing, typosquats (`dig`, `whois`, `subfinder`, `amass`, `dnsx`, `httpx`, `dnstwist`). |
| **06. Document & Metadata Harvesting** | `modules/06_document_harvesting.sh` | Documents (PDF, DOCX, XLSX): properties, text, embedded objects, macros, secret keys (`poppler`, `oletools`, `exiftool`, `qpdf`, `ripgrep`). |
| **07. OPSEC & Environment Audit** | `modules/07_opsec_anonymization.sh` | Local audit: public IP detection, DNS leak test, Tor/proxy status, encryption tools (`mat2`, `tor`, `proxychains`, `gnupg`, `age`). |

---

## Directory Structure

```text
TraceForge/
├── main.sh                        # Interactive console and CLI dispatcher
├── install_all.sh                 # Multi-ecosystem package installer
├── pyproject.toml                 # Modern Python package configuration
├── go.mod                         # Go module declaration (zero external dependencies)
├── VERSION                        # Project version (1.0.0)
├── LICENSE                        # MIT License
├── NOTICE                         # Copyright and attribution
├── THIRD_PARTY_NOTICES.md         # Upstream tool licenses and repository links
├── README.md                      # Main project documentation
├── SECURITY.md                    # Vulnerability reporting guidelines
├── CONTRIBUTING.md                # Contribution guidelines
├── CODE_OF_CONDUCT.md             # Contributor Covenant 2.1
├── CHANGELOG.md                   # Version release history
├── RELEASE_NOTES.md               # 1.0.0 release notes
│
├── go/                            # Go-native high-performance engine
├── traceforge/                    # Pure Python implementation and package
│
├── catalog/
│   ├── README.md                  # Catalog schema specification
│   ├── tools.tsv                  # 15-column registry of 152 tools
│   └── TOOLS.md                   # Formatted markdown tool index
│
├── lib/
│   ├── common.sh                  # Terminal UI, colors, logging, string helpers
│   ├── platform.sh                # OS and architecture detection
│   ├── packages.sh                # Package managers (Homebrew, APT, pipx, Go, Gem, Cargo)
│   ├── catalog.sh                 # Catalog search and filter functions
│   ├── case.sh                    # Case lifecycle, evidence, findings, IOCs, timeline
│   ├── export.sh                  # Export coordinator, packaging, and hashing
│   └── report.sh                  # Markdown, HTML, and PDF report generators
│
├── modules/                       # 7 investigation modules (01 to 07)
├── scripts/
│   ├── doctor.sh                  # System and dependency diagnostic checker
│   ├── export_case.sh             # CLI export wrapper
│   ├── export_engine.py           # Multi-format export engine
│   ├── install_tool.sh            # Single tool installer
│   ├── generate_catalog_docs.sh   # Markdown documentation generator
│   └── release_check.sh           # Pre-flight release verification test
│
├── docs/                          # In-depth operator, architecture, and OPSEC guides
├── tests/                         # Automated test suite and regression tests
└── workspace/                     # Local investigation cases
    └── .gitkeep
```

---

## Development & Branching Workflow

TraceForge maintains two long-lived branches:
* **`master`**: Stable, production-ready public release branch.
* **`beta`**: Active integration branch for new features, bug fixes, platform improvements, and release candidates.

```text
                  master  (Stable Releases)
                    ▲
                    │  Release PR (Validated & Tagged)
                    │
                   beta   (Active Development & RC)
                ▲   ▲   ▲
                │   │   │  Pull Requests
           feature fix docs  (Short-Lived Working Branches)
```

Pull requests and contributions must target the **`beta`** branch. See [docs/BRANCHING.md](docs/BRANCHING.md) for the branching model, [docs/BUG_REPORTING.md](docs/BUG_REPORTING.md) for issue guidelines, and [CONTRIBUTING.md](CONTRIBUTING.md) for pull request workflows.

---

## Automated Testing

Run the test suite before submitting pull requests or publishing releases:

```bash
# Run Python & Platform test suites
python3 -m unittest discover -s tests

# Run master shell & integration test harness
./tests/test.sh

# Run pre-flight release check
./scripts/release_check.sh beta
```

All shell scripts are validated with `bash -n` and `shellcheck -x`.

---

## Responsible Use

TraceForge is intended for lawful OSINT, digital forensics, incident response, security research, authorized testing, education, and lab work.

You are responsible for ensuring that your use of the software is authorized and complies with applicable law, regulation, contracts, platform terms, privacy requirements, and organizational policy.

TraceForge does not grant permission to access any system, account, network, service, or data.

See [DISCLAIMER.md](DISCLAIMER.md) and [RESPONSIBLE_USE.md](RESPONSIBLE_USE.md) for details.

---

## Documentation & Policies

* [docs/BRANCHING.md](docs/BRANCHING.md) — Two-branch Git model, release lifecycle, and PR rules.
* [docs/BUG_REPORTING.md](docs/BUG_REPORTING.md) — Bug reporting protocol, reproduction requirements, and fix lifecycle.
* [docs/RUNTIME_PROFILES.md](docs/RUNTIME_PROFILES.md) — Architecture and configuration of the 6 runtime profiles.
* [docs/FIRST_PARTY_TOOLS.md](docs/FIRST_PARTY_TOOLS.md) — Documentation for first-party analytical tools.
* [docs/platforms/termux.md](docs/platforms/termux.md) — Termux and Android platform guide and storage configuration.
* [DISCLAIMER.md](DISCLAIMER.md) — Legal disclaimer, liability boundaries, accuracy notices, and jurisdictional advice.
* [RESPONSIBLE_USE.md](RESPONSIBLE_USE.md) — Responsible use standards, authorized scope, and prohibited activities.
* [PRIVACY.md](PRIVACY.md) — Privacy principles, local storage layout, data minimization, and redaction.
* [SECURITY.md](SECURITY.md) — Vulnerability reporting policy and defensive engineering standards.
* [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) — Upstream tool licenses, attribution, and license boundaries.
* [CONTRIBUTING.md](CONTRIBUTING.md) — Contribution workflow, code formatting, and review guidelines.
* [LICENSE](LICENSE) — Official MIT License terms and conditions.

---

## Authors & Credits

* **Lead Architect & Core Developer**: Aman Kumar Pandey
* **Credits**: Thanks to the Global Open-Source Security Community
* **License**: [MIT License](LICENSE)
* **Copyright**: Copyright (c) 2026 Aman Kumar Pandey
