Metadata-Version: 2.4
Name: osint-darkweb-pkg
Version: 0.1.0
Summary: A Darkweb OSINT tool for scraping and searching onion sites.
Author-email: Your Name <your.email@example.com>
License: MIT
Project-URL: Homepage, https://bitbucket.org/yourusername/osint-darkweb-pkg
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.28.0
Requires-Dist: beautifulsoup4>=4.11.0
Requires-Dist: pysocks>=1.7.1

# Darkweb OSINT Package

A modular Open Source Intelligence (OSINT) tool designed to search and scrape data from the Darkweb (Tor network). This package aggregates results from multiple darkweb search engines and provides a concurrent scraper to extract content from `.onion` sites.

## Features

* **Multi-Engine Search:** Queries 15+ darkweb search engines (Ahmia, Torch, OnionLand, etc.) concurrently.
* **Concurrent Scraping:** Rapidly extracts text content from multiple `.onion` URLs simultaneously.
* **Tor Integration:** Built-in SOCKS5 proxy configuration for seamless routing through the Tor network.
* **User-Agent Rotation:** Randomizes user agents to minimize blocking.

## Prerequisites

**Crucial:** You must have the Tor service running on your machine for this package to function. This package routes traffic through the local Tor SOCKS5 proxy (default port `9050`).

* **Linux (Debian/Ubuntu):**
    ```bash
    sudo apt install tor
    sudo service tor start
    ```
* **Windows:**
    Download the [Tor Browser](https://www.torproject.org/download/) or the Tor Expert Bundle and ensure it is running.

## Installation

This package is managed via `pyproject.toml`. You can install it directly:

```bash
pip install osint-darkweb.pkg
```


### Usage
** 1. Searching**
Search for a keyword across multiple hidden service search engines.

from osint_darkweb_pkg import get_search_results

query = "example query"
# Returns a list of dictionaries: [{'title': '...', 'link': '...'}]
results = get_search_results(query)

print(f"Found {len(results)} links.")

** 2. Scraping**
Extract text content from a list of .onion URLs.
from osint_darkweb_pkg import scrape_multiple

# specific .onion links or results from the search step
urls = [
    {'link': '[http://example1.onion](http://example1.onion)', 'title': 'Example 1'},
    {'link': '[http://example2.onion](http://example2.onion)', 'title': 'Example 2'}
]

# Returns a dictionary: {'[http://example1.onion](http://example1.onion)': 'Page content...'}
data = scrape_multiple(urls)

for url, content in data.items():
    print(f"URL: {url}\nContent: {content[:100]}...\n")
    

## Structure 
    search.py: Handles sending queries to search engine endpoints and parsing results.

    scrape.py: Handles visiting specific .onion URLs and extracting text.

    utils.py: Manages shared resources like Tor proxy configuration and User-Agent rotation.

## Disclaimer
    This tool is intended for educational and research purposes only (e.g., security research, OSINT investigations). The authors are not responsible for any misuse of this tool. Accessing the dark web may be illegal or monitored in certain jurisdictions. Use responsibly.
