Metadata-Version: 2.4
Name: websential
Version: 0.1.0
Summary: Advanced Python library for website URL discovery, security protection detection, database identification, and URL classification.
Author: 0xSPARK
Author-email: HabibSarkisofficial@gmail.com
Keywords: web security,web scraping,URL detection,database detection,WAF,Cloudflare,reCAPTCHA
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Security
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENCE
Requires-Dist: requests
Requires-Dist: beautifulsoup4
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: keywords
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Websential

**Websential** is a Python library designed for advanced website reconnaissance and security analysis. It allows you to:

- Collect all visible and hidden URLs from a website.  
- Detect website protections such as Cloudflare, reCAPTCHA, WAFs, and more.  
- Identify the underlying database type based on response content.  
- Classify URLs into functional groups like login, token, payment, OAuth, and more.

---

### Installation

pip install websential

---

### Usage

from Websential import urldetect, urlfilter

url = "https://www.example.com"

# Collect URLs from the site
urls = urldetect.urls(url)
print("Collected URLs:", urls)

# Filter URLs by type (e.g., token)
token_urls = urlfilter.type(urls, 'token')
print("Token-related URLs:", token_urls)

# Detect website protections
protections = urldetect.protection(url)
print("Detected protections:", protections)

# Detect database type
databases = urldetect.database(url)
print("Detected databases:", databases)

---

### Features

1. **URL Detection**  
   - Fetches all HTTP/HTTPS links on a website.  
   - Parses `robots.txt` to detect allowed/disallowed paths and sitemaps.  

2. **Protection Detection**  
   - Detects multiple protection mechanisms including: Cloudflare, reCAPTCHA v2/v3, hCaptcha, Datadome, Akamai, Imperva, F5 BIG-IP, Fortinet, Wordfence, and more.  

3. **Database Detection**  
   - Detects common databases like MySQL, PostgreSQL, Oracle, SQL Server, MongoDB, SQLite by analyzing HTTP responses and headers.  

4. **URL Classification**  
   - Groups URLs based on functional categories: login/auth, token, payment, OAuth, verification, accounts, registration, admin panels, etc.

---

### License

MIT License
