Metadata-Version: 2.4
Name: humanbot
Version: 0.1.0
Summary: A human-like HTTP client: randomized browser headers, TLS impersonation, delays, and built-in HTML parsing.
Author-email: Arad Ahmadi <arad.xml@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/aradxml/humanbot
Project-URL: Repository, https://github.com/aradxml/humanbot
Project-URL: Issues, https://github.com/aradxml/humanbot/issues
Keywords: http,scraping,requests,web-scraping,user-agent,curl_cffi,beautifulsoup
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: curl_cffi>=0.6.0
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: fake-useragent>=1.4.0
Requires-Dist: lxml>=4.9.0
Requires-Dist: requests>=2.31.0
Dynamic: license-file

<div align="center">

# 🤖 humanbot

**A human-like HTTP client for Python.**

Randomized browser headers · TLS/JA3 impersonation · Smart delays · Built-in HTML parsing

[![PyPI version](https://img.shields.io/pypi/v/humanbot.svg)](https://pypi.org/project/humanbot/)
[![Python versions](https://img.shields.io/pypi/pyversions/humanbot.svg)](https://pypi.org/project/humanbot/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

</div>

---

## ✨ Why humanbot?

Plain `requests` looks nothing like a real browser — no realistic headers, no TLS fingerprint,
no natural pacing between requests. **humanbot** fixes that. It wraps
[`curl_cffi`](https://github.com/yifeikong/curl_cffi) and
[`BeautifulSoup`](https://www.crummy.com/software/BeautifulSoup/) into one clean, `requests`-style
client that just *feels* human.

- 🧑‍💻 **Realistic headers** — randomized per request, tuned for desktop, mobile, and tablet
- 🔒 **TLS/JA3 impersonation** — looks like real Chrome/Firefox/Safari at the network level
- ⏱️ **Natural pacing** — optional randomized delays between requests
- 🥣 **Parsing built-in** — `client.soup`, CSS selectors, tables, lists, links, images — no extra imports
- 💾 **One-line saving** — dump HTML, JSON, text, or binary responses straight to disk
- 🧩 **Drop-in feel** — if you know `requests`, you already know `humanbot`

## 📦 Installation

```bash
pip install humanbot
```

**Requirements:** Python 3.8+

## 🚀 Quickstart

```python
from humanbot import Human

with Human(device="desktop") as client:
    client.get("https://example.com")
    title = client.get_text("h1")
    links = client.get_links()
    print(title, links)
```

That's it — realistic headers, TLS impersonation, and HTML parsing, all in three lines. 🎉

## 🛠️ Usage

### Creating a client

```python
from humanbot import Human

client = Human(
    device="desktop",         # 📱 "mobile" · 💻 "desktop" · 📱 "tablet"
    delay_min_s=1,             # ⏱️ minimum delay before each request (seconds)
    delay_max_s=3,             # ⏱️ maximum delay before each request (seconds)
    lang_code="en-US",         # 🌍 override Accept-Language
    impersonate="chrome120",   # 🎭 curl_cffi TLS impersonation target
    parser="lxml",             # 🥣 BeautifulSoup parser
)
```

### Making requests

```python
client.get("https://example.com")
client.post("https://example.com/api", json={"key": "value"})
client.put(...)
client.patch(...)
client.delete(...)
client.head(...)
client.options(...)
```

Every method returns a standard `Response` object, and HTML responses are automatically parsed
into `client.soup` behind the scenes. 🪄

### 🔍 Parsing helpers

```python
client.get("https://example.com")

client.find("div.article")               # all matching elements
client.find_one("h1")                    # first matching element
client.get_text("h1")                    # text of first match, whitespace-normalized
client.get_attribute("img", "src")       # attribute of first match
client.get_all_attributes("img", "src")  # attribute of all matches
client.get_links()                       # all href values
client.get_images()                      # all img src values
client.extract_table("table")            # 2D list of cell text
client.extract_list("ul.items")          # list of item text
```

### 💾 Saving responses

```python
client.save_html("page.html")
client.save_json("data.json")
client.save_text("page.txt")
client.save_binary("file.bin")
client.save_soup("pretty.html")     # prettified parsed HTML
client.download_file(url, "output.zip")
```

### 🧢 Managing headers

```python
client.get_headers()
client.set_header("X-Custom", "value")
client.update_headers({"X-A": "1", "X-B": "2"})
```

Or generate a header set on its own, without spinning up a client:

```python
from humanbot import HeadersGenerator

gen = HeadersGenerator()
gen.desktop_headers()
gen.mobile_headers()
gen.tablet_headers()
gen.random_headers()
```

## 📋 Full example

```python
from humanbot import Human

with Human(device="mobile", delay_min_s=1, delay_max_s=2) as client:
    client.get("https://news.ycombinator.com")

    for title, link in zip(
        client.extract_list("table.itemlist", "span.titleline"),
        client.get_links("span.titleline a"),
    ):
        print(f"📰 {title} → {link}")

    client.save_html("hn.html")
```

## ⚠️ A responsible-use note

`humanbot` is built for legitimate scraping, testing, and automation. Always respect a site's
`robots.txt`, terms of service, and rate limits — please don't use this to abuse or overload
services you don't own or have permission to test.

## 🤝 Contributing

Issues and pull requests are welcome! Open one on
[GitHub](https://github.com/aradxml/humanbot).

## 📄 License

MIT © [Arad Ahmadi](https://github.com/aradxml)

---

<div align="center">
Made with 🖤 by <a href="https://github.com/aradxml">Arad Ahmadi</a>
</div>
