Metadata-Version: 2.2
Name: osn-requests
Version: 1.0.0
Summary: osn-requests simplifies web scraping and requests in Python. It provides easy-to-use functions for fetching HTML, finding web elements using XPath, managing proxies, and generating random user agents.
Author: oddshellnick
Author-email: oddshellnick.programming@gmail.com
Description-Content-Type: text/markdown
Requires-Dist: setuptools>=75.1.0
Requires-Dist: requests>=2.32.3
Requires-Dist: lxml>=5.3.0
Requires-Dist: bs4>=0.0.2
Requires-Dist: beautifulsoup4>=4.12.3
Dynamic: author
Dynamic: author-email
Dynamic: description
Dynamic: description-content-type
Dynamic: requires-dist
Dynamic: summary

# osn-requests: Simplified Web Scraping and Requests

osn-requests is a lightweight Python library designed to simplify common web scraping and request tasks. It builds upon popular libraries like `requests`, `lxml`, and `BeautifulSoup`, providing a cleaner and more convenient interface for fetching and extracting data from websites.

## Key Features:

* **Easy HTML Parsing:** Quickly parse HTML content using `get_html`, which returns an `lxml` etree object ready for XPath queries.
* **Simplified Element Finding:** Locate specific web elements using `find_web_element` and `find_web_elements`, abstracting away the complexities of XPath handling.
* **Integrated Proxy Support:** Seamlessly integrate proxies into your requests using the `proxies` parameter in `get_html` and `get_json`.
* **Dynamic User-Agent Generation:** Easily obtain random user agents using `get_random_user_agent` to avoid being blocked by websites. This function generates `~5 * 10^777` unique user-agents.
* **Free Proxy List Retrieval:** Fetch a list of free proxies with `get_free_proxies`, filtering by protocol if desired.

## Installation:

* **With pip:**
    ```bash
    pip install osn-requests
    ```

* **With git:**
    ```bash
    pip install git+https://github.com/oddshellnick/osn-requests.git
    ```

## Example Usage:

```python
from osn_requests import find_web_element, get_req, get_html
from osn_requests.user_agents import generate_random_user_agent
from osn_requests.proxies import get_free_proxies

user_agent = generate_random_user_agent()
print(f"Using User-Agent: {user_agent}")

http_proxies = get_free_proxies("http")
print(f"Found {len(http_proxies)} HTTP proxies")

html = get_html("https://www.example.com", headers={"User-Agent": user_agent}, proxies=http_proxies)

title_element = find_web_element(html, "//title")
if title_element is not None:
    print(f"Page Title: {title_element.text}")

json_data = get_req("https://api.example.com/data", headers={"User-Agent": user_agent}).json()
print(f"JSON Data: {json_data}")
```

## Future Notes

osn-requests is continually being developed and improved. Future plans include adding support for more advanced scraping techniques, expanding proxy management features, and incorporating additional utilities for handling various web data formats. Contributions and feature requests are welcome!
