Metadata-Version: 2.4
Name: urlcache2db
Version: 0.1.2
Summary: A lightweight Python library that fetches URL content and stores it in a local SQLite database for a specified amount of time.
Home-page: https://github.com/artofscripting/urlcache2db
Author: ArtOfScripting
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.0.0
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# URLcache2db

A lightweight Python library that fetches URL content and stores it in a local SQLite database for a specified amount of time. If a URL is requested again within the cache expiration window, the library returns the cached content instead of making a new HTTP request.

## Features
- Avoid redundant HTTP requests by caching responses locally.
- Easy integration with standard SQLite.
- Supports sending custom `headers` and `data` with requests, differentiating cache entries automatically based on request payload.
- Cleans up expired cache entries automatically.

## Requirements
- Python 3.x
- `requests`

## Installation
You can install the dependencies via the included `requirements.txt`:
```bash
pip install -r requirements.txt
```

## Basic Usage

```python
from urlcache import URLCache

# Initialize to cache for 1 hour (3600 seconds) in "my_cache.db"
cache = URLCache(db_path="my_cache.db", expiration_seconds=3600)

# Fetch the content. Will make an HTTP request on the first call.
content = cache.get("https://httpbin.org/get")

# Call it again. If it happens within an hour, it skips the HTTP request 
# and returns the cache from SQLite immediately.
cached_content = cache.get("https://httpbin.org/get")
```

## Advanced Usage (Headers & Data)
The cache key is uniquely generated based on the URL, headers, and data. Changing the headers or payload data will result in a separate cache entry.

```python
from urlcache import URLCache

cache = URLCache(expiration_seconds=600)

custom_headers = {
    "Authorization": "Bearer 1234567890"
}

payload_data = {
    "search": "python",
    "limit": 10
}

# First time: makes the GET request
content = cache.get("https://httpbin.org/get", headers=custom_headers, data=payload_data)

# Second time within 10 minutes: uses the cached version
cached_content = cache.get("https://httpbin.org/get", headers=custom_headers, data=payload_data)
```

## SSL Verification & Custom User-Agent
You can easily bypass SSL certificate verification for self-signed or expired certificates using `verify=False`, and conveniently set a custom User-Agent string:

```python
from urlcache import URLCache

cache = URLCache()

# Fetch content bypassing SSL verification and setting a specific User-Agent
content = cache.get(
    "https://expired.badssl.com/", 
    user_agent="MyCustomApp/2.0", 
    verify=False
)
```

## Maintenance
You can periodically remove expired entries from your database using:
```python
cache.clear_expired()
```
