Metadata-Version: 2.4
Name: gemini-scraper
Version: 0.1.0
Summary: Gemini scraper and Gemini API endpoint client built on ScrapingBee. Send prompts to Google Gemini, get text, markdown and citations as JSON.
Author: wordstotech
License: MIT
Project-URL: Homepage, https://www.scrapingbee.com/features/gemini/
Project-URL: Documentation, https://www.scrapingbee.com/documentation/gemini/
Project-URL: Repository, https://github.com/ScrapingBee/gemini-scraper
Keywords: gemini-api,gemini-scraper,gemini-api-endpoint,gemini-scraper-api,google-gemini,ai-scraping,web-scraping-api,scraping-api,llm-scraping,scrapingbee
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.25.0
Dynamic: license-file

# gemini-scraper

A Python client for the [Gemini API endpoint](https://www.scrapingbee.com/documentation/gemini/)
on ScrapingBee. Send a prompt to Google Gemini, get back the answer as text, as markdown, and with
citations when Gemini attaches them. No headless browser to run, no Google account to manage.

Verified against the live endpoint on 2026-08-25: `GET https://app.scrapingbee.com/api/v1/gemini`,
15 credits per successful call.

## Install

```bash
pip install gemini-scraper
```

Python 3.8+ and `requests`. You need a ScrapingBee key from
[app.scrapingbee.com](https://app.scrapingbee.com/), which starts with 1,000 free credits.

## One prompt

```python
from gemini_scraper import GeminiScraper

scraper = GeminiScraper("YOUR-API-KEY")
answer = scraper.ask("Best programming languages for data science")

print(answer.markdown)
print(answer.citations)
print(answer.cost, answer.request_id)
```

## What the Gemini scraper returns

The endpoint answers with five fields. These are the real keys observed on a live call, not a
guess at the shape:

| Field | Property | What it holds |
| --- | --- | --- |
| `results_text` | `answer.text` | The answer as plain text |
| `results_markdown` | `answer.markdown` | The same answer with formatting preserved |
| `citations` | `answer.citations` | Sources Gemini attached, when it attaches any |
| `full_html` | `answer.html` | Rendered HTML of the answer pane |
| `prompt` | `answer.prompt` | The prompt echoed back |

Citations are returned when the prompt produces them. Treat the list as best effort rather than a
guarantee, and check `len(answer.citations)` before relying on it.

## Localising an answer

Gemini answers differently depending on where the request comes from. Anything involving pricing,
availability, regulation or local providers is worth running per market:

```python
for market in ["us", "gb", "de", "fr"]:
    answer = scraper.ask("best project management software", country_code=market)
    print(market, len(answer.citations), answer.text[:120])
```

## Running a prompt set

Brand visibility work means the same prompt list on a schedule. `ask_many` keeps going when a
single prompt fails, returning `None` in that slot rather than losing the batch:

```python
prompts = [
    "best web scraping api",
    "how do I scrape a javascript heavy site",
    "cheapest way to collect serp data",
]

for prompt, answer in zip(prompts, scraper.ask_many(prompts, country_code="us")):
    if answer is None:
        print("failed:", prompt)
        continue
    print(prompt, "->", answer.text[:100])
```

At 15 credits per call, a 100-prompt set costs 1,500 credits per run. Check the balance first:

```python
print(scraper.usage())
# {'max_api_credit': ..., 'used_api_credit': ..., 'max_concurrency': ..., 'current_concurrency': ...}
```

`usage()` is free and capped at 6 calls per minute.

## Tagging requests

`tag` attaches your own identifier so a call can be found later in the dashboard, which matters
when several jobs share one key:

```python
scraper.ask("best crm for small teams", tag="visibility-run-2026-08")
```

## Errors

`GeminiScraperError` carries `status_code`, `body` and `request_id`. Quote the request id to
support. HTTP 500 responses are not charged credits, so a retry costs nothing:

```python
from gemini_scraper import GeminiScraper, GeminiScraperError

try:
    answer = scraper.ask("...")
except GeminiScraperError as error:
    print(error.status_code, error.request_id)
```

## Cost

| Call | Credits |
| --- | --- |
| One Gemini prompt | 15 |
| `usage()` | 0 |
| Failed request (HTTP 500) | 0 |

Plan tiers at [scrapingbee.com/pricing](https://www.scrapingbee.com/pricing/).

## Related scrapers

The same account also reaches
[ChatGPT](https://www.scrapingbee.com/features/chatgpt/),
[Google search](https://www.scrapingbee.com/features/google/),
[Google AI Mode](https://www.scrapingbee.com/scrapers/google-ai-mode-api/),
[Google News](https://www.scrapingbee.com/scrapers/google-news-scraper-api/) and
[Perplexity](https://www.scrapingbee.com/scrapers/perplexity-api/), which is the usual set for
tracking how a brand shows up across AI answers.

## Scope

Public, pre-login content only. Scraping behind login credentials is prohibited by the
[ScrapingBee terms](https://www.scrapingbee.com/terms-and-conditions/). Keep your API key out of
shared environments, including AI coding assistants.

More: [Gemini feature page](https://www.scrapingbee.com/features/gemini/) .
[documentation](https://www.scrapingbee.com/documentation/) .
[code in eight languages](https://github.com/ScrapingBee/gemini-scraper)

MIT licensed.
