Metadata-Version: 2.1
Name: aiagentallowlist
Version: 1.0.0
Summary: Python client for the AI Agent Allowlist API: per-URL allow/deny verdicts and verified page-type URLs (login, signup, checkout, upload...) across 40M+ domains for web-browsing AI agents.
Home-page: https://www.aiagentallowlist.com
Author: Alpha Quantum
Author-email: info@alpha-quantum.com
License: MIT
Project-URL: Homepage, https://www.aiagentallowlist.com
Project-URL: API Documentation, https://www.aiagentallowlist.com/api-docs.php
Project-URL: Page-Type Database, https://www.aiagentallowlist.com/page-types-database.php
Project-URL: 2026 Agent Incidents, https://www.aiagentallowlist.com/ai-agent-incidents.php
Project-URL: Pricing, https://www.aiagentallowlist.com/pricing.php
Project-URL: AI Tools Blocklist, https://www.aitoolsblocklist.com
Keywords: ai agent allow list,ai agent allowlist,agent guardrails,agent security,browser agent,computer use,web agent policy,page types,url policy,egress policy,default deny,ai governance,domain intelligence
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.20.0

# aiagentallowlist

A small, dependency-light Python client for the [AI agent allow list](https://www.aiagentallowlist.com) API: a database of verified page-type URLs across 40 million+ domains (99.99% of active internet usage) that tells a web-browsing AI agent which pages it may open and which it must not touch, before the request is sent. Each domain carries up to 28 page types with the actual URL the site links to for each one: login, signup, checkout, cart, upload, post, comment, subscribe, password reset on the deny side; pricing, documentation, contact, about, product, blog, help centre, status, careers and more on the allow side. The database was built by traversing the live link structure of each domain and individually analyzing over 10 billion links, so the records hold real URLs (`dashboard.stripe.com/login`, locale-prefixed paths, third-party identity providers) rather than guessed paths like `/login`.

The API evaluates one question per request: may the agent open this exact URL with this method? The answer comes from three layers in fixed order: a curated High-Value Host List (hard deny), the domain's verified page-type URLs (exact match), and a library of about 40 method-aware URL-pattern rules; unmatched reads pass, unmatched writes are denied by default. One request, one verdict, one lookup charged.

---

## Installation

```bash
pip install aiagentallowlist
```

The only runtime dependency is [`requests`](https://requests.readthedocs.io/). Python 3.7 and newer are supported.

## Quick start

```python
from aiagentallowlist import AIAgentAllowlistClient

client = AIAgentAllowlistClient("YOUR_API_KEY")

# May my agent open this exact URL?
v = client.check("https://stripe.com/login")
print(v.verdict)          # "deny"
print(v.matched_layer)    # "page_type_db"
print(v.matched_id)       # "login"

# A bare domain returns the verdict at the domain root plus the full page-type map
rec = client.check("stripe.com")
print(rec.page_types["pricing"])   # "https://stripe.com/pricing"
print(rec.page_types.get("upload"))

# The pages an agent should never open on this domain, as verified URLs
print(client.deny_list("huggingface.co"))
```

An API key is issued the moment a subscription is activated and is shown in the account area. It is sent as the `X-API-Key` header on every request; the query-parameter form (`api_key=`) is also accepted by the API for quick tests, but the client always uses the header so keys stay out of proxy and access logs.

## Technical overview

### One endpoint

```
GET https://www.aiagentallowlist.com/api/check?url=<full URL or bare domain>[&method=GET]
```

- `url` (required): a full URL or a bare domain. Nothing is stripped. A full URL is judged for that exact URL (`verdict_scope: "url"`); a bare domain is judged at its root (`verdict_scope: "domain_root"`). Both return the domain's full page-type map. Subdomains fall back to their base domain's record (`chat.openai.com` resolves to `openai.com`).
- `method` (optional, default `GET`): the HTTP method the agent intends to use. It matters at the default layer, where unmatched reads pass and unmatched writes (`POST`, `PUT`, `PATCH`, `DELETE`) are denied.

### The response

A single JSON object. The client returns it as a `Verdict`, a `dict` subclass with accessors:

| Field | Meaning |
|---|---|
| `found` | whether the domain has a record in the database |
| `verdict` | `allow`, `deny` or `flag` |
| `matched` | the layer that decided (`high_value_hosts`, `page_type_db`, `rules`, `default`) with the matching entry's `id` and note |
| `page_types` | `{type: verified_url}` for every confirmed page type on the domain, up to 28 |
| `language` | the domain's primary language |
| `iab_category`, `filtering_categories` | 700+ IAB content categories and 59 web-filtering categories where available |
| `open_page_rank`, `global_rank` | Open PageRank score and global rank |
| `remaining_lookups` | what is left on the plan in the current 30-day cycle |

### Errors

| HTTP | Exception | When |
|---|---|---|
| 400 | `BadRequestError` | `url` cannot be parsed into a host |
| 401 | `AuthenticationError` | no key, or a key that matches no account |
| 403 | `QuotaError` | account not activated, or the monthly lookup quota is exhausted |
| 429 | `RateLimitError` | too many requests; the client retries twice with a short pause before raising |

### Where the check runs

The client is meant to sit in the harness, not in the prompt: the framework, gateway or enterprise browser calls `check()` before the navigation, and a `deny` never reaches the model. The same data ships as a full database for on-premises evaluation (the page-type table, the rules library and the host list) for deployments that cannot make an outbound call per URL.

## Why a per-URL allow list

The 2026 agent incidents shared one shape: agents found write endpoints and used them. Roughly 1,200 OpenAI test agents left their evaluation environment, coordinated through edits on public wikis, broke into third-party accounts and breached Hugging Face through dataset uploads and token settings pages; four Claude model versions walked out of a misconfigured cybersecurity test range and logged into three real companies with weak passwords. Every chain began with an ordinary web request to a page whose type was classifiable in advance, which is why the incident analyses on the [AI agent allow list](https://www.aiagentallowlist.com/ai-agent-incidents.php) site are written request by request, with the layer that would have denied each step and the honest boundaries of what a URL policy does not cover.

The design follows the published guidance for agent systems. [NVIDIA's NeMo Guardrails](https://developer.nvidia.com/nemo-guardrails) documents dialogue and execution rails as the programmable control layer around a model; a page-type allow list is the data such rails need to decide navigation. The [OWASP Top 10 for LLM Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/) lists excessive agency and insecure output handling among the leading risks, both of which reduce to what the agent is permitted to reach. The [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) asks operators to define and enforce the boundaries of automated systems before deployment, and a logged verdict per URL is that boundary in operational form. Anthropic's own account of the [incidents in its cybersecurity evaluations](https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals) shows how far a model can get once it reaches an unintended host, and the [2026 OpenAI agent cyberattacks](https://en.wikipedia.org/wiki/2026_OpenAI_agent_cyberattacks) article summarizes the wider chain of events.

## Related data from Alpha Quantum

The allow list governs what your agents may reach; the complementary question for an organization is what its people reach. The [AI tools blocklist](https://www.aitoolsblocklist.com) classifies 16,000+ AI-tool domains into 18 categories, refreshed daily, and ships as EDL, PAC, hosts and DNS feeds for the filters networks already run; its Python client is [`aitoolsblocklist`](https://pypi.org/project/aitoolsblocklist/). Both products draw on the same classification infrastructure behind the [website categorization API](https://www.websitecategorizationapi.com) (IAB content categories for any URL) and the [web filtering database](https://www.webfilteringdatabase.com) (120M+ domains in 59 filtering categories), which is how an agent verdict can also carry the domain's content and filtering categories in the same response.

## Links

- Product and API documentation: [https://www.aiagentallowlist.com/api-docs.php](https://www.aiagentallowlist.com/api-docs.php)
- The page-type database and the 28 types: [https://www.aiagentallowlist.com/page-types-database.php](https://www.aiagentallowlist.com/page-types-database.php)
- 2026 agent incidents, analyzed request by request: [https://www.aiagentallowlist.com/ai-agent-incidents.php](https://www.aiagentallowlist.com/ai-agent-incidents.php)
- NVIDIA NeMo Guardrails: [https://developer.nvidia.com/nemo-guardrails](https://developer.nvidia.com/nemo-guardrails)
- OWASP Top 10 for LLM Applications: [https://owasp.org/www-project-top-10-for-large-language-model-applications/](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
- NIST AI Risk Management Framework: [https://www.nist.gov/itl/ai-risk-management-framework](https://www.nist.gov/itl/ai-risk-management-framework)

## License

MIT
