Metadata-Version: 2.1
Name: aiblocklist
Version: 1.0.1
Summary: AI blocklist client: classify any domain as an AI tool (20,000+ domains, 18 categories, daily refresh, vendor training-on-your-data verdicts) and download the feed database for DNS sinkholes, proxies, firewalls and SOAR playbooks.
Home-page: https://www.aitoolsblocklist.com
Author: Alpha Quantum
Author-email: info@alpha-quantum.com
License: MIT
Project-URL: Homepage, https://www.aitoolsblocklist.com
Project-URL: Documentation, https://www.aitoolsblocklist.com/ai-blocklist-api.php
Project-URL: Source, https://github.com/explainableaixai/aiblocklist
Project-URL: Mirror, https://gitlab.com/url-classifications/aiblocklist
Project-URL: Tracker, https://www.aitoolsblocklist.com/contact.php
Project-URL: AI Tools Blocklist, https://www.aitoolsblocklist.com
Project-URL: AI Agent Allow List, https://www.aiagentallowlist.com
Project-URL: Shadow AI Tools, https://www.shadowaitools.com
Keywords: ai blocklist,ai tools blocklist,block ai tools,shadow ai,dns sinkhole,external dynamic list,web filtering,secure web gateway,dlp,acceptable use policy,generative ai,ai governance,domain classification,training on your data
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: Proxy Servers
Classifier: Topic :: Security
Classifier: Topic :: System :: Networking :: Firewalls
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests >=2.20.0

# aiblocklist

Python client for the AI Tools Blocklist API, a database of [classified AI tool domains](https://www.aitoolsblocklist.com) built for the teams that run proxies, DNS resolvers, DLP pipelines and security automation. One call classifies a domain; one download gives you the whole list for local matching.

What each record carries:

- 20,000+ AI-tool domains in 18 functional categories with subcategories, multi-label, rebuilt daily
- an AI type: `ai_native` for tools that are the AI, `ai_enabled` for products with an AI feature inside
- the vendor's data-use position: trains on input, opt-out available, enterprise tier exempt, API tier exempt, and the date the terms were checked
- for feed and database plans, the full CSV plus hosted EDL, PAC, hosts and DNS feeds

Only `requests` is required. Python 3.7 and newer.

---

## Installation

```bash
pip install aiblocklist
```

## Quick start

```python
from aiblocklist import AIBlocklistClient

client = AIBlocklistClient("YOUR_API_KEY")

r = client.check("chatgpt.com")
r.blocked               # True
r.primary_category      # "Text & Language"
r.ai_type               # "ai_native"
r.category_names        # ["Text & Language"]
r.subcategory_names     # ["General assistants & chatbots"]
r.trains_on_data        # "opt_out_default"
r.terms_checked         # "2026-09-17"
r.quota_remaining       # lookups left in the current 30-day cycle

client.check("example.com").blocked   # False
```

`check()` returns a `Lookup`, a `dict` subclass with the raw JSON and convenience properties, so `r["blocked"]` and `r.blocked` are the same value. The key comes from the account area at aitoolsblocklist.com and is sent as `X-API-Key`. Public methods (`stats()`, `categories()`, `clause()`) work without a key.

From the command line:

```bash
export ATB_API_KEY=YOUR_API_KEY
python -m aiblocklist chatgpt.com midjourney.com example.com
python -m aiblocklist --stats
```

## Methods

| Method | Endpoint | Key | Returns |
|---|---|---|---|
| `check(domain)` | `GET /api/check?domain=` | yes | `Lookup` |
| `is_blocked(domain)` | same | yes | `bool` |
| `check_many(domains, pause=0)` | same, sequential, deduplicated | yes | `list[Lookup]` |
| `data_use(domain)` | same | yes | dict of the five data-use fields |
| `feeds.status()` | `GET /api/database/?action=status` | feed or database plan | plan and file state |
| `feeds.database_info()` | `GET /api/database/?action=database_info` | feed or database plan | file name, timestamp, size |
| `feeds.download_database(path)` | `GET /api/database/?action=download_database` | feed or database plan | `{"file", "bytes"}` |
| `feeds.download_categories(path)` | `GET /api/database/?action=download_categories` | feed or database plan | `{"file", "bytes"}` |
| `stats()` | `GET /api/stats.php` | no | totals and 18 categories with counts |
| `categories()` | same | no | the category list |
| `clause(domain, field)` | `GET /api/data-use-clause.php` | no | `{"clause", "url"}` or `None` |

Pass a bare domain or any URL; `clean_domain()` strips scheme, path, port, credentials and a leading `www.`. Subdomains resolve to the registrable domain on the server.

## The response, field by field

```json
{
  "domain": "chatgpt.com",
  "blocked": true,
  "primary_category": "Text & Language",
  "ai_type": "ai_native",
  "categories": [{"category": "Text & Language", "subcategory": "General assistants & chatbots"}],
  "trains_on_data": "opt_out_default",
  "opt_out_available": "yes",
  "enterprise_no_training": "yes",
  "api_no_training": "yes",
  "terms_checked": "2026-09-17",
  "quota_remaining": 9999986
}
```

| Field | Values |
|---|---|
| `blocked` | `true` when the domain is a known AI tool, otherwise `false` with an empty `categories` list |
| `primary_category` | one of the 18 categories |
| `ai_type` | `ai_native` or `ai_enabled` |
| `categories` | every category and subcategory the tool belongs to |
| `trains_on_data` | `yes`, `no`, `opt_out_default`, `unstated` |
| `opt_out_available` | `yes`, `no`, `unstated` |
| `enterprise_no_training` | `yes`, `no`, `unstated` |
| `api_no_training` | `yes`, `no`, `unstated` |
| `terms_checked` | ISO date the vendor terms were last read |
| `quota_remaining` | lookups left on the plan |

Both found and not-found are HTTP 200. Errors are raised as exceptions (see below).

## Worked examples

### 1. FastAPI middleware for an internal egress service

An internal HTTP egress service (or any FastAPI app that proxies outbound requests) checks the destination host once, keeps the verdict for a day, and enforces a category policy. Assistants that train on input by default are allowed only with a header that the DLP layer downstream can key on.

```python
import time
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from aiblocklist import AIBlocklistClient, AIBlocklistError

app = FastAPI()
client = AIBlocklistClient("YOUR_API_KEY")
cache = {}                          # host -> (verdict, expires)
TTL = 86400
BLOCK = {"Image & Visual", "Audio & Voice", "Companions & Social"}


def verdict_for(host: str) -> str:
    now = time.time()
    hit = cache.get(host)
    if hit and hit[1] > now:
        return hit[0]
    try:
        r = client.check(host)
    except AIBlocklistError:
        return "allow"              # fail open on API trouble, log it
    verdict = "allow"
    if r.blocked:
        if BLOCK & set(r.category_names):
            verdict = "block"
        elif r.trains_on_data in ("yes", "opt_out_default"):
            verdict = "warn"
    cache[host] = (verdict, now + TTL)
    return verdict


@app.middleware("http")
async def ai_policy(request: Request, call_next):
    host = request.headers.get("x-target-host", "")
    v = verdict_for(host) if host else "allow"
    if v == "block":
        return JSONResponse({"error": "AI tool blocked by policy", "host": host}, status_code=403)
    response = await call_next(request)
    if v == "warn":
        response.headers["X-AI-Policy"] = "trains-on-input"
    return response
```

The same shape works as a Flask `before_request` hook. With a one-day cache, a gateway that sees a few thousand distinct hosts uses a few thousand lookups a month.

### 2. Classifying a proxy export with pandas

Take the unique hosts from a proxy or DNS export, classify them, and produce a table of AI tools with their categories and training terms. `check_many()` deduplicates, so the cost is one lookup per unique host.

```python
import pandas as pd
from aiblocklist import AIBlocklistClient

client = AIBlocklistClient("YOUR_API_KEY")

log = pd.read_csv("proxy_export.csv")               # columns: timestamp, user, host, bytes
hosts = log["host"].dropna().unique().tolist()

results = client.check_many(hosts, pause=0.05)
found = pd.DataFrame([
    {
        "domain": r.domain,
        "primary_category": r.primary_category,
        "subcategories": "; ".join(r.subcategory_names),
        "ai_type": r.ai_type,
        "trains_on_data": r.trains_on_data,
        "enterprise_no_training": r["enterprise_no_training"],
        "terms_checked": r.terms_checked,
    }
    for r in results if r.blocked
])

# join back to users so the report says who reached what
usage = log.merge(found, left_on="host", right_on="domain")
summary = usage.groupby(["domain", "primary_category", "trains_on_data"])["user"].nunique()
print(summary.sort_values(ascending=False).head(25))
found.to_csv("ai_tools_found.csv", index=False)
```

For exports with hundreds of thousands of lines, the hosted [shadow AI inventory](https://www.shadowaitools.com) service does this with per-user breakdowns, sanctioned lists and a PDF evidence pack, without writing the join yourself.

### 3. A nightly feed refresh for a DNS resolver

Feed and database plans download the full CSV. This cron job fetches it, keeps only the categories the organisation blocks, and writes an Unbound local-zone file. Swap the output format for dnsmasq, a hosts file or a firewall EDL as needed.

```python
#!/usr/bin/env python3
import csv, os, subprocess
from aiblocklist import AIBlocklistClient, PlanError

client = AIBlocklistClient(os.environ["ATB_API_KEY"])
BLOCK = {"Image & Visual", "Audio & Voice", "Companions & Social", "Agents & Automation"}
CSV_PATH = "/var/lib/aiblocklist/ai_tools_full.csv"
ZONE_TMP = "/etc/unbound/unbound.conf.d/ai-block.conf.tmp"
ZONE = "/etc/unbound/unbound.conf.d/ai-block.conf"

try:
    info = client.feeds.database_info()
except PlanError as exc:
    raise SystemExit("this key is lookup-only: %s" % exc.body.get("plan"))

print("database %s updated %s (%s)" % (info["database_file"], info["last_updated"], info["file_size_human"]))
client.feeds.download_database(CSV_PATH)

n = 0
with open(CSV_PATH, newline="", encoding="utf-8") as src, open(ZONE_TMP, "w") as out:
    out.write("server:\n")
    for row in csv.DictReader(src):
        if row.get("primary_category") in BLOCK:
            out.write('  local-zone: "%s." always_nxdomain\n' % row["domain"])
            n += 1
os.replace(ZONE_TMP, ZONE)
subprocess.run(["unbound-control", "reload"], check=False)
print("%d domains sinkholed" % n)
```

```
30 5 * * * /usr/bin/python3 /opt/aiblocklist/refresh_zone.py >> /var/log/aiblocklist.log 2>&1
```

### 4. Citing the vendor's own terms

`clause()` returns the verbatim sentence behind a public data-use field with the URL it came from, which is what an approval ticket or a policy exception needs.

```python
c = client.clause("chatgpt.com", "trains_consumer_default")
if c:
    print(c["clause"])
    print(c["url"])
```

Fields: `trains_consumer_default`, `optout_available`, `enterprise_no_training`, `api_no_training`.

## Error handling

```python
from aiblocklist import (
    AIBlocklistError, AuthenticationError, QuotaError, PlanError,
    BadRequestError, NotFoundError, ServiceUnavailableError,
)

try:
    r = client.check("notion.so")
except AuthenticationError:      # 401: no key or unknown key
    ...
except QuotaError as exc:        # 403: inactive account or quota exhausted
    print(exc.status, exc.body)
except ServiceUnavailableError:  # 503 after retries
    ...
except AIBlocklistError:         # anything else, including network failures
    ...
```

| Exception | HTTP | Meaning |
|---|---|---|
| `BadRequestError` | 400 | empty domain, or an unknown clause field |
| `AuthenticationError` | 401 | no key, or a key that matches no account |
| `QuotaError` | 403 | inactive account or monthly quota exhausted |
| `PlanError` | 403 | database endpoints on a lookup-only plan; `body["plan"]` names it |
| `NotFoundError` | 404 | database file not available for the account |
| `ServiceUnavailableError` | 503 | lookup service busy; retried `max_retries` times first |

The client is a context manager, so the HTTP session closes cleanly:

```python
with AIBlocklistClient("YOUR_API_KEY") as client:
    print(client.stats()["total_tools"])
```

## Configuration

```python
client = AIBlocklistClient(
    api_key="YOUR_API_KEY",
    base_url="https://www.aitoolsblocklist.com",   # default
    timeout=30,                                     # seconds; downloads use at least 300
    max_retries=2,                                  # on 503 and network errors
)
```

Keep the key out of source: read it from an environment variable or a secrets manager and pass it to the constructor. The command line entry reads `ATB_API_KEY` for that reason. Lookups are metered per call, so cache verdicts in your own process or datastore for a day; the list is rebuilt daily and a shorter cache buys nothing. Downloads are streamed to disk in 64 KB chunks, so a full database file never sits in memory.

## Why a classified AI tool list belongs in the security stack

The risk in AI tool usage is not the tool, it is the input. A contract pasted into a summariser, a customer list dropped into a spreadsheet assistant, a repository fed to a code explainer: each is an outbound request to a hostname, and the hostname is the only thing every control point on the network can see without an agent or TLS inspection. Classifying that hostname is the step that turns a policy into an enforceable rule.

### Data loss prevention needs the destination, not just the content

[Data loss prevention](https://en.wikipedia.org/wiki/Data_loss_prevention_software) systems inspect what leaves. They cannot decide whether leaving is acceptable without knowing where it is going and what the recipient does with it. The [AI tool classification database](https://www.aitoolsblocklist.com) supplies both: the functional category of the destination and the vendor's own position on training, opt-out and enterprise exemptions. A DLP rule that says "block source code to AI assistants that train on input" is expressible only with that data.

### Frameworks ask for an inventory first

The [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) starts its Map function with knowing which AI systems are in use and which third parties receive data. The [ENISA](https://www.enisa.europa.eu/) guidance on AI cybersecurity takes the same order: identify, then govern. Under the [EU AI Act](https://eur-lex.europa.eu/eli/reg/2024/1689/oj), deployers carry obligations that depend on knowing which systems staff use. None of this is possible from a survey; it is possible from network evidence joined to a classified list.

### Prompt-borne exposure is a recognised risk class

The [OWASP Top 10 for LLM Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/) lists sensitive information disclosure among its leading risks. For an organisation that does not build LLM applications but whose staff use hundreds of them, the mitigation is at the network edge: know which destinations are AI tools, permit the ones whose terms are acceptable, and block or warn on the rest. Categories and subcategories make that policy fine-grained; the daily rebuild keeps it current as tools launch.

## Neighbouring products

This package governs what people reach. What an organisation's own AI agents may open on the web is the reverse question, answered by the [AI agent allow list](https://www.aiagentallowlist.com): verified page-type URLs for 40 million+ domains, up to 28 types per domain, and a per-URL allow or deny verdict so a browsing agent can read documentation and pricing while staying off login, checkout and upload pages. Its client is `aiagentallowlist`. Together the two give a [per-URL policy for browsing agents](https://www.aiagentallowlist.com) and a per-domain policy for staff.

Before either policy is written, most teams want to [find the AI tools employees use](https://www.shadowaitools.com) today. That service takes a DNS, proxy or firewall export, matches it against this same list, and returns the tools found with categories, risk flags, training verdicts, a per-user breakdown and a dated CSV and PDF.

## Frequently asked questions

**What is the AI Tools Blocklist?**
A daily-refreshed database of 20,000+ AI-tool domains classified into 18 functional categories with subcategories, with the vendor's data-use terms attached to each record. It is delivered as a lookup API, as downloadable CSV and JSON, and as hosted feeds in EDL, PAC, hosts and DNS formats from [aitoolsblocklist.com](https://www.aitoolsblocklist.com).

**How do I check whether a domain is an AI tool in Python?**
`pip install aiblocklist`, then `AIBlocklistClient("YOUR_API_KEY").check("domain.com").blocked`. The same `Lookup` object carries the category, subcategories, AI type and the five data-use fields.

**How do I find out whether an AI vendor trains on my data?**
`client.data_use("domain.com")` returns `trains_on_data`, `opt_out_available`, `enterprise_no_training`, `api_no_training` and `terms_checked`. `client.clause("domain.com", "trains_consumer_default")` returns the verbatim clause with its URL.

**Can I download the whole list?**
Feed and database plans can: `client.feeds.download_database(path)` streams the CSV, `download_categories(path)` the category tree. The Lookup API plan is lookup-only and receives a `PlanError` on those calls.

**Is there a bulk endpoint?**
No. `check_many()` runs sequential lookups, deduplicated, with an optional pause. For large lists, download the database once and match locally.

**How current is the list?**
Rebuilt daily. `client.stats()` shows the live totals; `client.feeds.database_info()` shows the timestamp of your file.

**Does the list work with schools and CIPA filtering?**
Yes. Districts use the categories to allow approved tutors while blocking essay generators, deepfake tools and companion chatbots, alongside the general filtering database at [cipawebfiltering.com](https://www.cipawebfiltering.com).

**Who builds it?**
Alpha Quantum, also behind the [website categorization API](https://www.websitecategorizationapi.com), the [web filtering database](https://www.webfilteringdatabase.com), the [AI agent allow list](https://www.aiagentallowlist.com) and the [shadow AI inventory](https://www.shadowaitools.com) service.

## Related packages

- [`aiblocklist` on npm](https://www.npmjs.com/package/aiblocklist), the Node.js version of this client
- [`aitoolsblocklist`](https://pypi.org/project/aitoolsblocklist/) on PyPI and [on npm](https://www.npmjs.com/package/aitoolsblocklist), the original AI Tools Blocklist client
- [`shadowaitools`](https://pypi.org/project/shadowaitools/) on PyPI and [on npm](https://www.npmjs.com/package/shadowaitools), local log scanner for shadow AI tools
- [`aiagentallowlist`](https://pypi.org/project/aiagentallowlist/) on PyPI and [on npm](https://www.npmjs.com/package/aiagentallowlist), client for the AI agent allow list
- [`phishingdetectionapi`](https://pypi.org/project/phishingdetectionapi/) on PyPI and [on npm](https://www.npmjs.com/package/phishingdetectionapi), from [phishingdetectionapi.com](https://www.phishingdetectionapi.com)
- [`webfilteringdatabase`](https://www.npmjs.com/package/webfilteringdatabase) on npm, from [webfilteringdatabase.com](https://www.webfilteringdatabase.com)
- [`websiteclassificationapi`](https://pypi.org/project/websiteclassificationapi/) on PyPI and [`websitecategorization`](https://www.npmjs.com/package/websitecategorization) on npm, from [websitecategorizationapi.com](https://www.websitecategorizationapi.com)
- [`cipawebfiltering`](https://pypi.org/project/cipawebfiltering/) on PyPI and [on npm](https://www.npmjs.com/package/cipawebfiltering)
- [PII detection API](https://www.piidetectionapi.com) for prompts and DLP logs

Source: [github.com/explainableaixai/aiblocklist](https://github.com/explainableaixai/aiblocklist), mirror at [gitlab.com/url-classifications/aiblocklist](https://gitlab.com/url-classifications/aiblocklist).

## Links

- Product, plans and feeds: [https://www.aitoolsblocklist.com](https://www.aitoolsblocklist.com)
- NIST AI Risk Management Framework: [https://www.nist.gov/itl/ai-risk-management-framework](https://www.nist.gov/itl/ai-risk-management-framework)
- OWASP Top 10 for LLM Applications: [https://owasp.org/www-project-top-10-for-large-language-model-applications/](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
- ENISA: [https://www.enisa.europa.eu/](https://www.enisa.europa.eu/)
- EU AI Act, Regulation (EU) 2024/1689: [https://eur-lex.europa.eu/eli/reg/2024/1689/oj](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)
- Data loss prevention software: [https://en.wikipedia.org/wiki/Data_loss_prevention_software](https://en.wikipedia.org/wiki/Data_loss_prevention_software)

## License

MIT
