Metadata-Version: 2.4
Name: httpcheck
Version: 1.5.1
Summary: HTTP status code checker with advanced features
Author-email: Thomas Juul Dyhr <thomas@dyhr.com>
License: MIT
Project-URL: Homepage, https://github.com/docdyhr/httpcheck
Project-URL: Bug Tracker, https://github.com/docdyhr/httpcheck/issues
Project-URL: Documentation, https://github.com/docdyhr/httpcheck/blob/master/README.md
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: System :: Networking :: Monitoring
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.33.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: tabulate>=0.9.0
Requires-Dist: tqdm>=4.67.0
Requires-Dist: validators>=0.22.0
Requires-Dist: tomli>=2.0.0; python_version < "3.11"
Requires-Dist: urllib3>=2.8.0
Requires-Dist: certifi>=2026.10.19
Requires-Dist: idna>=3.15
Requires-Dist: anyio>=4.14.2
Requires-Dist: h11>=0.16.0
Provides-Extra: macos
Requires-Dist: rumps>=0.4.0; extra == "macos"
Requires-Dist: pyobjc-framework-Cocoa>=10.0; extra == "macos"
Requires-Dist: pyobjc-framework-Foundation>=10.0; extra == "macos"
Requires-Dist: pyobjc-framework-AppKit>=10.0; extra == "macos"
Requires-Dist: pyobjc-core>=10.0; extra == "macos"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-mock>=3.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: pytest-benchmark>=5.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
Requires-Dist: pylint>=3.1.0; extra == "dev"
Requires-Dist: mypy==2.4.0; extra == "dev"
Requires-Dist: types-requests>=2.31.0; extra == "dev"
Requires-Dist: types-tabulate>=0.9.0; extra == "dev"
Requires-Dist: types-tqdm>=4.66.0; extra == "dev"
Requires-Dist: tomli>=2.0.0; extra == "dev"
Provides-Extra: build
Requires-Dist: py2app>=0.28; extra == "build"
Dynamic: license-file

<div align="right">

<a href="https://railway.com?referralCode=QhjuBc">

  <img width="160" src="https://raw.githubusercontent.com/docdyhr/.github/main/assets/railway-corner-v2@2x.png" alt="Deploy on Railway — $20 free credits">

</a>

</div>

[![PyPI version](https://badge.fury.io/py/httpcheck.svg)](https://badge.fury.io/py/httpcheck)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/httpcheck)](https://pypi.org/project/httpcheck/)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/httpcheck)](https://pypi.org/project/httpcheck/)
[![CI/CD Pipeline](https://github.com/docdyhr/httpcheck/actions/workflows/ci.yml/badge.svg)](https://github.com/docdyhr/httpcheck/actions/workflows/ci.yml)
[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit)
[![GitHub issues](https://img.shields.io/github/issues/docdyhr/httpcheck)](https://github.com/docdyhr/httpcheck/issues)
![GitHub repo size](https://img.shields.io/github/repo-size/docdyhr/httpcheck)
![GitHub](https://img.shields.io/github/license/docdyhr/httpcheck)
[![Docs](https://img.shields.io/badge/docs-readthedocs-brightgreen)](https://httpcheck.readthedocs.io/)

# Check Websites HTTP Status Codes with httpcheck

<img src="images/onSiteLogo.png" alt="onSite Logo" width="50%">

* **Name**: httpcheck (CLI) / onSite (Menu Bar App)
* **Current Version**: 1.5.1 (security patch for 1.5.0: Async I/O & Configuration)
* **Next Version**: 1.6.0 (Monitoring Mode, target Q1 2027)
* **Programming Language**: Python 3.10+
* **Author**: Thomas Juul Dyhr
* **Purpose**: Advanced HTTP status checker with monitoring capabilities
* **Development Status**: Active - See [ROADMAP.md](ROADMAP.md) for development plan
* **PyPI**: https://pypi.org/project/httpcheck/

## 🚀 Release Status

**httpcheck v1.5.1 has been released! 🎉**

✅ **v1.5.1 (security patch):** installing httpcheck now also requires patched
versions of `urllib3`, `certifi`, `idna`, `anyio` and `h11`, so an install into
an existing environment cannot keep vulnerable copies. No behaviour changes.

✅ **NEW in v1.5.0:**
- **Configuration Files**: User-defined defaults in `~/.httpcheck.toml` and
  `./.httpcheck.toml` (the project file overrides the user file); CLI flags always
  take precedence
- **Async I/O** (experimental): `--async` uses an httpx-based concurrent path. In
  the [benchmark against v1.4.3](docs/development/ASYNC_BENCHMARK.md) it is not
  faster than `-f` at the same `--workers`; for speed, raise `-f --workers`
- **Python 3.10+**: v1.5.0 drops Python 3.9 support (pip on Python 3.9 keeps
  installing 1.4.3); Python 3.13 and 3.14 are tested in CI
- **Retry fix**: with `--follow-redirects http-only` or `https-only`, retries now
  work (1.4.3 reported the site as failed without retrying)

✅ **v1.4.2-1.4.3 (Published to PyPI):**
- **Structured Logging**: --debug, --log-file, --log-json flags for production monitoring
- **Comprehensive Testing**: 297 tests with 88% coverage (CLI: 94%)
- **Professional Documentation**: 24-page documentation with ReadTheDocs theme
- **Performance Benchmarks**: 18 automated regression tests
- **PyPI Published**: Available via `pip install httpcheck`
- **Automated Publishing**: GitHub Actions with Trusted Publishers (OIDC)

✅ **v1.4.0-1.4.1 Foundation:**
- **Modular Architecture**: 8 specialized modules
- **Enhanced Security**: Enterprise-grade input validation with pip-audit clean
- **Multiple Output Formats**: JSON and CSV support
- **Advanced Request Control**: Custom headers, SSL options

🔭 **Next: v1.6.0 (target Q1 2027)**
- **Monitoring Mode**: Continuous site monitoring with notifications
- **Enhanced UX**: Colored output and improved progress reporting

See detailed plans in:
- [DEVELOPMENT_PLAN.md](docs/development/DEVELOPMENT_PLAN.md) - Executive overview and timeline
- [DEVELOPMENT.md](docs/development/DEVELOPMENT.md) - Technical implementation guide
- [TODO.md](TODO.md) - Current prioritized task list
- [ROADMAP.md](ROADMAP.md) - Long-term vision through v2.0.0

## 📦 Installation

### From PyPI (Recommended)

```bash
pip install httpcheck
```

### From Source

```bash
git clone https://github.com/docdyhr/httpcheck.git
cd httpcheck
pip install -e .
```

### With Optional Dependencies

```bash
# For macOS menu bar app
pip install httpcheck[macos]

# For development
pip install httpcheck[dev]
```

### Upgrade to Latest Version

```bash
pip install --upgrade httpcheck
```

## usage:

```
httpcheck [-h] [-t] [--disable-tld-checks] [--tld-warning-only]
          [--update-tld-list] [--tld-cache-days TLD_CACHE_DAYS]
          [-q | -v | -c | -f] [--timeout TIMEOUT] [--retries RETRIES]
          [--workers WORKERS] [--file-summary]
          [--comment-style {hash,slash,both}]
          [--follow-redirects {always,never,http-only,https-only}]
          [--max-redirects MAX_REDIRECTS] [--show-redirect-timing]
          [--output {table,json,csv}] [--version] [site ...]
```

### positional arguments

  site                   return http status codes for one or more websites

### optional arguments

  -h, --help             show this help message and exit
  -t, --tld              check if domain is in global list of TLDs
  --disable-tld-checks   disable TLD validation checks
  --tld-warning-only     show warnings for invalid TLDs without failing
  --update-tld-list      force update of the TLD list from publicsuffix.org
  --tld-cache-days TLD_CACHE_DAYS
                         number of days to keep the TLD cache valid (default: 30)
  -q, --quiet            only print errors
  -v, --verbose          increase output verbosity
  -c, --code             only print status code
  -f, --fast             fast check wtih threading
  --timeout TIMEOUT      set the timeout for each request
  --retries RETRIES      set the number of retries for each request
  --retry-delay DELAY    delay in seconds between retry attempts (default: 1.0)
  --workers WORKERS      set the number of worker threads
  -H, --header HEADER    add custom HTTP header (can be used multiple times)
  --no-verify-ssl        disable SSL certificate verification
  --file-summary         show summary of file parsing results (valid URLs, comments, etc.)
  --comment-style {hash,slash,both}
                         comment style to recognize: hash (#), slash (//), or both (default: both)
  --follow-redirects {always,never,http-only,https-only}
                         control redirect following behavior (default: always)
  --max-redirects MAX_REDIRECTS
                         maximum number of redirects to follow (default: 30)
  --show-redirect-timing show detailed timing for each redirect in the chain
  --output {table,json,csv}
                         output format: table (default), json, or csv
  --version              show program's version number and exit

### additional information

  enter sites in url or 'no' url form: 'httpcheck duckduckgo.com'
  read sites from a file: 'httpcheck @domains.txt'.

  output formats:
  - Table (default): Human-readable columnar output
  - JSON: Machine-parsable format for automation
  - CSV: Comma-separated values for spreadsheets

  Example: 'httpcheck google.com github.com --output json'

  [List of HTTP status codes](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)

## installation

clone repo and install requirements.

### clone the repository

```shell
git clone https://github.com/docdyhr/httpcheck
cd httpcheck/
```

### install requirements

```shell
python3 -m pip install . --user
```

On macOS, install terminal-notifier for notifications:

```shell
brew install terminal-notifier
```

### alternative install

set up a virtual environment with venv.

```shell
python3 -m venv .venv
source .venv/bin/activate
pip install -e .
```

or

```shell
mv httpcheck.py ~/bin/httpcheck # any other bin folder
chmod + ~/bin/httpcheck
```

## features

* Check HTTP status codes for one or more websites
* Support for macOS notifications on failures
* **NEW: Menu bar app for continuous monitoring**
  * Live status indicator in menu bar (🟢/🔴)
  * Background checking with configurable intervals
  * Rich native macOS notifications with actions
  * Click notifications to open failing sites
  * Easy site management through menu interface
* Advanced redirect tracking and control
  * Follow all redirects, none, or limit by protocol (HTTP/HTTPS)
  * Configure maximum number of redirects to follow
  * View detailed per-hop timing information in redirect chains
* Enhanced TLD validation
  * Caches TLD list for better performance
  * Auto-updates from Public Suffix List
  * Configurable cache duration
  * Optional warning-only mode
* Enhanced file input handling
  * Support for comments using # or // style
  * Handles inline comments
  * Automatically strips whitespace
  * Detailed file parsing statistics
* Threading support for faster checks
* Progress bar for multiple site checks
* Configurable timeouts and retries
* Pipe and file input support
* **NEW: Multiple output formats**
  * JSON format for easy parsing and automation
  * CSV format for spreadsheet integration
  * Table format (default) for human readability
  * Verbose mode includes redirect details in all formats
* **NEW: Request customization options**
  * Custom HTTP headers support with -H flag
  * Configurable retry delay between attempts
  * SSL certificate verification control
  * Enhanced timeout configuration

## usage examples

```shell
python3 httpcheck.py https://duckduckgo.com
```

returns ex. 'duckduckgo.com 200'

### python from venv

```shell
python httpcheck.py -v https://api.github.com/invalid
```

returns verbose output ex. '[-] api.github.com --> Client errors: 404 Not Found'

### installed as binary

```shell
httpcheck -q @domains.txt
```

only returns http status code errors ex. 'api.github.com 404'

```shell
httpcheck -c 1.1.1.1
```

status code only ex. '200'

### notifications

On macOS, the tool will show notifications when sites fail checks. For sites with failures:

* Shows failure count
* Lists failed sites (when less than 10)
* Groups notifications to avoid duplicates

### advanced usage examples

Control redirect behavior:

```shell
# Don't follow any redirects
httpcheck --follow-redirects never example.com

# Only follow redirects to HTTP URLs (not HTTPS)
httpcheck --follow-redirects http-only example.com

# Only follow redirects to HTTPS URLs (not HTTP)
httpcheck --follow-redirects https-only example.com

# Show detailed timing for each redirect
httpcheck --show-redirect-timing http://example.com

# Limit redirect chain length
httpcheck --max-redirects 5 example.com

# Show timing for each redirect hop
httpcheck -v --show-redirect-timing example.com
```

### tld validation examples

Control TLD validation behavior:

```shell
# Enable TLD validation
httpcheck -t example.com

# Force update of the TLD list from Public Suffix List
httpcheck -t --update-tld-list example.com

# Warn about invalid TLDs but don't fail
httpcheck -t --tld-warning-only example.com

# Disable TLD checks completely
httpcheck --disable-tld-checks example.com

# Set cache expiration to 60 days
httpcheck -t --tld-cache-days 60 example.com
```

### output format examples

Export results in different formats:

```shell
# JSON output for single site
httpcheck google.com --output json

# JSON output for multiple sites with verbose details
httpcheck google.com github.com --output json -v

# CSV output for spreadsheet import
httpcheck @domains.txt --output csv

# CSV with full details including redirects
httpcheck @domains.txt --output csv -v

# Process JSON output with jq
httpcheck @sites.txt --output json | jq '.[] | select(.status != "200")'

# Save CSV report with date
httpcheck @sites.txt --output csv -v > report_$(date +%Y%m%d).csv

# Filter errors only in JSON format
httpcheck @sites.txt --output json -q | jq '.[] | .domain'
```

### request customization examples

Add custom headers and control SSL verification:

```shell
# Add authorization header
httpcheck -H "Authorization: Bearer token123" api.example.com

# Multiple custom headers
httpcheck -H "User-Agent: MyBot 1.0" -H "Accept: application/json" api.example.com

# Disable SSL verification for self-signed certificates
httpcheck --no-verify-ssl https://self-signed.example.com

# Custom timeout and retry configuration
httpcheck --timeout 30 --retries 5 --retry-delay 2.0 slow.example.com

# Combine with other options
httpcheck -H "API-Key: secret" --output json --no-verify-ssl https://api.example.com
```

### configuration file

httpcheck reads defaults from `~/.httpcheck.toml` (user-wide) and/or `./.httpcheck.toml` (project-local). Project-level settings override user-level settings; any CLI flag always takes final precedence.

```toml
# ~/.httpcheck.toml

[defaults]
timeout = 10.0
retries = 3
follow_redirects = "always"   # always | never | http-only | https-only
output_format = "table"       # table | json | csv
verify_ssl = true
workers = 20
retry_delay = 0.5
max_redirects = 10

[headers]
User-Agent = "mybot/1.0"
Accept = "application/json"

[notifications]
enabled = true
on_failure = true
sound = "Ping"
```

Unknown keys are silently ignored with a `UserWarning`. Invalid TOML falls back to no config (also a `UserWarning`).

### file input examples

Create a domains file with comments and use it with httpcheck:

```txt
# domains.txt - Example domains file
google.com  # Search engine
facebook.com // Social media
twitter.com
// Commented out: example.com
```

```shell
# Basic file usage
httpcheck @domains.txt

# Show file parsing statistics
httpcheck --file-summary @domains.txt

# Only recognize # style comments
httpcheck --comment-style hash @domains.txt

# Only recognize // style comments
httpcheck --comment-style slash @domains.txt
```

## onSite - menu bar app (macOS)

For continuous monitoring with a native macOS experience, use onSite - the menu bar app:

### installation

```shell
# Install additional dependencies for menu bar app
pip install rumps pyobjc-framework-Cocoa py2app

# Run the menu bar app directly
python3 httpcheck_menubar.py
```

### building a standalone app

Create a native macOS app bundle:

```shell
# Build the app (creates dist/onSite.app)
python setup.py py2app

# Copy to Applications folder
cp -r "dist/onSite.app" /Applications/

# Run from Applications or Spotlight
open "/Applications/onSite.app"
```

### menu bar app features

* **Live Status Indicator**: Menu bar shows ⚡ (white lightning - all
  good/checking), 🔴⚡ (red + lightning - failures)
* **Badge Count**: Shows number of failed sites next to the icon
* **Rich Notifications**: Native macOS notifications with:
  * Site down alerts with status codes
  * Recovery notifications when sites come back online
  * Click actions to open failing sites in browser
  * Custom sounds for different alert types
* **Background Monitoring**: Configurable check intervals (default 15 minutes, minimum 60 seconds)
* **Site Management**: Add, remove, and edit monitored sites through menu
* **Settings**: Configure check intervals, clear failed sites, view logs
* **Configuration Storage**: Sites and settings saved to `~/.httpcheck/`

### menu bar app usage

1. **Add Sites**: Click menu bar icon → "Add Site..." → Enter URL (automatically checks status)
2. **Manual Check**: Click "Check Now" for immediate status check
3. **Auto-Check**: Toggle automatic background checking on/off (default: 15 minutes)
4. **View Status**: Click on menu to see all sites with status indicators
5. **Settings**: Adjust check interval, clear failed sites, view logs

### keyboard shortcuts and tips

* Menu bar shows white lightning ⚡ for normal status, red lightning 🔴⚡ for failures
* Failed site count appears as badge number
* Notifications include action buttons (Check Now, View, Dismiss)
* Clicking notifications opens the failing site in your default browser
* Logs are stored in `~/Library/Logs/onSite/onsite.log` following macOS best practices
* Log rotation: 5MB max size, 5 backup files maintained automatically
* Logs viewable through Console.app or "View logs" menu item
* Configuration files stored in `~/.httpcheck/` for easy backup

## history

checking one or more websites by constantly opening a browser seems a waste of
time. Providing a cli tool that can be used in a bash shell, pipe or for
scheduling tasks in cron, notifying the user, seems the way to go. The python
requests library seems perfect for the job.

## todo

[TODO.md](https://github.com/docdyhr/httpcheck/blob/master/TODO.md)

## license

[MIT](https://github.com/docdyhr/httpcheck/blob/master/LICENSE)
