Metadata-Version: 2.4
Name: proxyspy
Version: 0.2.0
Summary: A debugging proxy that can log or intercept HTTPS requests
Project-URL: Homepage, https://github.com/anaconda/proxyspy
Project-URL: Repository, https://github.com/anaconda/proxyspy
Author-email: "Anaconda, Inc." <conda@anaconda.com>
License: BSD 3-Clause License
        
        Copyright (c) 2024, Anaconda, Inc.
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived from
           this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License-File: LICENSE.txt
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: Proxy Servers
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.10
Requires-Dist: cryptography
Provides-Extra: test
Requires-Dist: psutil; extra == 'test'
Requires-Dist: pytest; extra == 'test'
Requires-Dist: requests; extra == 'test'
Description-Content-Type: text/markdown

# HTTPS Debug Proxy

[![Build and Test](https://github.com/anaconda/proxyspy/actions/workflows/main.yaml/badge.svg)](https://github.com/anaconda/proxyspy/actions/workflows/main.yaml)
[![Latest Release](https://img.shields.io/github/v/release/anaconda/proxyspy?include_prereleases)](https://github.com/anaconda/proxyspy/releases/latest)
[![PyPI version](https://img.shields.io/pypi/v/proxyspy.svg)](https://pypi.org/project/proxyspy/)
[![Conda Version](https://img.shields.io/conda/v/mcg/proxyspy)](https://anaconda.org/mcg/proxyspy)

A debugging proxy that can log or intercept HTTPS requests. This tool can be used to:

- Monitor HTTPS traffic from applications
- Debug SSL/TLS issues
- Test applications against specific HTTP responses
- Simulate network delays

## Features

- Full HTTPS request/response logging
- Custom response injection
- Automatic certificate generation
- Connection delays for testing
- Concurrent connection support
- Binary data handling
- Automatic port selection

## Installation

ProxySpy is available on both Conda-Forge and PyPi:

```bash
conda install conda-forge::proxyspy  # or...
pip install proxyspy
```

Installing it this way ensures that its `cryptography` dependency is also
available, and add the `proxyspy` command to your `PATH` when this Python
environment is activated.

This utility has been deliberately designed to function as a single-file
Python script that depends only upon the standard Python 3 library and the
`cryptography` package. For that reason, you can also vendor the script
directly into your work environment if you wish.

1. Copy `proxyspy.py` directly into your project.
   [Here](https://raw.githubusercontent.com/anaconda/proxyspy/refs/heads/main/proxyspy.py)
   is a direct download link to the latest version of the script.
2. Ensure the `cryptography` package is available in your Python environment:
   ```
   conda install cryptography  # or...
   pip install cryptography
   ```

## Development Requirements

To develop or test the proxy itself, additional packages are required:

```bash
conda install --file requirements.txt
```

This will install:
- cryptography (required for proxy operation)
- requests (for tests)
- pytest (for running tests)

## Usage

```bash
proxyspy [options] -- command [args...]            # Installed as a package
python proxyspy.py [options] -- command [args...]  # Direct access
```

The tool starts a proxy server and then runs the specified command with appropriate proxy environment variables set.

### Options

- `--logfile FILE, -l FILE`: Write logs to FILE (default: stdout)
- `--port PORT, -p PORT`: Listen on PORT (default: auto-select)
- `--keep-certs`: Keep certificates in current directory
- `--delay TIME`: Add TIME seconds delay to each connection
- `--return-code N, -r N`: Return status code N for all requests
- `--return-header H`: Add header H to responses (can repeat)
- `--return-data DATA`: Return DATA as response body
- `--intercept-host HOST`: Only intercept requests to HOST (can repeat)
- `--prepare-host HOST`: Pre-generate the certificate for HOST to avoid first-connection delay (can repeat)
- `--standalone`: Run the forward proxy standalone: instead of running a command, print the proxy/CA environment variables to set in another terminal (and write them to `<cert-dir>/env` for sourcing), then block until interrupted (see below)
- `--reverse`: Run as a standalone reverse/transparent proxy instead of running a command (see below)
- `--manage-hosts`: Automatically add/remove the `/etc/hosts` redirects for declared hosts for the run's lifetime (reverse mode only; POSIX only; see below)
- `--restore-hosts`: Remove any proxyspy-managed block from `/etc/hosts` and exit (standalone; POSIX only)
- `--cert-dir DIR`: Directory for the persistent CA and host certificates (reverse and standalone mode default: `~/.proxyspy`)
- `--map HOST=IP`: Pin the real upstream IP for HOST, bypassing DNS (reverse mode; can repeat)
- `--upstream-port PORT`: Port to dial on the real upstream servers in reverse mode (default: 443)

### Examples

Log all HTTPS requests to test.log:

```bash
proxyspy --logfile test.log -- curl https://httpbingo.org/ip
```

Return 404 for all requests with a half-second delay:

```bash
proxyspy --return-code 404 --delay 0.5 -- python my_script.py
```

Return custom response with headers and body:

```bash
proxyspy --return-code 200 \
         --return-header "Content-Type: application/json" \
         --return-data '{"status": "ok"}' \
         -- ./my_script.py
```

Use specific port instead of auto-selection:

```bash
proxyspy.py --port 8888 -- curl https://httpbingo.org/ip
```

Intercept only requests to specific domains:

```bash
proxyspy --return-code 404 \
         --intercept-host "conda.anaconda.org" \
         --intercept-host "repo.anaconda.com" \
         -- python conda_script.py
```

Run the proxy standalone and drive it from a second terminal:

```bash
proxyspy --standalone -l spy.log
```

## How It Works

The proxy operates in two modes and can optionally add delays to any connection:

### Forwarding Mode (default)
- Creates a CA certificate and per-host certificates
- Establishes SSL tunnels to requested hosts
- Logs all traffic passing through
- Runs a command as a subprocess, or blocks standalone with `--standalone` (see [Standalone Forward Mode](#standalone-forward-mode))

### Interception Mode
- Activated by specifying any of: --return-code, --return-data, --return-header
- Returns custom responses instead of connecting to servers
- Useful for testing application behavior

### Connection Delays
- Optional delay can be added to any connection in either mode
- Delay occurs after connection but before SSL handshake
- Useful for testing timeout and connection handling

### Port Selection
- By default, the proxy automatically selects an available port
- This prevents socket reuse issues and allows running multiple instances
- A specific port can be chosen with the --port option

## Standalone Forward Mode

`--standalone` runs the ordinary forward proxy, but instead of launching a command as a subprocess it prints the environment variables a client needs, writes them to a source-able file, and blocks until you press Ctrl-C. Use it when the client you want to watch cannot run as a proxyspy subprocess — for example a long-lived service, an IDE, or a shell session you drive by hand.

On startup it prints a copy-pasteable banner to stdout (always, even with `-l`/`--logfile`), and writes the same `export` lines to `<cert-dir>/env` (default `~/.proxyspy/env`):

```
========================================================================
ProxySpy standalone (forward proxy) listening on http://localhost:54321

Paste into another terminal to route HTTPS through ProxySpy:

    export HTTP_PROXY=http://localhost:54321
    export HTTPS_PROXY=http://localhost:54321
    ...
    export SSL_CERT_FILE=/Users/you/.proxyspy/cert.pem
    export CONDA_SSL_VERIFY=/Users/you/.proxyspy/cert.pem

...or just:  source /Users/you/.proxyspy/env

Press Ctrl-C to stop.
========================================================================
```

Workflow:

* In terminal 1, start the proxy: `proxyspy --standalone -l spy.log`.
* In terminal 2, either paste the printed `export` lines or run `source ~/.proxyspy/env`, then run your client. Its HTTPS traffic is now proxied and logged to `spy.log`.
* Press Ctrl-C in terminal 1 to stop. The env file is removed on exit so it never points at a released port.

Notes:

* No root is needed — standalone auto-selects an unprivileged port (unlike reverse mode, which defaults to privileged port 443).
* Clients only trust the CA via the `*_CA_BUNDLE` / `SSL_CERT_FILE` / `CONDA_SSL_VERIFY` variables, so this exercises the *proxied* code path. Tools that ignore those variables (or bypass proxy env vars entirely) won't be intercepted; for those, use reverse mode.
* `--cert-dir` overrides where the persistent CA and the `env` file live.

## Reverse / Transparent Mode

`--reverse` runs proxyspy as a standalone transparent MITM that emulates a **TLS-intercepting corporate firewall**: it listens on `127.0.0.1` (default port 443) and learns the target hostname from the TLS SNI field. You redirect the hostnames you want to watch to `127.0.0.1` in `/etc/hosts`, and the client connects to proxyspy normally — with no proxy configured and no idea a proxy exists.

This matters because forward mode works by setting `HTTPS_PROXY`, so it can only exercise a client's *proxied* code path. Some clients behave differently with and without a proxy configured, so the proxied path is not a faithful stand-in for a user behind a transparent intercepting firewall. For example, in [conda/conda#16253](https://github.com/conda/conda/pull/16253) `requests` applied a `truststore` SSL context on its direct-connection path but dropped it on the separate proxy path, so verification against a MITM firewall's certificate silently failed *only* under a proxy. Reverse mode lets proxyspy intercept the TLS handshake while the client still takes its direct, no-proxy path, making that class of difference reproducible. The two modes are complementary test surfaces.

Because `/etc/hosts` redirects the target hostnames to proxyspy itself, proxyspy cannot use the OS resolver to reach the real upstream once those entries are in place. It therefore resolves and caches each declared host's real IP **at startup** — so you must start proxyspy *before* editing `/etc/hosts`. If a declared host already resolves to a loopback address, proxyspy refuses to start and tells you to remove it from `/etc/hosts` or pin it with `--map HOST=IP`.

Workflow:

* Start proxyspy as root (port 443 is privileged), declaring the hosts to watch with `--prepare-host` (forward+log) or `--intercept-host` (return canned responses):
  ```bash
  sudo proxyspy --reverse --prepare-host repo.anaconda.com -l spy.log
  ```
* Trust the CA certificate it prints (default `~/.proxyspy/cert.pem`) in your client/system trust store. The CA persists across runs, so you only trust it once.
* Add the redirects to `/etc/hosts`:
  ```
  127.0.0.1 repo.anaconda.com
  ```
* Run your client normally and watch `spy.log`.
* Press Ctrl-C to stop proxyspy, then remove the `/etc/hosts` entries.

Notes:

* `--cert-dir` overrides where the persistent CA and host certificates live. Under `sudo`, the default `~/.proxyspy` resolves to the invoking user's home (via `SUDO_USER`), not root's.
* `--map HOST=IP` pins an upstream IP, bypassing startup resolution. Use it when a host is already in `/etc/hosts`, or to target a specific backend.
* A host listed only in `--intercept-host` (with no forwarding) never connects upstream, so it is not resolved.

### Automatic `/etc/hosts` management (opt-in)

Add `--manage-hosts` to have proxyspy add and remove the `/etc/hosts` redirects itself, so a session is self-contained: start it, use it, Ctrl-C, and the file is back to how it was. This replaces steps 3 and 5 of the manual workflow above; everything else (running as root, trusting the CA) is unchanged:

```bash
sudo proxyspy --reverse --manage-hosts --prepare-host repo.anaconda.com -l spy.log
```

proxyspy writes a fenced block to `/etc/hosts` containing only the redirects for the declared hosts, keeping a one-time backup at `/etc/hosts.proxyspy.bak`, and removes the block on a clean exit (Ctrl-C or SIGTERM). If a previous run is killed uncatchably (`SIGKILL`, power loss) and leaves the block behind, the next `--manage-hosts` start self-heals by removing it before resolving upstreams. To force-clean a stale block without starting proxyspy, run:

```bash
sudo proxyspy --restore-hosts
```

`--manage-hosts` and `--restore-hosts` are POSIX-only (they edit `/etc/hosts` directly) and require root, since editing `/etc/hosts` needs the same privileges as binding port 443.

## Development

Run tests:
```bash
pytest -v
```

The test suite covers:
- Basic forwarding
- Response interception
- Binary data handling
- Connection delays
- Error conditions
- Sequential proxy starts

## Contributing

When submitting pull requests, please:
- Add tests for new features
- Ensure all tests pass
- Follow existing code style
- Do not add third-party dependencies beyond the required `cryptography` package
  - The proxy tester is designed to be a single, self-contained file
  - Additional dependencies make it harder for users to incorporate into their projects

## About This Project

This project was primarily developed through a series of conversations with Claude 3.5 and 3.7 Sonnet, an AI assistant from Anthropic (https://claude.ai). The majority of the code, including the test suite and GitHub Actions configuration, was written by Claude in response to requirements and refinements from human developers. This collaborative approach demonstrates how AI assistance can help create well-tested, maintainable code while adhering to strict dependency and design constraints.
