Metadata-Version: 2.4
Name: bytehaul
Version: 0.2.5
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: System :: Networking
Summary: Python bindings for the bytehaul Rust download library
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://github.com/triwinds/bytehaul/tree/main/bindings/python
Project-URL: Homepage, https://github.com/triwinds/bytehaul
Project-URL: Repository, https://github.com/triwinds/bytehaul

# bytehaul

Python bindings for the [bytehaul](https://github.com/triwinds/bytehaul) Rust download library. This guide targets the published **0.2.5** release.

[中文使用文档](../../docs/python.zh-CN.md)

## Requirements

- Python 3.9+
- Rust toolchain and `uv` only when building from source; neither is required to install an available wheel.

Each source-build command block below assumes you start from the repository root.

## Installation

### From PyPI

```bash
pip install "bytehaul==0.2.5"
```

See the [0.2.5 release notes](https://github.com/triwinds/bytehaul/releases/tag/v0.2.5).

### From source (development)

```bash
uv sync --project bindings/python
cd bindings/python
uv run --project . maturin develop -m Cargo.toml
```

### Build wheel

```bash
cd bindings/python
uv run --project . maturin build --release -m Cargo.toml
```

## Usage

### Simple download

```python
import bytehaul

bytehaul.download("https://example.com/file.bin", output_path="output.bin")

# Let bytehaul decide the filename and place it in downloads/
bytehaul.download("https://example.com/file.bin", output_dir="downloads")
```

### With options

```python
bytehaul.download(
    "https://example.com/file.bin",
    output_path="output.bin",
    max_connections=8,
    max_download_speed=1_000_000,  # 1 MB/s
    headers={"Authorization": "Bearer token"},
)
```

### Network settings

```python
bytehaul.download(
    "https://example.com/file.bin",
    output_path="output.bin",
    proxy="http://127.0.0.1:7890",
    dns_servers=["1.1.1.1", "8.8.8.8:53"],
    doh_servers=["https://dns.google/dns-query"],
    enable_ipv6=False,
)
```

`doh_servers` expects HTTPS URLs. If you pass a hostname such as `dns.google`, bytehaul will use the system resolver once during client construction to bootstrap the DoH endpoint addresses.

For the libcurl backend, `ca_info` may point to an additional PEM trust bundle,
`ca_path` to a directory of hashed CA certificates, and `client_cert` plus
`client_key` configure mutual TLS. Peer and hostname verification remain
enabled; these options add credentials rather than disabling verification.

### Logging

```python
# Enable debug logging on the convenience function
bytehaul.download(
    "https://example.com/file.bin",
    output_path="output.bin",
    log_level="debug",
)

# Or on the Downloader object
from bytehaul import Downloader

downloader = Downloader(log_level="info")
```

Valid levels: `"off"` (default), `"error"`, `"warn"`, `"info"`, `"debug"`, `"trace"`.

### Object API with progress and cancellation

```python
from bytehaul import Downloader

downloader = Downloader(
    connect_timeout=15.0,
    dns_servers=["1.1.1.1"],
    doh_servers=["https://dns.google/dns-query"],
    enable_ipv6=False,
)
task = downloader.download(
    "https://example.com/large.bin",
    output_dir="downloads",
    proxy="http://127.0.0.1:7890",
)

# Poll progress
snap = task.progress()
print(
    f"State: {snap.state}, Downloaded: {snap.downloaded}, "
    f"Speed: {snap.speed:.0f} B/s, ETA: {snap.eta_secs}"
)

# Pause or cancel if needed
# task.pause()
# task.cancel()

# Wait for completion
task.wait()
```

### Error handling

```python
from bytehaul import download, DownloadFailedError, CancelledError, PausedError, ConfigError

try:
    download("https://example.com/file.bin", output_path="output.bin")
except ConfigError as e:
    print(f"Invalid parameter: {e}")
except PausedError:
    print("Download was paused")
except CancelledError:
    print("Download was cancelled")
except DownloadFailedError as e:
    print(f"Download failed: {e}")
```

Response-body timeouts, connection resets and early EOF for a known-size body are retried within `max_retries`, which counts additional attempts after the first (`0` disables retries). With a known total and matching object validators, continuation starts at the durable prefix confirmed by the writer; ignored Range requests or changed object metadata trigger a safe restart from zero. Disk write and synchronization errors are not retried as network failures.

## API Reference

### `download(url, output_path=None, output_dir=None, **options)`

Blocking convenience function. Downloads a file and returns when complete.

- `output_path`: explicit filename or relative output path
- `output_dir`: destination directory for explicit or auto-detected filenames
- If `output_path` is omitted, bytehaul chooses `Content-Disposition` → URL path → `download`
- Absolute `output_path` values are still accepted when `output_dir` is omitted

### `Downloader(connect_timeout=None, proxy=None, http_proxy=None, https_proxy=None, dns_servers=None, doh_servers=None, enable_ipv6=None, log_level=None, ca_info=None, ca_path=None, client_cert=None, client_key=None)`

Reusable downloader instance.

- `downloader.download(url, output_path=None, output_dir=None, **options) -> DownloadTask`

Proxy settings passed to `Downloader(...)` act as defaults. You can override them per task by passing `proxy`, `http_proxy`, or `https_proxy` directly to `downloader.download(...)`.

### `DownloadTask`

Handle to a running download.

- `task.progress() -> ProgressSnapshot` — current download progress
- `task.pause()` — pause the download and persist resume metadata when available
- `task.cancel()` — cancel the download
- `task.wait()` — block until download completes (releases GIL)

`wait()` consumes the task handle. It cannot be called twice, and `progress()` is unavailable after it returns or raises.

### `ProgressSnapshot`

Frozen snapshot of download progress.

| Attribute      | Type           | Description                     |
|----------------|----------------|---------------------------------|
| `total_size`   | `int \| None`  | Total file size (if known)      |
| `downloaded`   | `int`          | UI-oriented received bytes, not a durable resume offset |
| `state`        | `str`          | `"pending"`, `"downloading"`, `"completed"`, `"failed"`, `"cancelled"`, `"paused"` |
| `speed`        | `float`        | Recent-window speed in bytes/second |
| `eta_secs`     | `float \| None`| Estimated remaining seconds     |
| `elapsed_secs` | `float \| None`| Elapsed time in seconds         |

`downloaded` may decrease during retries or already equal `total_size` when final synchronization fails. Final write or synchronization failures produce `failed` state, and the control file retains only confirmed durable progress. Use the result or exception from `task.wait()` to determine success; do not use the displayed byte count as a resume offset.

`speed` and `eta_secs` are computed from the same recent throughput window. `speed` is not a whole-download lifetime average, and `eta_secs` stays `None` until bytehaul has enough recent samples or a known total size.

### Download options

| Parameter           | Type             | Default       |
|---------------------|------------------|---------------|
| `output_path`       | `str \| Path \| None` | `None` |
| `output_dir`        | `str \| Path \| None` | `None` |
| `headers`           | `dict[str, str]` | `{}`          |
| `max_connections`   | `int`            | `4`           |
| `connect_timeout`   | `float` (secs)   | `30.0`        |
| `read_timeout`      | `float` (secs)   | `60.0`        |
| `request_headers_timeout` | `float` (secs) | `None` (inherit `read_timeout`) |
| `ca_info`          | `str \| Path \| None` | `None` (additional PEM trust bundle) |
| `ca_path`          | `str \| Path \| None` | `None` (hashed CA directory) |
| `client_cert`      | `str \| Path \| None` | `None` (mTLS client certificate) |
| `client_key`       | `str \| Path \| None` | `None` (mTLS private key) |
| `memory_budget`     | `int`            | `67108864`    |
| `file_allocation`   | `"none" \| "prealloc"` | `"prealloc"` |
| `resume`            | `bool`           | `True`        |
| `piece_size`        | `int`            | `1048576`     |
| `min_split_size`    | `int`            | `10485760`    |
| `max_retries`       | `int`            | `5`           |
| `retry_base_delay`  | `float` (secs)   | `1.0`         |
| `retry_max_delay`   | `float` (secs)   | `30.0`        |
| `max_retry_elapsed` | `float \| None` (secs) | `None` |
| `control_save_interval` | `float` (secs) | `5.0` |
| `autosave_sync_every` | `int` | `2` |
| `max_download_speed`| `int`            | `0` (unlimited)|
| `checksum_sha256`   | `str \| None`    | `None`        |
| `log_level`         | `str \| None`    | `None` (`"off"`) |
| `request_batch_size` | `int` | `4194304` |
| `range_scheduling_mode` | `"fixed" \| "dynamic"` | `"dynamic"` |
| `dynamic_min_split_size` | `int` | `1048576` |
| `dynamic_max_request_size` | `int` | `67108864` |

`max_retries` counts additional retries after the initial request/transfer attempt; `0` disables retries. Single-connection body failures resume from the writer's flushed contiguous prefix, while Range or object-metadata mismatches reset the file before restarting.

`control_save_interval` checks whether a checkpoint is due; `autosave_sync_every` batches those checks when unsaved progress exists. Set `log_level` on the convenience `download(...)` function or the `Downloader(...)` constructor.

Valid `log_level` values: `"off"`, `"error"`, `"warn"`, `"info"`, `"debug"`, `"trace"` (case-insensitive).

### Contiguous requests

Version 0.2.3 adds `request_batch_size` to both download
APIs, after existing positional parameters. `None` selects the Rust default of
4 MiB; `0` keeps one lease per request. Other positive values, such as
`request_batch_size=8 * 1024 * 1024`, group adjacent pieces into bounded Range
requests. Completion/checkpoint granularity remains `piece_size`, with at most
64 leases per batch. Values below a piece do not split it, and grouping stops at
completed/active/partially processed pieces.

`range_scheduling_mode="dynamic"` is the default and chooses ranges from
currently free contiguous pieces and actual request slots. Explicit
`range_scheduling_mode="fixed"` is the compatibility mode and applies
`request_batch_size`. Dynamic mode
uses `dynamic_min_split_size` (default 1 MiB, rounded up to a piece boundary)
and `dynamic_max_request_size` (default 64 MiB) instead. The byte and 64-lease
limits are hard; when possible the scheduler backs up to a legal minimum-split
boundary to avoid a short tail and records a conflict otherwise. The latter still
allows one complete piece when configured below `piece_size`; each request is
also limited to 64 piece leases. `request_batch_size` is ignored in dynamic
mode and `0` does not mean automatic scheduling.

```python
bytehaul.download(
    "https://example.com/large.bin",
    output_path="large.bin",
    max_connections=8,
    range_scheduling_mode="dynamic",
    dynamic_min_split_size=2 * 1024 * 1024,
    dynamic_max_request_size=64 * 1024 * 1024,
)
```

Strong-ETag multi-connection transfers can also retain writer-confirmed prefixes
on interrupted-body retries or adaptive reassignment. Incomplete pieces remain
incomplete across process restarts. These engine behaviors apply to both Python
entry points. Python uses the Rust default HTTP idle pool (4 connections per
host, 30 seconds); explicit pool tuning remains available through the Rust API.

### Slow-transfer recovery

Available starting with version 0.2.2. These options apply to both `download(...)` and `Downloader.download(...)`; `None` selects the Rust engine default.

| Parameter | Default | Meaning |
| --- | --- | --- |
| `slow_transfer_mode` | `"adaptive"` | `"disabled"`, `"adaptive"`, or `"adaptive_with_hedging"` (case-insensitive) |
| `low_speed_limit` | `None` | Optional positive absolute floor, bytes/second |
| `low_speed_duration` | `15.0` | Sustained low-speed time, seconds |
| `slow_start_grace` | `5.0` | Startup grace, seconds |
| `slow_sample_window` | `5.0` | Speed observation window, seconds |

Slow-transfer policy durations must be finite, positive and at most 86,400 seconds. Adaptive recovery is on by default for multi-connection Range downloads. Hedging is opt-in, needs a strong ETag and a spare connection slot, and stages at most one small spare response before choosing a writer. It does not duplicate progress or exceed `max_connections`. All network payload shares the configured rate limit. Performance recovery and hedging share an extra-body budget of `min(total_size / 100, 16 MiB)`. Protected cancel/resume reserves consumed data that may be discarded, excluding the necessary suffix; hedging still reserves its full range. Unread transport buffers and TCP/TLS overhead are outside this accounting. Midbatch handoff preserves shared retry budgets and Retry-After for released pieces. Normal error retries use the existing retry policy.

```python
from bytehaul import Downloader

task = Downloader(log_level="debug").download(
    "https://example.com/file.bin",
    "file.bin",
    slow_transfer_mode="adaptive_with_hedging",
)
task.wait()
```

Use `slow_transfer_mode="disabled"` to disable performance-triggered cancellation and hedging. Single-connection and non-Range fallback behavior is unchanged. Recovery excludes intentional rate limiting and local backpressure; it cannot remove a shared origin bandwidth limit.

When `max_download_speed` is nonzero, automatic slow-request recovery and hedging are suppressed to avoid treating intentional rate limiting as a network fault. Ordinary timeouts and error retries still apply.

### Network options

Use these on `Downloader(...)` to set defaults, or pass `proxy`, `http_proxy`, and `https_proxy` directly to `downloader.download(...)` or the blocking `download(...)` helper.

| Parameter      | Type                | Default |
|----------------|---------------------|---------|
| `proxy`        | `str \| None`      | `None`  |
| `http_proxy`   | `str \| None`      | `None`  |
| `https_proxy`  | `str \| None`      | `None`  |
| `dns_servers`  | `list[str] \| None`| `None`  |
| `doh_servers`  | `list[str] \| None`| `None`  |
| `enable_ipv6`  | `bool \| None`     | `True`  |
| `ca_info`      | `str \| Path \| None` | `None` |
| `ca_path`      | `str \| Path \| None` | `None` |
| `client_cert`  | `str \| Path \| None` | `None` |
| `client_key`   | `str \| Path \| None` | `None` |

## Running tests

```bash
uv sync --project bindings/python
cd bindings/python
uv run --project . maturin develop -m Cargo.toml
uv run --no-sync --project . pytest tests/ -v
```

After `maturin develop`, use `--no-sync` for tests so uv does not replace the freshly built development extension during another environment sync.

## Building wheels for release

Single platform:

```bash
cd bindings/python
uv run --project . maturin build --release -m Cargo.toml
```

Cross-platform (via CI):

```bash
# Linux x86_64 + aarch64, macOS x86_64 + arm64, Windows x86_64
# Use maturin's GitHub Actions: https://github.com/PyO3/maturin-action
```

The project uses `abi3-py39`, so a single wheel per platform covers all Python 3.9+ versions.

## License

MIT. See the repository LICENSE file.

### Request response-headers deadline

Rust `DownloadSpec::request_headers_timeout(Duration)` and the appended Python
`request_headers_timeout` argument (seconds, default `None`) bound each request
from invocation through response headers, including pool waiting and DNS/TCP/TLS
connection setup. This is not pure server TTFB. Omission preserves the existing
header deadline inherited from `read_timeout`; body reads still use `read_timeout`.
The value must be positive and representable as a monotonic-clock deadline.
The connector's timeout can expire earlier.

Each retry and redirect hop gets a fresh deadline, including probes, GET fallback,
resume, and ordinary Range requests. This is not a total redirect-chain or download
deadline: existing retry counts, `max_retry_elapsed` check boundaries, and 429/503
`Retry-After` behavior remain unchanged. There is no automatic deadline shortening
or response-headers hedging.

Existing probe transport failures (including timeouts) may enter GET fallback;
these retain their separate retry scopes. `max_retry_elapsed` is not a hard
whole-download deadline or a combined deadline for both phases.

