Metadata-Version: 2.4
Name: remote-download
Version: 0.1.0
Summary: Stream remote content OUT of any origin (S3/MinIO, Azure Blob, GCS, SFTP, FTP, HTTP) to any sink through one tiny, framework-agnostic API. The read-side twin of remote-upload.
Project-URL: Homepage, https://github.com/calcifux/remote-download-python
Project-URL: Repository, https://github.com/calcifux/remote-download-python
Project-URL: Changelog, https://github.com/calcifux/remote-download-python/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/calcifux/remote-download-python/issues
Author: Carlos Guillermo Reyes Ramiro (@Calcifux)
License-Expression: MIT
License-File: LICENSE
Keywords: azure-blob,download,ftp,gcs,http,minio,s3,sftp,storage,streaming
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: System :: Archiving
Classifier: Typing :: Typed
Requires-Python: >=3.14
Provides-Extra: all
Requires-Dist: azure-storage-blob>=12.19; extra == 'all'
Requires-Dist: boto3>=1.34; extra == 'all'
Requires-Dist: google-cloud-storage>=2.14; extra == 'all'
Requires-Dist: httpx>=0.27; extra == 'all'
Requires-Dist: paramiko>=3.4; extra == 'all'
Provides-Extra: azure
Requires-Dist: azure-storage-blob>=12.19; extra == 'azure'
Provides-Extra: gcs
Requires-Dist: google-cloud-storage>=2.14; extra == 'gcs'
Provides-Extra: httpx
Requires-Dist: httpx>=0.27; extra == 'httpx'
Provides-Extra: s3
Requires-Dist: boto3>=1.34; extra == 's3'
Provides-Extra: sftp
Requires-Dist: paramiko>=3.4; extra == 'sftp'
Description-Content-Type: text/markdown

# remote-download

[![CI](https://github.com/calcifux/remote-download-python/actions/workflows/ci.yml/badge.svg)](https://github.com/calcifux/remote-download-python/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/remote-download.svg)](https://pypi.org/project/remote-download/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.14+](https://img.shields.io/badge/Python-3.14%2B-blue.svg)](https://www.python.org/)
[![Typed](https://img.shields.io/badge/typing-strict-brightgreen.svg)](https://peps.python.org/pep-0561/)

> **Stream remote content OUT of any origin** (S3 / MinIO, Azure Blob, GCS, SFTP, FTP, authenticated HTTP) to any sink through one tiny, framework-agnostic API — the **read-side twin** of `remote-upload`.

```python
from remote_download import RemoteDownload

result = (
    RemoteDownload.from_("https://cdn.example.com/report.pdf")
    .checksum("sha256")
    .write_to(out)
)
```

`remote-upload` pushes bytes *into* a remote destination. `remote-download` is the other half: it pulls bytes *out* of a remote origin and forwards them to any sink. Same shape, mirrored — works in any web framework (Django, FastAPI, Flask), task queues, AWS Lambda, or plain CLI scripts: anywhere you can write bytes to a stream.

| | remote-download | remote-upload |
|---|---|---|
| Port | `DownloadOrigin.open() -> RemoteContent` | `UploadTarget.upload(UploadContent) -> UploadResult` |
| Facade | `RemoteDownload.from_(src).write_to(out)` | `RemoteUpload.to(target).body(stream, length).upload()` |
| Direction | remote -> your backend -> client | client -> your backend -> remote |

## Install

The core (`HttpOrigin` and `FtpOrigin`) is **pure standard library** — no third-party dependencies. Cloud and SSH backends each pull in one SDK, gated behind an extra so you install only what you use:

```bash
pip install remote-download
```

| Extra | Install | Backend | Brings in |
|---|---|---|---|
| *(none)* | `pip install remote-download` | `HttpOrigin`, `FtpOrigin` | stdlib only — always available |
| `s3` | `pip install "remote-download[s3]"` | `S3Origin` (S3 / MinIO / Ceph / LocalStack) | `boto3` |
| `azure` | `pip install "remote-download[azure]"` | `AzureBlobOrigin` | `azure-storage-blob` |
| `gcs` | `pip install "remote-download[gcs]"` | `GcsOrigin` | `google-cloud-storage` |
| `sftp` | `pip install "remote-download[sftp]"` | `SftpOrigin` | `paramiko` |
| `httpx` | `pip install "remote-download[httpx]"` | `HttpxOrigin` (retries / auth / proxy) | `httpx` |
| `all` | `pip install "remote-download[all]"` | everything above | all of the above |

Origins are importable straight from the package root. The extra-gated ones are loaded lazily, so importing one without its SDK installed raises a clear `ImportError` telling you exactly which extra to install:

```python
from remote_download import RemoteDownload, S3Origin, AzureBlobOrigin  # lazily resolved
```

Requires **Python 3.14+**.

## Quick start

Every download follows the same fluent shape: pick an origin (or a URL), optionally decorate the transfer, then consume the content.

### Plain HTTP GET from a URL

A bare string is treated as an absolute HTTP/HTTPS URL and wrapped in a default `HttpOrigin` (GET, no auth, redirects followed):

```python
from remote_download import RemoteDownload

with open("/tmp/report.pdf", "wb") as out:
    result = RemoteDownload.from_("https://cdn.example.com/report.pdf").write_to(out)

print(result.bytes_transferred, "bytes", result.content_type)
```

### The ways to consume the content

```python
# 1. write_to(sink) — copy the full payload into any binary sink, then release
#    every resource. The sink is NOT closed (you own it). The common case.
RemoteDownload.from_(origin).write_to(response)        # e.g. an HTTP response stream

# 2. write_to_file(path) — copy straight to a file on disk.
RemoteDownload.from_(origin).write_to_file("/tmp/photo.jpg")

# 3. fetch() — get the RemoteContent (metadata + live stream); you close it.
with RemoteDownload.from_(origin).fetch() as content:
    print(content.content_type, content.content_length, content.filename)
    head = content.stream.read(1024)

# 4. as_stream() — get only a live binary stream; closing it releases the
#    connection and any provider resources (SDK client, SSH session).
with RemoteDownload.from_(origin).as_stream() as stream:
    shutil.copyfileobj(stream, out)
```

> The origin is opened on demand per consumption. Build a fresh request per download — instances are not reusable.

### Everything together

```python
def on_progress(read: int, total: int | None) -> None:
    pct = (read * 100 // total) if total else -1
    print(f"downloaded {read} / {total} bytes ({pct}%)")

result = (
    RemoteDownload.from_(origin)
    .chunk_size(64 * 1024)         # copy buffer size (default 8 KiB)
    .checksum("sha256")            # also accepts Java-style "SHA-256"
    .on_progress(on_progress)
    .write_to(out)
)

print(result.filename, result.content_type)
print(result.checksum_hex)
print(f"{result.bytes_per_second / 1_048_576:.1f} MiB/s")
```

## Backends

Every origin is constructed with **keyword arguments** and then handed to `RemoteDownload.from_(...)`. The keyword names below match each origin's constructor exactly.

### HttpOrigin — authenticated HTTP GET (stdlib, always available)

```python
from remote_download import RemoteDownload, HttpOrigin

origin = HttpOrigin(
    "https://api.example.com/files/123",
    headers={"X-Tenant": "acme"},
    bearer="<token>",                 # or basic_auth=("user", "pass")
    connect_timeout=10.0,
    request_timeout=300.0,
)
RemoteDownload.from_(origin).write_to_file("/tmp/file.bin")
```

Follows redirects. For automatic retries, NTLM, or authenticated proxies use `HttpxOrigin`.

### FtpOrigin — FTP / FTPS (stdlib, always available)

```python
from remote_download import RemoteDownload, FtpOrigin

origin = FtpOrigin(
    host="ftp.example.com",
    path="/pub/file.zip",
    user="anonymous",                 # default
    password="guest@example.com",
    secure=False,                     # set True for explicit FTPS (TLS)
    passive=True,                     # default; the right setting behind NAT
)
RemoteDownload.from_(origin).write_to_file("/tmp/file.zip")
```

### S3Origin — S3 / MinIO / Ceph / LocalStack (`[s3]`)

```python
from remote_download import RemoteDownload, S3Origin

origin = S3Origin(
    bucket="my-bucket",
    key="tenant-1/uploads/abc/photo.jpg",
    endpoint="http://localhost:9000",   # MinIO; omit for real AWS
    access_key="minioadmin",
    secret_key="minioadmin",
    region="us-east-1",
)
result = RemoteDownload.from_(origin).checksum("sha256").write_to(out)
print(result.content_length, "bytes", result.checksum_hex)
```

Setting `endpoint` enables path-style addressing automatically (needed by most S3-compatible services); override with `path_style=...`. Omit `access_key` / `secret_key` to fall back to the default boto3 credential chain (env vars, `~/.aws/credentials`, IAM roles). For high throughput inject a shared boto3 client with `client=...` — an injected client is reused and never closed by the origin.

### AzureBlobOrigin — Azure Blob Storage (`[azure]`)

```python
from remote_download import RemoteDownload, AzureBlobOrigin

origin = AzureBlobOrigin(
    container="downloads",
    blob="tenant-1/photo.jpg",
    connection_string="DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...",
)
RemoteDownload.from_(origin).write_to_file("/tmp/photo.jpg")
```

Authenticate one of three ways: a full `connection_string`, an `endpoint` URL plus an optional `sas_token`, or a pre-built `BlobClient` passed as `client=...`.

### GcsOrigin — Google Cloud Storage (`[gcs]`)

```python
from remote_download import RemoteDownload, GcsOrigin

origin = GcsOrigin(
    bucket="my-bucket",
    object_name="tenant-1/photo.jpg",
    project_id="my-gcp-project",
    credentials_path="/etc/secrets/service-account.json",
)
RemoteDownload.from_(origin).write_to_file("/tmp/photo.jpg")
```

Credentials resolve in order: an explicit `credentials` object, then a service-account JSON file via `credentials_path` (an optional `file:` prefix is stripped), then Application Default Credentials. Pass a pre-built `storage.Client` via `client=...` for reuse / tests — an injected client is never closed by the origin.

### SftpOrigin — SFTP over SSH (`[sftp]`)

```python
from remote_download import RemoteDownload, SftpOrigin

origin = SftpOrigin(
    host="sftp.example.com",
    user="deploy",
    path="/uploads/photo.jpg",
    port=22,
    password="s3cr3t",                       # or use private_key_path=...
)
RemoteDownload.from_(origin).write_to_file("/tmp/photo.jpg")
```

Authenticate with a `password` or a `private_key_path` (Ed25519 / ECDSA / RSA are tried in order). Authentication failures surface as `TerminalDownloadError`; connection and I/O failures as `RetryableDownloadError`. Tune `connect_timeout` / `auth_timeout` as needed.

### HttpxOrigin — HTTP GET with retries / auth / proxy (`[httpx]`)

```python
from remote_download import RemoteDownload, HttpxOrigin

origin = HttpxOrigin(
    "https://api.example.com/files/report.pdf",
    bearer="<token>",                        # or basic_auth=("user", "pass")
    retries=3,
    connect_timeout=30.0,
    response_timeout=300.0,
    proxy="http://corp-proxy:3128",
)
RemoteDownload.from_(origin).write_to_file("/tmp/report.pdf")
```

The richer twin of the stdlib `HttpOrigin`: transport-level retries, `Bearer` / `Basic` auth, granular timeouts and an optional forward `proxy`. Pass an existing `httpx.Client` via `client=...` to reuse a connection pool.

## Concepts

The library is built around a single port (`DownloadOrigin`) and a small set of plain data types. Implement the port and you can read bytes from anything.

### `DownloadOrigin` — the port

A `Protocol` with one method:

```python
def open(self) -> RemoteContent: ...
```

Each backend supplies its own implementation; consumers read bytes through the same API regardless of where they originate. Custom sources only need this single method. Implementations open the resource and return a live `RemoteContent` whose stream the caller closes.

### `RemoteContent` — the opened resource

A context manager coupling the live stream with the metadata the origin resolved:

| Member | Meaning |
|---|---|
| `stream` | live binary stream to read from |
| `content_type` | MIME type the origin advertised, or `None` |
| `content_length` | size in bytes, or `None` when unknown |
| `filename` | suggested filename (Content-Disposition / object key), or `None` |
| `close()` | closes the stream and runs the provider cleanup hook |

### `DownloadResult` — the outcome

A frozen dataclass combining the transfer stats measured by the copy loop with the metadata the origin advertised:

| Field / property | Meaning |
|---|---|
| `bytes_transferred` | total bytes copied from origin to sink |
| `duration` | wall-clock `timedelta` of the copy |
| `checksum_algorithm` | algorithm requested via `.checksum(...)`, or `None` |
| `checksum_hex` | lower-case hex digest, or `None` if none requested |
| `content_type` | content type the origin advertised |
| `content_length` | content length the origin advertised |
| `filename` | suggested filename resolved from the origin |
| `bytes_per_second` | computed throughput (`0` when duration is zero/`None`) |

### `ProgressListener` — progress callback

A callable `(bytes_read: int, total_bytes: int | None) -> None`, fired once per chunk during the copy. `total_bytes` is `None` for chunked / unknown-length origins. Register it with `.on_progress(...)`:

```python
RemoteDownload.from_(origin).on_progress(
    lambda read, total: print(f"{read}/{total}")
).write_to(out)
```

## Error handling — retryable vs terminal

Origins translate provider failures into one of two exceptions so callers can branch on retry semantics **without parsing messages**. Both subclass `RemoteDownloadError`:

- **`RetryableDownloadError`** — transient: a network blip, a 5xx response, a timeout. Callers with a retry budget (a fetch queue, a sync coordinator) should re-enqueue with backoff.
- **`TerminalDownloadError`** — permanent: invalid credentials, a 4xx, a missing object, validation. Retrying the same request will fail again; change something (re-auth, fix the URL / key, escalate) instead.

```python
from remote_download import (
    RemoteDownload,
    RetryableDownloadError,
    TerminalDownloadError,
)

try:
    RemoteDownload.from_(origin).write_to_file("/tmp/photo.jpg")
except RetryableDownloadError:
    enqueue_for_retry(...)        # backoff and try again later
except TerminalDownloadError:
    mark_failed_and_alert(...)    # do not retry; surface to the user
```

This retryable/terminal split is the deliberate improvement over a single exception type: it lets a sync coordinator decide between "keep retrying" and "mark failed, surface to user".

## Java -> Python mapping

This package is a faithful port of the Java library [`remote-download-java`](https://github.com/calcifux/stream-remote-utils-java21). If you know one, you know the other:

| Java | Python |
|---|---|
| `RemoteDownload.from(url)` / `from(origin)` | `RemoteDownload.from_(url)` / `from_(origin)` |
| `.writeTo(out)` / `.fetch()` / `.asInputStream()` | `.write_to(out)` / `.fetch()` / `.as_stream()` |
| `.chunkSize(n)` / `.onProgress(...)` / `.checksum("SHA-256")` | `.chunk_size(n)` / `.on_progress(...)` / `.checksum("sha256")` (or `"SHA-256"`) |
| `S3Origin.builder().bucket(...).key(...).credentials(ak, sk).build()` | `S3Origin(bucket=..., key=..., access_key=..., secret_key=...)` |
| `RemoteDownloadException` | `RemoteDownloadError` (+ `Retryable` / `Terminal` subtypes) |
| `RemoteContent.getContentType()` / `.contentLength()` | `RemoteContent.content_type` / `.content_length` (plain attributes) |

In short: `*Exception` becomes `*Error`, fluent builders become keyword arguments, and getters become attributes.

## License

MIT (c) Carlos Guillermo Reyes Ramiro. See [LICENSE](https://github.com/calcifux/remote-download-python/blob/main/LICENSE).
