Metadata-Version: 2.4
Name: fallbacks3
Version: 0.5.0
Summary: S3-compatible storage library with automatic fallback support
Author-email: Mila de Oliveira <mila.oliveira@palver.com.br>
Project-URL: Homepage, https://github.com/palverdata/fallbacks3
Project-URL: Repository, https://github.com/palverdata/fallbacks3
Project-URL: Issues, https://github.com/palverdata/fallbacks3/issues
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: boto3>=1.26.0
Requires-Dist: pydantic-settings>=2.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: black>=23.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: build>=1.0.0; extra == "dev"
Requires-Dist: twine>=4.0.0; extra == "dev"
Requires-Dist: boto3-stubs[s3]>=1.34.0; extra == "dev"

# fallbacks3

S3-compatible storage library with automatic fallback for the upload process.


## Installation

```bash
pip install fallbacks3
```

For development:

```bash
pip install -e .
```


## Configuration

You must configure a comma-separated list of providers, as well as the scheme of the fallback provider, with environment variables named `PROVIDERS` and `FALLBACK_PROVIDER`.

```bash
# Comma-separated list of provider URIs
PROVIDERS="ps3://access_key:secret_key@ps3.palver.com,r2://access_key:secret_key@account.r2.cloudflarestorage.com"

# Fallback provider scheme (must be one of the providers in PROVIDERS)
FALLBACK_PROVIDER="r2"
```

`FALLBACK_PROVIDER` may also be set as a `<scheme>://<bucket>` URI to redirect fallback
uploads to a specific bucket instead of reusing the original request's bucket. The scheme
must still be one of the providers in `PROVIDERS`.

```bash
# On fallback, upload to bucket "bucket-v2" on the r2 provider
FALLBACK_PROVIDER="r2://bucket-v2"
```


### Provider URI Format

Provider URIs follow the format:
```
<provider_scheme>://<access_key>:<secret_key>@<endpoint>
```

Examples: `r2://key:secret@account.r2.cloudflarestorage.com`, `local://test:testkey@localhost:9000`


### Remote File URI Format

File URIs should specify the exact location from a provider bucket:

```
<provider>://<bucket>/<file_path>
```

Examples: `ps3://palver-whatsapp/audio.mp3`, `s3://my-bucket/documents/report.pdf`

## Usage

```python
from fallbacks3 import Storage

# Initialize storage using environment variables (PROVIDERS and FALLBACK_PROVIDER)
storage = Storage()

# Upload a local file to the provided remote path
stored_file_uri = storage.upload_file(
    remote_file_uri="ps3://palver-whatsapp/audio.mp3",
    local_file_path="/local/path/audio.mp3"
)
print(uri)  # "ps3://palver-whatsapp/audio.mp3"

# Async upload (for async environments)
stored_file_uri = await storage.upload_file_async(
    remote_file_uri="ps3://palver-whatsapp/audio.mp3",
    local_file_path="/local/path/audio.mp3"
)

# Download a file
downloaded_file_path = storage.download_file(
    remote_file_uri="ps3://palver-whatsapp/audio.mp3",
    local_file_path="/local/path/audio.mp3"
)
print(local_path)  # "/local/path/audio.mp3"

# Generate a signed URL for temporary access (default: 60 seconds)
signed_url = storage.generate_signed_url(
    remote_file_uri="ps3://palver-whatsapp/audio.mp3",
    expiration=3600
)
print(signed_url)  # "https://ps3.palver.com/palver-whatsapp/audio.mp3?signature=..."
```

### Upload with Automatic Fallback

If the primary provider fails for the upload method, the library automatically retries with the fallback provider (using the same bucket and file path, unless `FALLBACK_PROVIDER` specifies a bucket), as to ensure files are not lost.

```python
# If ps3 fails, automatically falls back to r2
stored_file_path = storage.upload_file(
    remote_file_uri="ps3://palver-whatsapp/audio.mp3",
    local_file_path="/local/path/audio.mp3"
)
# Returns "r2://palver-whatsapp/audio.mp3" if ps3 failed
```

Raises an exception if both primary and fallback providers fail.

#### Async Upload

For async environments, use `upload_file_async()`:

```python
import asyncio
from fallbacks3 import Storage

storage = Storage()

async def upload():
    stored_file_uri = await storage.upload_file_async(
        remote_file_uri="ps3://palver-whatsapp/audio.mp3",
        local_file_path="/local/path/audio.mp3"
    )
    print(stored_file_uri)  # "ps3://palver-whatsapp/audio.mp3"

asyncio.run(upload())
```

The async method also supports automatic fallback, just like the synchronous version.

### Download and Signed URLs

Download and signed URL generation use the provider specified in the remote file URI.


## Development

Install development dependencies:

```bash
pip install -e ".[dev]"
```

You can also develop with `uv` if available.

```bash
uv pip install -e ".[dev]"
```

Run tests:

```bash
pytest --cov=fallbacks3

# alternatively, with uv:
uv run pytest --cov=fallbacks3
```

## Publishing to PyPI

The project is set up to be automatically uploaded to PyPI when a new tag is pushed. For a successful push, all tests must pass and the project must pass the `check` and `format` requirements.

### One-time setup

Before publishing for the first time:

1. **Configure Trusted Publishing on PyPI**:
   - Go to https://pypi.org/manage/account/publishing/
   - Add a new pending publisher:
     - **PyPI Project Name**: `fallbacks3`
     - **Owner**: `palverdata`
     - **Repository name**: `fallbacks3`
     - **Workflow name**: `publish.yaml`
     - **Environment name**: `pypi`

You must have access to the `pypi` GitHub environment or be approved by required reviewers (`milasd` or `giancarlopro`) to publish.

### Publishing a new version

1. Update version in `pyproject.toml`
2. Create and push a tag:
   ```bash
   git tag v[new version] # eg.: git tag v0.1.0
   git push origin v[new version] # eg.: git push origin v0.1.0
   ```
