Metadata-Version: 2.4
Name: unlibcore-storage-provider
Version: 1.0.0
Summary: Configurable storage providers for unlib.
Author-email: Matt Coulter <mattcoul7@gmail.com>, Jai Lafferty <jai.lafferty@gmail.com>
Requires-Python: <4.0,>=3.12
Requires-Dist: fsspec<2027,>=2025.7.0
Requires-Dist: pydantic<3,>=2.11.0
Provides-Extra: all
Requires-Dist: adlfs<2027,>=2025.7.0; extra == 'all'
Requires-Dist: gcsfs<2027,>=2025.7.0; extra == 'all'
Requires-Dist: s3fs<2027,>=2025.7.0; extra == 'all'
Provides-Extra: azure
Requires-Dist: adlfs<2027,>=2025.7.0; extra == 'azure'
Provides-Extra: gcs
Requires-Dist: gcsfs<2027,>=2025.7.0; extra == 'gcs'
Provides-Extra: s3
Requires-Dist: s3fs<2027,>=2025.7.0; extra == 's3'
Description-Content-Type: text/markdown

<div align="center">

# unlibcore-storage-provider

Python package for the `unlib` monorepo enabling abstract storage interface (local vs cloud).

![Python](https://img.shields.io/badge/Python-3.12+-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)
![UV](https://img.shields.io/badge/UV-Fast-6E40C9?style=for-the-badge)
![Hatchling](https://img.shields.io/badge/Hatchling-PEP517-6E40C9?style=for-the-badge)
![Ruff](https://img.shields.io/badge/Ruff-Lint-000000?style=for-the-badge)
![Pytest](https://img.shields.io/badge/Pytest-Unit-08979C?style=for-the-badge)
![PyPI](https://img.shields.io/badge/PyPI-Publish-6E40C9?style=for-the-badge)

[![Package CI / unlibcore-storage-provider](https://github.com/mattcoulter7/unlib/actions/workflows/package-unlibcore-storage-provider-ci.yaml/badge.svg?branch=main)](https://github.com/mattcoulter7/unlib/actions/workflows/package-unlibcore-storage-provider-ci.yaml)
[![Package CD / unlibcore-storage-provider](https://github.com/mattcoulter7/unlib/actions/workflows/package-unlibcore-storage-provider-cd.yaml/badge.svg?branch=main)](https://github.com/mattcoulter7/unlib/actions/workflows/package-unlibcore-storage-provider-cd.yaml)

</div>

## Quick Start

Install base dependencies:

```shell
uv sync
```

The base install supports local filesystem configuration without installing
cloud backend SDKs. Install provider extras only for the cloud protocols you
need:

```shell
uv sync --extra s3
uv sync --extra gcs
uv sync --extra azure
uv sync --all-extras
```

Run local checks:

```shell
make lint
make test
```

Format code:

```shell
make format
```

## Development

Packages in this monorepo use `uv`, `ruff`, `pytest`, and `hatchling`. Tox is
not used.

Packages target Python 3.12 or newer. This package contributes to the shared
`unlib` namespace:

```python
from pydantic import TypeAdapter
from unlibcore.storage_provider import StorageProvider

storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "local",
        "root": "./data",
    }
)
```

## Supported Providers

`unlibcore-storage-provider` keeps provider configuration models in the base
package, while cloud filesystem backends are optional extras:

| Provider | Config discriminator | Backend package | Extra |
| --- | --- | --- | --- |
| Local filesystem | `local` | Built into `fsspec` | none |
| Amazon S3 / S3-compatible storage | `s3` | `s3fs` | `s3` |
| Google Cloud Storage | `gcs` | `gcsfs` | `gcs` |
| Azure Blob Storage | `az` | `adlfs` | `azure` |

Install a single backend extra when only one cloud provider is needed:

```shell
pip install "unlibcore-storage-provider[s3]"
pip install "unlibcore-storage-provider[gcs]"
pip install "unlibcore-storage-provider[azure]"
```

Install all cloud backends when a deployment needs multiple protocols:

```shell
pip install "unlibcore-storage-provider[all]"
```

Example S3 configuration:

```python
storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "s3",
        "bucket": "media-library",
        "prefix": "originals",
        "region_name": "ap-southeast-2",
    }
)
```

Example GCS configuration:

```python
storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "gcs",
        "bucket": "media-library",
        "project": "google-project",
    }
)
```

Example Azure Blob configuration:

```python
storage_provider = TypeAdapter(StorageProvider).validate_python(
    {
        "provider": "az",
        "container": "application-data",
        "account_name": "productionstorage",
    }
)
```

## CI/CD

Package CI runs with `uv run pytest` through the reusable
`.github/workflows/package-ci.yaml` workflow. This package's PR workflow is
`.github/workflows/package-unlibcore-storage-provider-ci.yaml`, which only runs when
files under `packages/unlibcore-storage-provider/` or its workflow files change.

Package publishing is handled by `.github/workflows/package-cd.yaml`. Run it
manually and provide `unlibcore-storage-provider` as the package name, or publish a
GitHub Release tagged `packages/unlibcore-storage-provider@X.Y.Z`.
