Metadata-Version: 2.4
Name: muscles-data-s3
Version: 0.1.1
Summary: S3-compatible ObjectStorePort adapter for muscles-data
Author-email: "Denis B." <denis@butko.info>
Project-URL: Homepage, https://github.com/butkoden/muscles-data-s3
Project-URL: Repository, https://github.com/butkoden/muscles-data-s3
Project-URL: Documentation, https://github.com/butkoden/muscles-data-s3#readme
Project-URL: PyPI, https://pypi.org/project/muscles-data-s3/
Project-URL: Releases, https://github.com/butkoden/muscles-data-s3/releases
Project-URL: muscles-data, https://pypi.org/project/muscles-data/
Project-URL: muscles-data-elasticsearch, https://pypi.org/project/muscles-data-elasticsearch/
Project-URL: muscles-data-opensearch, https://pypi.org/project/muscles-data-opensearch/
Project-URL: muscles-data-qdrant, https://pypi.org/project/muscles-data-qdrant/
Project-URL: muscles-data-redis, https://pypi.org/project/muscles-data-redis/
Project-URL: muscles-data-mongodb, https://pypi.org/project/muscles-data-mongodb/
Project-URL: muscles-data-sqlalchemy, https://pypi.org/project/muscles-data-sqlalchemy/
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: muscles-data<1.0.0,>=0.1.0
Requires-Dist: boto3<2.0,>=1.34
Provides-Extra: dev
Requires-Dist: pytest<9.0,>=8.0; extra == "dev"

# muscles-data-s3

S3-compatible adapter package for `muscles-data`.

This package is intentionally separate from `muscles-data`: the core package
owns typed ports and runtime, while this package owns the boto3-backed
`ObjectStorePort` adapter.

## Related packages

- Core runtime and port contracts:
  [`muscles-data`](https://github.com/butkoden/muscles-data)
- Elasticsearch search adapter:
  [`muscles-data-elasticsearch`](https://github.com/butkoden/muscles-data-elasticsearch)
- OpenSearch search adapter:
  [`muscles-data-opensearch`](https://github.com/butkoden/muscles-data-opensearch)
- Redis key-value/lock/stream adapter:
  [`muscles-data-redis`](https://github.com/butkoden/muscles-data-redis)
- Qdrant vector adapter:
  [`muscles-data-qdrant`](https://github.com/butkoden/muscles-data-qdrant)
- MongoDB document-store adapter:
  [`muscles-data-mongodb`](https://github.com/butkoden/muscles-data-mongodb)
- SQLAlchemy direct SQL resource adapter:
  [`muscles-data-sqlalchemy`](https://github.com/butkoden/muscles-data-sqlalchemy)
- Executable example:
  [`example_data_s3_1`](https://github.com/butkoden/muscular-example/tree/master/example_data_s3_1)

## Install

```bash
python -m pip install muscles-data-s3
```

For local framework development:

```bash
PYTHONPATH=../muscles-data/src:src python3 -m pytest -q
```

## Configuration

```yaml
data:
  resources:
    objects.docs:
      type: s3
      endpoint_url: ${S3_ENDPOINT}
      bucket: documents
      region_name: us-east-1
      prefix: raw
      max_keys: 100
```

The adapter also supports standard boto3 credential options:

- `aws_access_key_id`;
- `aws_secret_access_key`;
- `aws_session_token`;
- `profile_name`;
- the default boto3 environment/provider chain.

## Usage

Register the external factory in the project composition root:

```python
from muscles_data.catalog import DataAdapterCatalog
from muscles_data.config import DataConfig
from muscles_data.ports import ObjectStorePort
from muscles_data.runtime import DataRuntime
from muscles_data_s3 import S3ObjectStoreFactory

catalog = DataAdapterCatalog.with_defaults()
catalog.register(S3ObjectStoreFactory())

runtime = DataRuntime(config=DataConfig.from_raw(config), catalog=catalog)
objects = runtime.require_port("objects.docs", ObjectStorePort)
```

Then use the narrow port:

```python
objects.put_object("docs/readme.txt", b"hello", content_type="text/plain")
blob = objects.get_object("docs/readme.txt")
items = objects.list_objects(prefix="docs", limit=20)
objects.delete_object("docs/readme.txt")
```

Backend-specific request options are available without exposing the boto3
client: `get_object()` accepts `range`, conditional headers and
version/checksum options; `list_objects()` accepts `delimiter` and continuation
helpers; and `delete_object()` accepts version and retention options.

If `prefix: raw` is configured, the public key `docs/readme.txt` is stored as
`raw/docs/readme.txt` in S3 and returned to port consumers without the configured
prefix. This keeps application code independent from bucket layout details.

## Capabilities

`S3ObjectStoreFactory` provides:

- `object_store`;
- `healthcheck`;
- `native_client` only when the resource declares `native_client: true`.

Native access is an advanced project escape hatch:

```python
from muscles_data import DataCapability

client = runtime.require_resource("objects.docs", DataCapability.NATIVE_CLIENT).native_client()
```

Use the native client for backend-specific operations such as presigned URLs,
multipart upload configuration, bucket policies or lifecycle rules. Keep normal
blob reads and writes on `ObjectStorePort`.

## Boundaries

This package owns:

- lazy boto3 client creation;
- S3-compatible endpoint and bucket binding;
- object put/get/list/delete;
- key normalization and optional prefix mapping;
- safe `inspect()` and `doctor()`.

It does not own:

- document parsing;
- application storage schemas;
- lifecycle policies;
- multipart/streaming abstractions in the MVP;
- cross-resource transactions.

`data.doctor` performs `head_bucket`. `data.resources.list` and package
initialization do not open an S3 connection.
