Metadata-Version: 2.4
Name: object-storage-client
Version: 0.0.32
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
License-File: LICENSE
Summary: Unified object storage client API
Keywords: object-store,s3,gcs,azure,python
Author-email: Olegs Korsaks <bixority@proton.me>
Requires-Python: >=3.13.9, <3.15.0
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://github.com/bixority/object-storage-client/blob/main/README.md
Project-URL: Homepage, https://github.com/bixority/object-storage-client
Project-URL: Repository, https://github.com/bixority/object-storage-client

# Object Storage Client

A unified object storage client for Rust and Python, supporting S3, GCS, Azure Blob Storage, HTTP/HTTPS, and Local Filesystem. It provides a simple, URL-based API for object operations, including cross-provider copy and move.

## Features

- **Unified API**: Single interface for various storage backends.
- **Cross-Provider**: Copy or move objects between different storage providers (e.g., S3 to Local FS).
- **Pre-signed URLs**: Generate time-limited, credential-free URLs for S3, GCS and Azure.
- **Multi-Language**: Native Rust library with Python 3.13+ bindings.
- **Streaming**: Async streaming support for both Rust and Python.
- **CLI**: `osc` command-line tool for quick operations.

## Supported Schemes

- `s3://bucket/path` (AWS S3)
- `gs://bucket/path` or `gcs://bucket/path` (Google Cloud Storage)
- `az://`, `wasb://`, `wasbs://`, `abfs://`, or `abfss://` (Azure Blob Storage)
- `http://host/path` or `https://host/path` (HTTP/HTTPS)
- `file:///absolute/path` or `local_path` (Local Filesystem)

---

## CLI Usage (`osc`)

The `osc` tool allows you to interact with object storage directly from your terminal.

### Installation

Install the `osc` binary directly from the [Bixority Codeberg crate registry](https://codeberg.org/bixority/object-storage-client) with Cargo. Point Cargo at the registry with an environment variable, then install:

```bash
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
cargo install object-storage-client --registry bixority
```

Alternatively, install straight from Git:

```bash
cargo install --git https://codeberg.org/bixority/object-storage-client
```

Or, if you have the source code, install it from the local checkout:

```bash
cargo install --path .
```

### Examples

- **Upload a local file**:
  ```bash
  osc put my_file.txt s3://my-bucket/remote_file.txt
  ```

- **Download an object**:
  ```bash
  osc get gs://my-bucket/data.json ./local_data.json
  ```

- **Copy between providers**:
  ```bash
  osc cp s3://source-bucket/image.png az://dest-container/image.png
  ```

- **Move an object**:
  ```bash
  osc mv s3://my-bucket/old_name.txt s3://my-bucket/new_name.txt
  ```

- **List objects**:
  ```bash
  osc ls s3://my-bucket/logs/
  ```

- **Delete an object**:
  ```bash
  osc rm s3://my-bucket/temp_file.tmp
  ```

- **Stream an object**:
  ```bash
  osc get-stream gs://my-bucket/large_file.bin
  ```

- **Generate a pre-signed URL** (S3, GCS, Azure):
  ```bash
  # Pre-signed download URL, valid for the default 1 hour
  osc sign s3://my-bucket/report.pdf

  # Pre-signed upload URL (PUT), valid for 15 minutes
  osc sign --method PUT --expires-in 900 s3://my-bucket/upload.bin

  # Pre-signed upload URL binding the exact size and type the client must send
  # (S3 only): the upload is rejected unless Content-Length and Content-Type
  # match, so the object store enforces size/type up front.
  osc sign --method PUT --content-length 1048576 \
      --content-type application/pdf s3://my-bucket/upload.pdf
  ```

---

## Rust Usage

### Installation

The crate is published to the [Bixority Codeberg crate registry](https://codeberg.org/bixority/object-storage-client). Point Cargo at the registry with an environment variable:

```bash
export CARGO_REGISTRIES_BIXORITY_INDEX="sparse+https://codeberg.org/api/packages/bixority/cargo/"
```

Then add `object-storage-client` to your `Cargo.toml`:

```toml
[dependencies]
object-storage-client = { version = "0.0.31", registry = "bixority" }
tokio = { version = "1.0", features = ["full"] }
```

Alternatively, you can depend on it directly from Git:

```toml
[dependencies]
object-storage-client = { git = "https://codeberg.org/bixority/object-storage-client" }
tokio = { version = "1.0", features = ["full"] }
```

### Example

```rust
use object_storage_client::ObjectStorageClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ObjectStorageClient::new();

    // Upload data
    let data = b"Hello from Rust!";
    client.put("s3://my-bucket/hello.txt", &data).await?;

    // Download data
    let retrieved = client.get("s3://my-bucket/hello.txt").await?;
    println!("Retrieved: {:?}", String::from_utf8(retrieved.to_vec())?);

    // Cross-provider copy (S3 to Local)
    client.copy("s3://my-bucket/hello.txt", "file:///tmp/hello_local.txt").await?;

    // Pre-signed URL: time-limited, credential-free access (S3/GCS/Azure)
    use object_storage_client::{SignMethod, SignOptions};
    use std::time::Duration;

    // Pre-signed download (GET) link, valid for one hour
    let download_url = client
        .get_pre_signed_url(
            "s3://my-bucket/hello.txt",
            SignMethod::Get,
            Duration::from_secs(3600),
            &SignOptions::default(),
        )
        .await?;
    println!("Share this download link: {download_url}");

    // Pre-signed upload (PUT) link binding the exact size and type the client
    // must send (S3 only); the store rejects mismatched uploads up front.
    let upload_url = client
        .get_pre_signed_url(
            "s3://my-bucket/upload.bin",
            SignMethod::Put,
            Duration::from_secs(900),
            &SignOptions {
                content_length: Some(1_048_576),
                content_type: Some("application/octet-stream".to_string()),
            },
        )
        .await?;
    println!("Upload directly to: {upload_url}");

    Ok(())
}
```

---

## Python 3.13+ Usage

### Installation

The package is published on [PyPI](https://pypi.org/project/object-storage-client/). Note that it requires Python 3.13+.

```bash
pip install object-storage-client
```

Or if you are developing locally, you can use `maturin`:

```bash
maturin develop
```

### Example

```python
import asyncio
from object_storage_client import ObjectStorageClient

async def main():
    client = ObjectStorageClient()

    # Upload data
    await client.put_object("s3://my-bucket/python_test.txt", b"Hello from Python!")

    # Download data
    data = await client.get_object("s3://my-bucket/python_test.txt")
    print(f"Retrieved: {data.decode()}")

    # List objects
    items = await client.list_objects("s3://my-bucket/")
    print(f"Bucket items: {items}")

    # Stream data
    stream = await client.get_object_stream("s3://my-bucket/python_test.txt")
    async for chunk in stream:
        print(f"Chunk size: {len(chunk)}")

    # Cross-provider move (GCS to S3)
    await client.move_object("gs://my-gcs-bucket/data.csv", "s3://my-s3-bucket/data.csv")

    # Pre-signed URL (S3/GCS/Azure): hand out credential-free, time-limited access
    download_url = await client.get_pre_signed_url("s3://my-bucket/python_test.txt")
    # Bind the exact Content-Length and Content-Type the client must send (S3
    # only); the store rejects uploads that don't match.
    upload_url = await client.get_pre_signed_url(
        "s3://my-bucket/upload.bin",
        method="PUT",
        expires_in_secs=900,
        content_length=1_048_576,
        content_type="application/octet-stream",
    )
    print(f"Download: {download_url}\nUpload: {upload_url}")

if __name__ == "__main__":
    asyncio.run(main())
```

## Developer Instructions

### Prerequisites

- Rust 1.85+ (or latest stable)
- Python 3.13+
- `maturin` (for Python bindings)

### Building

- **Rust**: `cargo build --release`
- **Python**: `maturin build --release`
- **CLI**: `cargo build --bin osc`

### Testing

```bash
cargo test
```
