Metadata-Version: 2.1
Name: serverless-llm-store
Version: 0.6.0
Description-Content-Type: text/markdown
Requires-Dist: accelerate>0.27.2
Requires-Dist: accelerate==0.29.3
Requires-Dist: grpcio==1.49.1
Requires-Dist: grpcio-tools==1.49.1
Requires-Dist: torch==2.3.0
Requires-Dist: transformers==4.42.0
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: accelerate>=0.27.2; extra == "test"
Requires-Dist: transformers>=4.37.2; extra == "test"
Requires-Dist: parameterized; extra == "test"

# ServerlessLLM Store (`sllm-store`)

## About

`sllm-store` is an internal library of ServerlessLLM that provides high-performance model loading from local storage into GPU memory. It can also be installed and used independently in other projects. For more details, see our [quick start guide](https://serverlessllm.github.io/docs/stable/store/quickstart). For experimental support for ROCm, see our [ROCm guide](https://serverlessllm.github.io/docs/stable/store/installation_with_rocm).

## Installation

### Install with pip

```bash
pip install serverless-llm-store
```

### Install from source
1. Clone the repository and navigate to the `store` directory:

``` bash
git clone https://github.com/ServerlessLLM/ServerlessLLM.git
cd ServerlessLLM/sllm_store
```

2. Install the package from source

```bash
rm -rf build
pip install .
```

## Usage
Refer to our [quick start guide](https://serverlessllm.github.io/docs/stable/store/quickstart) for detailed usage instructions.
