Metadata-Version: 2.1
Name: serverless-llm-store
Version: 0.7.0
Description-Content-Type: text/markdown
Requires-Dist: accelerate==1.2.1
Requires-Dist: bitsandbytes==0.45.1
Requires-Dist: click>=8.1.7
Requires-Dist: grpcio==1.49.1
Requires-Dist: grpcio-tools==1.49.1
Requires-Dist: peft==0.15.2
Requires-Dist: torch==2.5.1
Requires-Dist: transformers==4.47.1
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Requires-Dist: accelerate>=0.27.2; extra == "test"
Requires-Dist: transformers>=4.37.2; extra == "test"
Requires-Dist: parameterized; extra == "test"

# ServerlessLLM Store (`sllm-store`)

## About

`sllm-store` is an internal library of ServerlessLLM that provides high-performance model loading from local storage into GPU memory. It can also be installed and used independently in other projects. For more details, see our [quick start guide](https://serverlessllm.github.io/docs/stable/store/quickstart). For support for ROCm, see our [ROCm guide](https://serverlessllm.github.io/docs/stable/store/rocm_quickstart).

## Installation

### Install with pip

```bash
pip install serverless-llm-store
```

### Install from source
1. Clone the repository and navigate to the `store` directory:

``` bash
git clone https://github.com/ServerlessLLM/ServerlessLLM.git
cd ServerlessLLM/sllm_store
```

2. Install the package from source

```bash
rm -rf build
pip install .
```

## Usage
Refer to our [quick start guide](https://serverlessllm.github.io/docs/stable/store/quickstart) for detailed usage instructions.
