Metadata-Version: 2.4
Name: smg-grpc-servicer
Version: 0.5.3
Summary: SMG gRPC servicer implementations for LLM inference engines (vLLM, SGLang, MLX)
Author-email: Chang Su <mckvtl@gmail.com>, Simo Lin <linsimo.mark@gmail.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/lightseekorg/smg
Project-URL: Repository, https://github.com/lightseekorg/smg
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: smg-grpc-proto>=0.4.7
Requires-Dist: grpcio>=1.78.0
Requires-Dist: grpcio-reflection>=1.78.0
Requires-Dist: grpcio-health-checking>=1.78.0
Provides-Extra: vllm
Requires-Dist: vllm>=0.19.0; extra == "vllm"
Provides-Extra: sglang
Requires-Dist: sglang>=0.5.10; extra == "sglang"
Provides-Extra: mlx
Requires-Dist: smg-grpc-proto>=0.4.7; extra == "mlx"
Requires-Dist: mlx>=0.22.0; extra == "mlx"
Requires-Dist: mlx-lm>=0.22.0; extra == "mlx"

# smg-grpc-servicer

gRPC servicer implementations for LLM inference engines. Supports vLLM and SGLang.

## Installation

For vLLM:

```bash
pip install smg-grpc-servicer[vllm]
```

For SGLang:

```bash
pip install smg-grpc-servicer[sglang]
```

## Usage

### vLLM

```bash
vllm serve meta-llama/Llama-2-7b-hf --grpc
```

### SGLang

```bash
sglang serve --model-path meta-llama/Llama-2-7b-hf --grpc-mode
```

## Architecture

```
smg-grpc-servicer[vllm]    ──optional dep──>  vllm     (lazy import)
smg-grpc-servicer[sglang]  ──optional dep──>  sglang   (lazy import)
smg-grpc-servicer           ──depends on──>  smg-grpc-proto  (hard dependency)
vllm                        ──optional──>    smg-grpc-servicer (via vllm serve --grpc)
sglang                      ──optional──>    smg-grpc-servicer (via --grpc-mode)
```

Backend dependencies are isolated via extras to avoid conflicts between vLLM and SGLang.

## Development

See [DEVELOPMENT.md](DEVELOPMENT.md) for local development setup, CI, and release workflows.
