Metadata-Version: 2.4
Name: smg-grpc-servicer
Version: 0.4.2
Summary: SMG gRPC servicer implementations for vLLM (and future backends)
Author-email: Chang Su <mckvtl@gmail.com>, Simo Lin <linsimo.mark@gmail.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/lightseekorg/smg
Project-URL: Repository, https://github.com/lightseekorg/smg
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: smg-grpc-proto>=0.4.2
Requires-Dist: vllm>=0.16.0
Requires-Dist: grpcio>=1.78.0
Requires-Dist: grpcio-reflection>=1.78.0

# smg-grpc-servicer

gRPC servicer implementations for LLM inference engines. Currently supports vLLM,
with future support for SGLang and TensorRT-LLM.

## Installation

```bash
pip install smg-grpc-servicer
```

Or with vLLM's optional dependency:

```bash
pip install vllm[grpc]
```

## Usage

With `vllm serve`:

```bash
vllm serve meta-llama/Llama-2-7b-hf --grpc
```

Or directly:

```bash
python -m smg_grpc_servicer.vllm.server --model meta-llama/Llama-2-7b-hf --port 50051
```

## Architecture

```
smg-grpc-servicer  ──depends on──>  vllm            (hard dependency)
smg-grpc-servicer  ──depends on──>  smg-grpc-proto  (hard dependency)
vllm               ──optional──>    smg-grpc-servicer (lazy import via vllm serve --grpc)
```

This avoids circular dependencies: vLLM only imports `smg-grpc-servicer` at runtime
when `--grpc` is passed, via a lazy import.

## Development

See [DEVELOPMENT.md](DEVELOPMENT.md) for local development setup, CI, and release workflows.
