Metadata-Version: 2.4
Name: llm-jp-vllm
Version: 0.1.0
Summary: vLLM plugins for LLM-jp models
Project-URL: Homepage, https://github.com/llm-jp/llm-jp-vllm
Project-URL: Bug Tracker, https://github.com/llm-jp/llm-jp-vllm/issues
Author-email: Yusuke Oda <odashi@predicate.jp>
License: Apache Software License 2.0
Keywords: llm,llm-jp,transformers,vllm
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.12
Requires-Dist: torch>=2.10.0
Requires-Dist: transformers>=4.57.6
Requires-Dist: vllm>=0.18.0
Description-Content-Type: text/markdown

# vLLM plugins for LLM-jp models

This repository contains vLLM plugins which are necessary to launch LLM-jp models
with vLLM entrypoints.

## Installation

```shell
uv pip install llm-jp-vllm
```

The above command installs `llm-jp-vllm` directory onto your environment's `site-packages`.

## Usage

This repository works with the plugin mechanism on vLLM.

After [necessary change in vLLM](https://github.com/vllm-project/vllm/pull/45241) was merged,
you can use reasoning/tool parsers implemented in this repository
by passing appropriate module names to `--reasoning/tool-parser-plugin` option.

```shell
vllm serve {llm-jp-4 model} \
  --trust-remote-code \
  --reasoning-parser llmjp4 \
  --reasoning-parser-plugin llm_jp_vllm.llmjp4
```

### Tool calling

The `llmjp4` tool parser converts Harmony-format tool calls (including
the parallel call extension of LLM-jp-4) into OpenAI-compatible
`tool_calls`, for both streaming and non-streaming responses:

```shell
vllm serve {llm-jp-4 model} \
  --trust-remote-code \
  --reasoning-parser llmjp4 \
  --reasoning-parser-plugin llm_jp_vllm.llmjp4 \
  --enable-auto-tool-choice \
  --tool-call-parser llmjp4 \
  --tool-parser-plugin llm_jp_vllm.llmjp4
```

## Development

```shell
uv sync --all-extras --dev
./checks.sh  # ruff / mypy
uv run pytest
```