Metadata-Version: 2.5
Name: langchain-gmi
Version: 0.1.0
Summary: LangChain chat models for GMI Cloud's OpenAI-compatible Inference Engine
Project-URL: Homepage, https://github.com/dhruvil-codes/langchain-gmi
Project-URL: Documentation, https://github.com/dhruvil-codes/langchain-gmi#readme
Project-URL: Repository, https://github.com/dhruvil-codes/langchain-gmi
Project-URL: Issues, https://github.com/dhruvil-codes/langchain-gmi/issues
Project-URL: Changelog, https://github.com/dhruvil-codes/langchain-gmi/blob/main/CHANGELOG.md
License-Expression: MIT
License-File: LICENSE
Keywords: gmi,inference,langchain,llm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: httpx<1.0,>=0.27.0
Requires-Dist: langchain-core<2.0.0,>=0.3.0
Requires-Dist: langchain-openai<2.0.0,>=0.2.0
Requires-Dist: pydantic<3.0,>=2.0
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.9; extra == 'dev'
Requires-Dist: twine>=6; extra == 'dev'
Description-Content-Type: text/markdown

# langchain-gmi

[![GMI Cloud](https://img.shields.io/badge/GMI%20Cloud-Inference-5B5BD6)](https://www.gmicloud.ai/en/developers) [![Python](https://img.shields.io/badge/Python-3.10%2B-blue)](https://www.python.org/)

LangChain integration for GMI Cloud's OpenAI-compatible Inference Engine. Access DeepSeek V3.2, Claude Sonnet 4.5, and open models through one endpoint, alongside GMI Cloud's NVIDIA H100 GPU inference infrastructure. This is a community integration.

## Install

```bash
pip install langchain-gmi
```

Set an API key in your environment:

```bash
export GMI_API_KEY="your-gmi-api-key"
```

PowerShell:

```powershell
$env:GMI_API_KEY = "your-gmi-api-key"
```

Requires Python 3.10 or newer. The default endpoint is `https://api.gmi-serving.com/v1`. The integration accepts `gmi_api_key`, `api_key`, or `openai_api_key`; `GMI_API_KEY` is the environment variable fallback. Explicit credentials take precedence. Missing or empty credentials raise `ValueError` during construction. `OPENAI_API_KEY` and OpenAI endpoint environment variables are not used to configure this provider.

```python
llm = ChatGMI(
    model="deepseek-v3.2",
    # gmi_api_key=secret_from_your_secret_manager,
    base_url="https://api.gmi-serving.com/v1",
    timeout=60,
    max_retries=2,
    metadata={"application": "support"},
)
```

Runs include `provider="gmi-cloud"` metadata and `ls_provider="gmi-cloud"` for tracing. API keys use Pydantic secret fields and are excluded from provider-specific exports.

## Quickstart

```python
from langchain_gmi import ChatGMI

llm = ChatGMI(model="deepseek-v3.2")
response = llm.invoke("What makes GMI Cloud H100 clusters fast?")
print(response.content)
```

`ChatGMI` inherits LangChain's chat model methods, including `invoke`/`ainvoke`, `stream`/`astream`, `bind_tools`, and `with_structured_output`. Availability and behavior of model capabilities depend on the selected model and the API's current support.

## Async and streaming

```python
import asyncio

from langchain_gmi import ChatGMI


async def main() -> None:
    llm = ChatGMI(model="deepseek-v3.2")

    response = await llm.ainvoke("Give me a one-sentence summary of inference APIs.")
    print(response.content)

    async for chunk in llm.astream("Count from one to three."):
        print(chunk.content, end="", flush=True)
    print()


if __name__ == "__main__":
    asyncio.run(main())
```

For synchronous streaming:

```python
for chunk in llm.stream("Count from one to three."):
    print(chunk.content, end="", flush=True)
```

If your selected model supports usage reporting, pass `stream_options={"include_usage": True}` to `stream` or `astream` and inspect the final chunk's `usage_metadata`. Streaming usage requests are opt-in for this endpoint. Recent `langchain-openai` versions also accept `stream_usage=True`.

## Tool calling

Bind LangChain tools to the model, then invoke it with a message. The model may return tool calls in `AIMessage.tool_calls`; your application is responsible for executing them and returning the results.

```python
from langchain_core.tools import tool

from langchain_gmi import ChatGMI


@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b


llm = ChatGMI(model="deepseek-v3.2").bind_tools([multiply])
message = llm.invoke("What is 6 times 7?")
print(message.tool_calls)
```

See [`examples/02_agent_with_tools.py`](https://github.com/dhruvil-codes/langchain-gmi/blob/main/examples/02_agent_with_tools.py) for a bounded tool loop with calculator and web search tools.

## Structured output

Use a Pydantic model to validate extracted fields:

```python
from pydantic import BaseModel, Field

from langchain_gmi import ChatGMI


class SupportTicket(BaseModel):
    """Fields extracted from a short support request."""

    category: str = Field(description="Short issue category")
    urgency: int = Field(ge=1, le=5, description="Urgency from 1 (low) to 5 (high)")
    summary: str


extract = ChatGMI(model="deepseek-v3.2").with_structured_output(SupportTicket)
ticket = extract.invoke("My account is locked and our team is blocked from a deadline.")
print(ticket)
```

The integration's default structured-output strategy is `function_calling`. See [`examples/03_structured_output.py`](https://github.com/dhruvil-codes/langchain-gmi/blob/main/examples/03_structured_output.py).

## Models and discovery

The short names below are convenience aliases. You can also pass a raw model ID from GMI Cloud's model catalog; other strings pass through unchanged.

| `ChatGMI` model | GMI model ID | Upstream context reference |
| --- | --- | --- |
| `deepseek-v3.2` | `gmi/deepseek-ai/DeepSeek-V3.2` | [163,840 configured positions](https://huggingface.co/deepseek-ai/DeepSeek-V3.2/blob/main/config.json) |
| `claude-sonnet-4.5` | `gmi/anthropic/claude-sonnet-4.5` | [200K tokens](https://platform.claude.com/docs/en/models/overview) |
| `claude-opus-4.5` | `gmi/anthropic/claude-opus-4.5` | [200K tokens](https://platform.claude.com/docs/en/models/overview) |
| `qwen3-vl` | `gmi/Qwen/Qwen3-VL-235B-A22B-Instruct-FP8` | [256K tokens, native](https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct) |
| `gpt-5` | `gmi/openai/gpt-5` | [400,000 tokens](https://developers.openai.com/api/docs/models/gpt-5) |
| `gemini-3-flash` | `gmi/google/gemini-3-flash-preview` | [1,048,576 input tokens](https://ai.google.dev/gemini-api/docs/models/gemini-3-flash-preview) |

These are upstream references, not verified limits for GMI deployments. The DeepSeek value is its model configuration's position limit. Check the GMI catalog for deployment limits and current availability. The aliases above are fixed convenience mappings; use a raw ID from discovery if your account's catalog uses a different ID.

`list_gmi_models()` returns each API model record with a normalized `context_length` integer or `None` when the API does not report a value. It recognizes positive integer `context_length`, `max_model_len`, `max_context_length`, and `context_window` fields in that order. It preserves other API metadata and does not infer missing context lengths from this table.

```python
from langchain_gmi import list_gmi_models

for model in list_gmi_models():
    print(model["id"], model["context_length"])
```

You can also call `list_gmi_models(api_key, base_url=..., timeout=30.0, client=httpx_client)`. A supplied HTTPX client stays open; internally created clients close after the request. HTTP errors raise `httpx.HTTPStatusError`, timeouts propagate, and malformed model-list responses raise `ValueError`. Discovery makes one request without automatic retries.

See [`examples/01_basic_chat.py`](https://github.com/dhruvil-codes/langchain-gmi/blob/main/examples/01_basic_chat.py) for chat, streaming, and discovery. GMI's [API reference](https://docs.gmicloud.ai/inference-engine/api-reference/llm-api-reference) documents `GET /v1/models` and `POST /v1/chat/completions`.

## Live validation

The integration was tested against GMI Cloud on November 14, 2026 with the catalog model `deepseek-ai/deepseek-v4.1-flash-1017`:

```python
llm = ChatGMI(model="deepseek-ai/deepseek-v4.1-flash-1017")
```

Model discovery, sync/async chat, sync/async streaming, tool calling, and Pydantic structured output passed. See the [validation record](https://github.com/dhruvil-codes/langchain-gmi/blob/main/docs/live-validation.md) for scope and dependency versions. These are functional checks, not performance benchmarks.

## How it fits together

```mermaid
flowchart LR
    A[Your application] --> B[LangChain ChatGMI]
    B -->|OpenAI-compatible chat request| C[GMI Cloud API]
    C --> D[Selected model]
    D -->|chat response or streamed chunks| C
    C --> B
    B --> A
    E[list_gmi_models] -->|GET /v1/models| C
```

## Development

From a checkout of this repository:

```bash
pip install -e ".[dev]"
pytest
ruff check .
ruff format --check .
python -m build
twine check dist/*
```

Tests use mocked HTTP transports, with live HTTP blocked. CI checks Python 3.10 through 3.13 and a minimum LangChain dependency configuration. The MIT license is in [LICENSE](https://github.com/dhruvil-codes/langchain-gmi/blob/main/LICENSE).

## Links

- [GMI Cloud developers](https://www.gmicloud.ai/en/developers)
- [GMI Cloud inference API reference](https://docs.gmicloud.ai/inference-engine/api-reference/llm-api-reference)
- [LangChain chat models](https://docs.langchain.com/oss/python/langchain/models)
- [GMI Cloud console](https://console.gmicloud.ai/)
