Metadata-Version: 2.5
Name: divyam-llm-interop
Version: 0.3.4
Summary: A lightweight, provider-neutral library for translating LLM requests and responses across model APIs.
Project-URL: Homepage, https://github.com/Divyam-AI/divyam-llm-interop
Project-URL: Source, https://github.com/Divyam-AI/divyam-llm-interop
Project-URL: Issues, https://github.com/Divyam-AI/divyam-llm-interop/issues
Author: DivyamAI Technologies Private Limited
License: Apache-2.0
License-File: LICENSE
License-File: LICENSE_HEADER.txt
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Typing :: Typed
Requires-Python: <4.0,>=3.10
Requires-Dist: lark<2,>=1.2.2
Requires-Dist: pyyaml>=6.0.3
Requires-Dist: typing-extensions>=4.15.0
Description-Content-Type: text/markdown

# Divyam LLM Interop

A minimal, provider‑agnostic library for interoperable AI model requests and
responses. Divyam LLM Interop provides a unified interface for interacting with
models across providers while maintaining consistent request and response
semantics.

The chat interop layer supports Chat Completions, OpenAI Responses, native
Gemini, and the text/client-tool subset of Anthropic Messages. It translates
requests, non-streaming responses, and streaming events through a shared
semantic representation. Documents, native provider tools, citations,
Realtime/Live, batch, and token-count APIs are outside the supported profile and
fail closed rather than being silently flattened. Images are supported only from
Anthropic Messages to Chat Completions and Responses targets: user images become
image parts, and images returned in a `tool_result` follow the turn's tool
messages as a user message, since Chat Completions tool messages carry only
text. A target declaring `supports_vision: false`, or native Gemini, refuses
them.

Anthropic Messages requests translated to another protocol keep what the target
can express and drop what is only a hint:

- Prompt-cache controls, citations on text blocks, `defer_loading`,
  `eager_input_streaming`, metadata keys other than `user_id`, and the
  operational fields `context_management`, `safeguards`, `service_tier`,
  `inference_geo`, `speed`, `fallbacks`, `container` and `diagnostics` are
  dropped. Explicit nulls on optional fields count as absent.
- `output_config.effort` maps to the target's reasoning effort, with `xhigh` and
  `max` lowered to `high`. Without an explicit effort, an enabled thinking budget
  maps to `low` up to 4096 tokens, `medium` up to 16384 and `high` above that,
  and adaptive thinking maps to the target's `default_reasoning_effort`
  (`medium` when it declares none). Targets without a reasoning control drop
  the effort.
- A tool's `strict` flag maps to the function `strict` flag when the schema
  meets OpenAI's strict requirements (every property required and
  `additionalProperties: false` on every object); otherwise the tool is sent
  non-strict.

A request from another protocol to Claude maps `reasoning_effort` or
`reasoning.effort` to adaptive thinking with `output_config.effort` (`minimal`
becomes `low`), or to a thinking budget of 4096, 16384 or 32000 tokens for models
that only take one. `none`, or no effort on a model that reasons by default,
sends thinking disabled; a model that cannot stop reasoning never receives
disabled and gets the lowest effort instead.

### Structured output

Each API's request for JSON reaches the target in the target's own form:

- Gemini's `responseJsonSchema` is JSON Schema already; `responseSchema`, its
  OpenAPI subset, is converted (`OBJECT` becomes `object`, `nullable` a `null`
  type). Either becomes a non-strict `json_schema` response format, since
  Gemini's schemas leave properties optional. The JSON MIME type without a
  schema is JSON mode. Gemini's `text/x.enum` answers with a bare value no
  other API can ask for, and is refused naming the field.
- Claude takes the schema in `output_config.format`. As Anthropic's SDKs do,
  every object gets `additionalProperties: false`, and constraints Claude
  rejects (`minimum`, `maximum`, `multipleOf`, `minLength`, `maxLength`,
  `maxItems`, `minItems` above 1) move into the field's description.
- JSON mode carries an instruction to answer with JSON alone. Claude has no
  JSON mode, and OpenAI's requires the word in the messages; on the Responses
  API it must be in the input rather than the instructions.

### Reasoning across endpoints

Each endpoint's reasoning state belongs to a signing domain: the declared
`anthropic_wire_profile` (or `responses_wire_profile`), otherwise the endpoint's
protocol, provider and model. Endpoints sharing an `anthropic_wire_profile` are
forwarded natively. On any other route to an Anthropic-format client, reasoning
travels back in the content blocks the client returns on its next request:

- Thinking from Claude keeps its text; its signature, and redacted thinking
  data, are wrapped in an envelope (`dvm1.` plus base64url JSON naming the
  domain) so they are replayed only to that domain.
- Readable reasoning from another provider becomes a thinking block for
  display. Opaque state another provider returns travels in a
  `redacted_thinking` envelope block: an OpenRouter-style gateway's
  `reasoning_details`, Gemini `thoughtSignature` values keyed to the function
  call (or text) they came on, or OpenAI Responses reasoning items with their
  `encrypted_content`, in output order. The block goes ahead of the text and
  tool calls, where Claude puts its thinking, since clients read the answer
  from the blocks after it; state reported only after the text has started
  ends the answer. An answer with no such state gets no extra block.

On the next request, each endpoint receives only its own state, and envelopes
never reach a provider: Gemini gets each signature back beside its
`functionCall` or text part, and a Responses endpoint gets its reasoning items
back ahead of the call or message they led to. Requests from Anthropic-format
clients to a reasoning Responses endpoint ask for
`include: ["reasoning.encrypted_content"]`, unless it declares
`supports_encrypted_reasoning: false` or the request uses server-side state.

A Responses-format client (Codex) returns reasoning items itself: encrypted
items go back unchanged on a native route (a shared `responses_wire_profile`)
and are left out on any other, and readable items reach a Chat Completions
endpoint as `reasoning_content`. No route is refused for the reasoning state a
conversation carries; see
[Reasoning across providers](docs/reasoning-across-providers.md).

Tool calls a model cannot verify are rewritten as narration of past events, in
the form `[Earlier, another assistant called the tool Read with input {...}.]`
and `[The tool Read returned: ...]`, without call IDs:

- Claude, with thinking on: the unfinished tool exchange at the end of the
  conversation, when its assistant turn has no thinking this endpoint can
  verify, whoever produced it. Claude then starts a fresh turn with thinking
  on.
- Gemini 3: every step of the current turn (since the latest user message other
  than function responses) whose first function call has no signature from this
  endpoint, with the function responses answering it. An endpoint can declare
  `gemini_foreign_call_policy: "skip_signature"` to keep such calls with
  Google's `skip_thought_signature_validator` value instead, at some cost to
  reasoning quality. Base64 images, including those a tool returned, reach
  native Gemini as `inlineData`.

### Endpoint reasoning overrides

Any capability can be overridden per endpoint. In the Divyam Router this is the
endpoint's `model_configs.endpoint_capabilities`, for example:

```json
{"endpoint_capabilities": {"default_reasoning_effort": "low", "reasoning_dialect": "openrouter"}}
```

`default_reasoning_effort` (`low`, `medium` or `high`) is the effort used when
a client asks for adaptive thinking without a level; the Claude catalogue
declares `high`, and `medium` for Claude Opus 5.5 and Claude Haiku 5.5.
`anthropic_wire_profile`, `reasoning_dialect`, `supports_encrypted_reasoning`
and `gemini_foreign_call_policy` are set the same way.

## Installation

```shell
# Install from PyPI
pip install divyam-llm-interop
```

See [PyPI](https://pypi.org/project/divyam-llm-interop/)

## Usage

The primary API for text based chat request and response conversion
is [ChatTranslator](./src/divyam_llm_interop/translate/chat/translate.py).

### Translate a chat request

```python
from divyam_llm_interop.translate.chat.api_types import ModelApiType
from divyam_llm_interop.translate.chat.translate import ChatTranslator
from divyam_llm_interop.translate.chat.types import ChatRequest, Model

# Translate gemini-1.5-pro Chat Completions API request to a gpt-4.1
# Responses API request
translator = ChatTranslator()
chat_request = ChatRequest(
    body={
        "model": "gemini-1.5-pro",
        "messages": [
            {
                "role": "system",
                "content": (
                    "You are a highly knowledgeable trivia assistant. "
                    "Provide clear, accurate answers across history, geography, "
                    "science, pop culture, and general knowledge. "
                    "When explaining, keep it concise unless asked otherwise."
                ),
            },
            {"role": "user", "content": "What is the capital of India?"},
        ],
        "temperature": 0.7,
        "top_p": 1.0,
        "max_tokens": 100000,
        "presence_penalty": 0.5,
    }
)
source = Model(name="gemini-1.5-pro", api_type=ModelApiType.COMPLETIONS)
target = Model(name="gpt-4.1", api_type=ModelApiType.RESPONSES)
translated = translator.translate_request(chat_request, source, target)
```

Anthropic Messages and Chat Completions both use `messages`, so request-body
heuristics cannot distinguish them safely. Callers must supply the ingress API
type explicitly:

```python
source = translator.find_request_model(
    model_name=request_body["model"],
    request_body=request_body,
    api_type=ModelApiType.ANTHROPIC_MESSAGES,
)
```

Callers that omit `api_type` keep the existing detection behavior: a
`messages` body is treated as Chat Completions. A model name containing
`claude` never changes protocol detection.

The supported profile fails closed when a destination cannot preserve a
requested semantic. In particular, Anthropic `stop_sequences` can map to Chat
Completions, Gemini, or Anthropic, but not to OpenAI Responses because Responses
has no exact stop-sequence request control.

Tool continuations targeting OpenAI Responses use separate top-level
`function_call` and `function_call_output` input items for every source
protocol. The existing
`DIVYAM_RESPONSES_TRANSLATOR_FLATTEN_FUNCTION_OUTPUT=True` opt-in remains for
vLLM versions that require the legacy flattened function-output workaround.

### Translate chat response

```python
from divyam_llm_interop.translate.chat.api_types import ModelApiType
from divyam_llm_interop.translate.chat.translate import ChatTranslator
from divyam_llm_interop.translate.chat.types import ChatResponse, Model

# Translate Responses API response to Chat Completions API Response.
translator = ChatTranslator()

# Response body most likely obtained from a LLM call.
chat_response = ChatResponse(
    body={
        "id": "resp_abc123",
        "object": "response",
        "model": "gpt-4.1",
        "created": 1733400000,
        "output": [
            {
                "role": "assistant",
                "content": [
                    {
                        "type": "output_text",
                        "text": "The capital of India is New Delhi.",
                    }
                ],
            }
        ],
        "usage": {"input_tokens": 35, "output_tokens": 10, "total_tokens": 45},
        "metadata": {"temperature": 0.7, "top_p": 1.0, "presence_penalty": 0.5},
    }
)

source = Model(name="gpt-4.1", api_type=ModelApiType.RESPONSES)
target = Model(name="gpt-4.1", api_type=ModelApiType.COMPLETIONS)
translated = translator.translate_response(chat_response, source, target)
```

## Model Name Resolution and Fallback

When a request model name is resolved against the catalog, matching happens in
this order:

1. Exact normalized name match (`provider/model-name` and case differences are
   normalized).
2. Explicit catalog override via `name_match.regex` in model YAML.
3. Generic best-effort fallback in code:
    - strips punctuation (`-`, `_`, `.`) for comparison,
    - matches runtime names that extend a known catalog name's canonical form
      (longest match wins).

Runtime names that include `-instruct` in the segment you care about (for
example
`llama-3.2-3b-instruct-ft-v1`) align with the `*-instruct` catalog entry; a name
like `llama-3.2-3b-experiment_2026` aligns with the non-instruct base if both
exist. Use `name_match.regex` if you need a different mapping.

This means fine-tuned/runtime names like `gemini-2.0-flash-001`,
`llama-3.2-3b-instruct-ft-custom-v1`, or `qwen-3-8b-adapter_x` can resolve
without adding model-specific regex in config.

### Adding New Models

To add a new model family, start with canonical names only in:
`src/divyam_llm_interop/config/translate/chat/models/*.yaml`.

Example:

```yaml
- name: mymodel-4b
- name: mymodel-4b-instruct
```

In most cases, this is enough because fallback matching handles runtime
suffixes. Add `name_match.regex` only when you need an explicit override or a
non-standard alias.

Example override:

```yaml
- name: mymodel-4b-instruct
  name_match:
    regex:
      - "^vendor-special-4b-v\\d+$"
```

Use override regex when:

- naming does not share a stable base with catalog names,
- multiple catalog names could match and you must force one,
- you need provider-specific alias behavior.

## Development Environment Setup

This project uses [uv](https://docs.astral.sh/uv/) to manage Python, the virtual
environment, and all dependencies. You do not need to install Python or create a
virtual environment manually — uv handles all of that.

### Quick start

```shell
./scripts/setup-dev.sh
```

This will install uv (if not present), find or install a compatible Python (>
=3.10), sync all dependency groups (dev, test, lint), and create a `.venv`
in the project root.

To upgrade all dependencies and regenerate the lock file:

```shell
./scripts/setup-dev.sh --upgrade
```

After setup, point your IDE's Python interpreter at:

```
<project-root>/.venv/bin/python
```

Manage dependencies via `uv add` / `uv remove` or by editing `pyproject.toml`
and running `uv sync --all-groups` — do not use your IDE's built-in package
manager.

## Contributing

We welcome contributions to improve the library!

### How to contribute

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/my-improvement`
3. Make your changes
4. Run tests and linters (see below)
5. Submit a pull request

### Contribution guidelines

* Follow existing code style
* Write clear commit messages
* Include tests when adding features or fixing bugs
* Ensure documentation reflects changes

If you're unsure about a change, feel free to open a discussion or draft PR.

### Code Quality Checks

Before submitting your PR, make sure the code passes all checks.

For in-editor linting, formatting, and type checking, open the repo in VS Code
or Cursor and install the recommended extensions (`.vscode/extensions.json`).
Settings use `pyproject.toml` (ruff) and `pyrightconfig.json` (types).

Agent instructions for any AI tool: see [AGENTS.md](AGENTS.md).

#### Run all checks at once

```shell
./scripts/lint.sh
```

#### Auto-fix formatting and lint issues

```shell
./scripts/lint.sh --fix
```

#### Run individual checks

```shell
# Format code
uv run ruff format .

# Check formatting (without modifying files)
uv run ruff format --check .

# Lint code
uv run ruff check .

# Auto-fix linting issues (where possible)
uv run ruff check --fix .

# Type check
uv run pyright .
```

#### License Headers

All `.py` files must include the project license header. The `insert-license`
pre-commit hook checks this automatically on every commit. If a file is missing
the header, you'll see an error like:

```
insert-license..............................................................Failed
- hook id: insert-license
- exit code: 1
- files were modified by this hook

Fixing file: src/divyam_llm_interop/new_module.py
```

The hook inserts the missing headers for you, but the commit is aborted so you
can review the changes. To complete the commit:

```shell
git add -u
git commit
```

To fix all files at once (outside of a commit):

```shell
pre-commit run insert-license --all-files
git add -u
```

The expected header is defined in `LICENSE_HEADER.txt` at the repository root.

### Running Tests

Unit tests (no API keys needed):

```shell
./scripts/test.sh
```

With coverage report:

```shell
./scripts/test.sh --coverage
```

Or manually:

```shell
uv run pytest
uv run pytest --cov=src --cov-report=term-missing
```

#### Integration Tests

Integration tests make live API calls to OpenAI and Google Gemini, translate
responses across all three API formats (Completions, Responses, Gemini), and
validate the output. They are skipped automatically when API keys are not set.

```shell
# Run all integration tests (skips providers without keys)
OPENAI_API_KEY=sk-... GEMINI_API_KEY=... ./scripts/test.sh --integration

# Or via pytest directly
OPENAI_API_KEY=sk-... pytest -m integration

# Run a subset
pytest -m integration -k Completions        # only completions source
pytest -m integration -k Gemini             # only gemini source
pytest -m integration -k streaming          # only streaming tests
pytest -m integration -k tool_call          # only tool call tests
```

| Environment variable       | Default                                            | Description                       |
|----------------------------|----------------------------------------------------|-----------------------------------|
| `OPENAI_API_KEY`           | —                                                  | OpenAI API key                    |
| `OPENAI_BASE_URL`          | `https://api.openai.com/v1`                        | Override for compatible endpoints |
| `OPENAI_COMPLETIONS_MODEL` | `gpt-4.1-mini`                                     | Model for completions calls       |
| `OPENAI_RESPONSES_MODEL`   | `gpt-4.1-mini`                                     | Model for responses calls         |
| `GEMINI_API_KEY`           | —                                                  | Google AI Studio API key          |
| `GEMINI_BASE_URL`          | `https://generativelanguage.googleapis.com/v1beta` | Override for Vertex               |
| `GEMINI_MODEL`             | `gemini-2.5-flash`                                 | Model for Gemini calls            |

## Releasing

Releases are published to PyPI by the **Release** GitHub workflow
(`.github/workflows/release.yml`), never from a local machine: a local
`uv publish` skips CI and leaves the published version without a tag.

To release, run the workflow from `main` (Actions → Release → Run workflow):

- **version** — leave empty to release `main`'s version if it is not yet on
  PyPI, or otherwise the next patch. Set it (for example `0.4.0`) to choose the
  version instead.
- **test-pypi** — publish to TestPyPI only, as a dry run. `main` is not bumped
  and nothing is tagged.

The workflow refuses a version already on PyPI or not higher than the latest,
and a run from any branch but `main`. After CI passes it commits the version to
`pyproject.toml` and `uv.lock` on `main`, tags that commit `vX.Y.Z`, then builds
and publishes the tagged commit and creates the GitHub release. The commit and
the tag are pushed together, before publishing: if `main` moved while CI ran,
the push is refused and nothing is published, so run the workflow again. If a
run fails after tagging, running it again with the same version releases the
existing tag.

Pushing a `vX.Y.Z` tag yourself also releases it, provided the tagged commit is
on `main` and its `pyproject.toml` already names that version.

After a release, raise the pin in the consumers, which pin an exact version:
`divyam-llm-interop` in `divyam_python_libs/pyproject.toml` and
`divyam_router_controller/pyproject.toml`, then relock both
(`poetry lock`). Bump them together: the Router depends on both, so differing
pins do not resolve.

## License

This project is licensed under the Apache License, Version 2.0. You may obtain a
copy of the License at:

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the [LICENSE](LICENSE)
file for the full license text.

---

Copyright © 2025 DivyamAI Technologies Private Limited. All rights reserved.

### Portable Responses tools and endpoint capabilities

See [Portable Responses tools](docs/portable-responses-tools.md) for generic custom-tool and namespace translation, endpoint overrides, native state compatibility and streaming validation.
