jeevesagent.model.litellm

LiteLLM-backed model adapter — one adapter, every provider.

LiteLLM normalises 100+ provider APIs to OpenAI’s chat-completion shape, including:

  • Anthropic (claude-*) — though AnthropicModel is a faster direct path

  • OpenAI (gpt-*) — same; OpenAIModel is the direct path

  • Cohere (command-r, command-r-plus)

  • Mistral (mistral-large, mistral-small, …)

  • AWS Bedrock (bedrock/anthropic.claude-3-...)

  • Google Vertex AI (vertex_ai/gemini-pro)

  • Together AI (together_ai/...)

  • Groq, Replicate, Ollama, …

Because LiteLLM produces OpenAI-shaped streaming chunks, this adapter can subclass OpenAIModel and reuse its entire chunk aggregation / tool-call delta accumulation logic. The only difference: where OpenAIModel calls self._client.chat.completions.create, this one routes through litellm.acompletion.

Usage:

from jeevesagent import Agent
from jeevesagent.model.litellm import LiteLLMModel

agent = Agent(
    "...",
    model=LiteLLMModel("mistral-large", api_key="..."),
)

The string-based resolver in jeevesagent.agent.api recognises several common LiteLLM prefixes (mistral-, command-, bedrock/, vertex_ai/, together_ai/, ollama/, gemini/) so passing the bare model spec works too.

Classes

LiteLLMModel

Talks to any LiteLLM-supported provider.

Module Contents

class jeevesagent.model.litellm.LiteLLMModel(model: str, *, api_key: str | None = None, client: Any | None = None, **litellm_kwargs: Any)[source]

Bases: jeevesagent.model.openai.OpenAIModel

Talks to any LiteLLM-supported provider.

Inherits chunk normalisation, tool-call delta aggregation, and message-conversion from OpenAIModel because LiteLLM produces OpenAI-shaped outputs.