Metadata-Version: 2.5
Name: rishi
Version: 0.1.34
Summary: fast local llm driver with tools
Project-URL: Repository, https://github.com/vedicreader/rishi
Project-URL: Documentation, https://vedicreader.github.io/rishi/
Author-email: Karthik <karthik.rajgopal@hotmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: nbdev
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.12
Requires-Dist: aidialog>=0.0.10
Requires-Dist: claude-agent-sdk>=0.2
Requires-Dist: fastcore>=2.0.0
Requires-Dist: httpx
Requires-Dist: huggingface-hub>=1.23.0
Requires-Dist: liteparse<2.13
Requires-Dist: litert-lm-api>=0.14.0
Requires-Dist: llmsurgery>=0.0.13
Requires-Dist: python-fastllm==0.0.41
Requires-Dist: safepyrun>=0.2.3
Requires-Dist: truststore>=0.10
Requires-Dist: uraiyadal>=0.0.1
Provides-Extra: all
Requires-Dist: llama-cpp-python==0.3.30; extra == 'all'
Requires-Dist: mlx-lm>=0.31.3; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'all'
Requires-Dist: mlx-vlm>=0.6.8; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'all'
Requires-Dist: numpy; extra == 'all'
Requires-Dist: soundfile>=0.14.0; extra == 'all'
Requires-Dist: zstandard>=0.22; (sys_platform == 'linux' and python_version < '3.14') and extra == 'all'
Provides-Extra: llama
Requires-Dist: llama-cpp-python==0.3.30; extra == 'llama'
Requires-Dist: numpy; extra == 'llama'
Requires-Dist: soundfile>=0.14.0; extra == 'llama'
Provides-Extra: mlx
Requires-Dist: mlx-lm>=0.31.3; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'mlx'
Provides-Extra: mlx-vlm
Requires-Dist: mlx-vlm>=0.6.8; (sys_platform == 'darwin' and platform_machine == 'arm64') and extra == 'mlx-vlm'
Provides-Extra: ollama
Requires-Dist: zstandard>=0.22; (sys_platform == 'linux' and python_version < '3.14') and extra == 'ollama'
Description-Content-Type: text/markdown

# rishi


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

Rishi gives local and hosted models one `Chat` API. A conversation is portable Python history, so one backend can continue where another stopped.

## Install

Base `rishi` includes LiteRT and the hosted backends. Add an extra for another local runtime:

``` sh
pip install rishi              # LiteRT, hosted APIs, Claude Code, and GitHub Copilot
pip install 'rishi[llama]'     # local GGUF models
pip install 'rishi[mlx]'       # Apple Silicon
pip install 'rishi[ollama]'    # Ollama models
pip install 'rishi[all]'       # every available backend
```

Extras combine, for example `rishi[llama,mlx]`. Hosted providers need their usual credentials. Backend modules load only when selected.

## Chat

`Chat(model)` creates one conversation. Each call appends to `chat.hist`. `resp_text` returns the answer text.

``` python
chat = Chat(gemma4_e2b)
print(resp_text(chat('Give one fact about lobsters.')))
print(resp_text(chat('Give one more.')))
chat.close()
```

    Lobsters are crustaceans, which means they have a hard exoskeleton and eight legs.
    Lobsters are known for their ability to change their color and texture to blend in with their surroundings, a behavior called camouflage.

## Streaming

Pass `stream=True` to iterate markdown strings rendered by Urai. Rishi normalizes each backend’s events; use `stream='raw'` to receive those chunk dictionaries instead.

``` python
chat = Chat(gemma4_e2b)
for markdown in chat('Write a two-line poem about rain.', stream=True):
    print(markdown, end='', flush=True)
chat.close()
```

    Soft drops tap on the pane,
    Washing the world clean again.

## Image

Place an image beside the prompt. `Path`, bytes, and `PIL.Image` inputs use the same message API.

``` python
chat = Chat(gemma4_e2b)
image = repo_root()/'nbs/images.jpeg'
display(Image.open(image))
print(resp_text(chat(['Describe this image in one sentence.', image])))
chat.close()
```

![](index_files/figure-commonmark/cell-4-output-1.png)

    A medium-sized, reddish-brown German Shepherd is looking attentively off to the side with its mouth slightly open.

## Audio

Place an audio file beside the prompt. Use a model with an audio input tower, such as Gemma 4 LiteRT.

``` python
chat = Chat(gemma4_e2b)
audio = repo_root()/'nbs/speech.wav'
print(resp_text(chat(['Transcribe this clip.', audio])))
chat.close()
```

    Dancing in the masquerade, idol truth in plain sight jaded, pop, roll, click, dot, who will I be today or not? But such a tide as moving seems asleep, too full for sound and foam, when that drew from out the boundless deep turns again home, twilight and evening bell and after that.

## One conversation across seven backends

Every backend stores the same canonical messages in `chat.hist`. Each leg below receives the previous leg through `messages=`, adds one turn, and passes the enlarged history on.

The example downloads four local models. The remote, Claude Code, and Copilot legs require their usual credentials.

``` python
legs = [
    ('LiteRT', gemma4_e2b, {}),
    ('Ollama', f'ollama/{ollama_qwen}', {'think': True}),
    ('MLX', mlx_qwen, {}),
    ('llama.cpp', llama_qwen, {'n_ctx': 4096}),
    ('cloud', 'gpt-4.1', {}),
    ('Claude', 'claude/haiku', {}),
    ('Copilot', 'copilot/gpt-4.1', {}),
]

history = []
for i, (name, model, options) in enumerate(legs):
    chat = Chat(model, messages=history, **options)
    prompt = (
        'The code word is amber. Start a travel log with one short sentence.'
        if i == 0 else
        f'What is the code word? Add one short sentence saying {name} received this conversation.'
    )
    reply = chat(prompt)
    print(f'{name}: {resp_text(reply)}')
    history = list(chat.hist)
    chat.close()

print([m['role'] for m in history])
```

    LiteRT: Amber awaits.
    Ollama: The code word is **amber**. Ollama received this conversation.

    Fetching 9 files:   0%|                                                                                                                 | 0/9 [00:00<?, ?it/s]Fetching 9 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 1912.59it/s]

    MLX: The code word is **amber**. MLX received this conversation.

    llama_context: n_ctx_seq (4096) < n_ctx_train (40960) -- the full capacity of the model will not be utilized

    llama.cpp: The code word is **amber**. llama.cpp received this conversation.
    cloud: The code word is amber. Cloud received this conversation.
    Claude: The code word is amber. Claude received this conversation.
    Copilot: The code word is amber. Copilot received this conversation.
    ['user', 'assistant', 'user', 'assistant', 'user', 'assistant', 'user', 'assistant', 'user', 'assistant', 'user', 'assistant', 'user', 'assistant']

## Backend guides

| notebook | backend-specific topics |
|----|----|
| [`00_core.ipynb`](core.html) | runtime registration, compatibility names, skill install |
| [`01_llama.ipynb`](llama.html) | GGUF loading, GPU offload, KV cache, mtmd media |
| [`02_litert.ipynb`](litert.html) | `.litertlm` loading, GPU and NPU, [`bench()`](https://vedicreader.github.io/rishi/litert.html#bench) |
| [`03_mlx.ipynb`](mlx.html) | MLX text and media models, speculative decoding, caches |
| [`04_remote.ipynb`](remote.html) | hosted providers and server-side tools |
| [`06_claude.ipynb`](claude.html) | Claude Code sessions, transcripts, MCP policy |
| [`07_copilot.ipynb`](copilot.html) | authentication, editor headers, model listing |
| [`08_ollama.ipynb`](ollama.html) | daemon lifecycle, thinking levels, `/api/show` |
