Metadata-Version: 2.4
Name: ailia_llm
Version: 1.5.1
Summary: ailia LLM
Home-page: https://ailia.ai/en/
Author: ailia Inc.
Author-email: contact@ailia.ai
License: https://ailia.ai/en/license/
Requires-Python: >3.6
Description-Content-Type: text/markdown
Requires-Dist: ailia
Requires-Dist: numpy
Dynamic: author
Dynamic: author-email
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# ailia LLM Python API

!! CAUTION !!
“ailia” IS NOT OPEN SOURCE SOFTWARE (OSS).
As long as user complies with the conditions stated in [License Document](https://ailia.ai/license/), user may use the Software for free of charge, but the Software is basically paid software.

## About ailia LLM

ailia LLM is a library for running LLMs on edge devices. It provides bindings for C++ and Unity.

## Install from pip

You can install the ailia LLM free evaluation package with the following command.

```
pip3 install ailia_llm
```

## Install from package

You can install the ailia LLM from Package with the following command.

```
python3 bootstrap.py
pip3 install ./
```

## Usage

```python
import ailia_llm

import os
import urllib.request

model_file_path = "gemma-2-2b-it-Q4_K_M.gguf"
if not os.path.exists(model_file_path):
	urllib.request.urlretrieve(
		"https://storage.googleapis.com/ailia-models/gemma/gemma-2-2b-it-Q4_K_M.gguf",
		model_file_path
	)

model = ailia_llm.AiliaLLM()
cpu_backend = next(backend for backend in model.get_backend_list()
                   if backend["backend_name"] == "CPU")
model.set_backend(cpu_backend["idx"])  # Optional; call before open().
model.open(model_file_path, n_ctx=8192)

messages = []
messages.append({"role": "system", "content": "語尾に「わん」をつけてください。"})
messages.append({"role": "user", "content": "あなたの名前は何ですか？"})

stream = model.generate(messages)

text = ""
for delta_text in stream:
	text = text + delta_text
print(text)

if model.context_full():
	raise Exception("Context full")

messages.append({"role": "assistant", "content": text})
```

## Tool Use (JSON API)

Use a model that supports tool calling, such as Gemma 4. Define tools with `set_tools()`,
generate with `generate_json()`, and read the complete assistant message with `get_response_json()`.
Append the response and tool results to the history, then generate again for the final answer.

```python
import json
import ailia_llm

model = ailia_llm.AiliaLLM()
model.open("gemma-4-E2B-it-Q4_K_M.gguf", n_ctx=8192)
model.set_tools([{"type":"function","function":{
    "name":"get_weather", "description":"Get the current weather.",
    "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}
}}])
messages = [{"role":"user","content":"What is the weather in Tokyo?"}]
try:
    for _ in range(8):
        for _ in model.generate_json(messages):
            pass  # Consume generation; collecting text deltas is optional.
        if model.context_full():
            raise RuntimeError("Context full; shorten the history.")
        response = model.get_response_json()
        messages.append(response)
        if not response.get("tool_calls"):
            print(response["content"])
            break
        for call in response["tool_calls"]:
            args = json.loads(call["function"]["arguments"])
            # Validate the requested tool and arguments, then execute it here.
            result = {"city": args["city"], "weather": "sunny"}  # Mock result
            messages.append({"role":"tool", "tool_call_id":call["id"],
                             "content":json.dumps(result)})
finally:
    model.set_tools(None)
```

While tools are enabled, use `generate_json()` from the first turn instead of `generate()`.
Read the response before clearing tools. Invalid input or incomplete output raises `AiliaLLMError`;
only append successfully parsed responses and execute their complete tool calls.

### Images and audio

Load a compatible projector with `open_multimodal_projector()` and pass media as content parts to `generate_json()`:

```python
model.open_multimodal_projector("mmproj.gguf")
messages = [{"role": "user", "content": [
    {"type": "text", "text": "Describe this image."},
    {"type": "image", "file_path": "/path/to/image.jpg"},
]}]
for delta in model.generate_json(messages):
    print(delta, end="", flush=True)
```

For audio, use `"type": "audio"` with a supported file such as WAV, MP3 or FLAC.
Use separate prompts for images and audio. Each media part accepts either `file_path` or `data`
(standard Base64-encoded file bytes). Media works with or without tools, and matching media and
conversation prefixes can reuse cached encoder outputs and KV cache.

## API specification

https://github.com/ailia-ai/ailia-sdk
