---
title: "Responses API"
---

The Responses API is the latest API released by Volcengine Ark. It natively supports efficient context management and offers a more concise input/output format. Its tool-calling approach is more convenient — it preserves the ease of use of the Chat API while combining stronger agentic capabilities.

As foundation model technology keeps advancing, the Responses API provides a more flexible foundation for building action-oriented applications of all kinds, and supports a variety of tool-calling extensions, making it well suited for building intelligent assistants, automation tools, and similar scenarios.

## Tutorial

Currently only the `veadk-python` version supports the Responses API, and it has a specific requirement on the `google-adk` version (`google-adk>=1.21.0`). Make sure your environment meets the requirement.

<Tabs items={['pip', 'uv']}>
<Tab value="pip">

```bash
pip install "google-adk>=1.21.0"
```

</Tab>
<Tab value="uv">

```bash
uv pip install "google-adk>=1.21.0"
```

</Tab>
</Tabs>

### Quick Start

**Enable the Responses API**

Just set `enable_responses=True`.

```python
from veadk import Agent

root_agent = Agent(
    enable_responses=True, # Enable the Responses API
)
```

**Model fallbacks**

The Responses API also accepts a `model_name` list for resilience. If a request fails, VeADK tries each subsequent model in order until one succeeds or all models fail.

```python
root_agent = Agent(
    enable_responses=True,
    model_name=["doubao-seed-1-8-251228", "deepseek-r1-250528"],
)
```

**Result**

![responses-api](/assets/images/agents/responses_api.png)

### Multimodal Support

Beyond text interaction, the Responses API also supports multimodal understanding of images, video, and files. You can use the `google.genai.types.FileData` field to pass multimodal data (such as an image path, a video URL, or a file_id generated by the Files API).

`FileData` supports the following ways of passing data:

1. **Files API resource (`file_id`)**
    - Passed via `file_data`; you must add a specific scheme prefix in `file_uri` to distinguish it.
    - Format: `file_uri="file_id://xxxxxxx"`; this value is automatically mapped to the `file_id` field.

2. **Generic resource identifier (URI)**
    All file upload methods supported by Volcengine Ark (including `image_url`, `video_url`, `file_url`) can be handled uniformly through `file_data`.
    1. **Network URL**
        - The system automatically detects the resource type (video, image, file, etc.) based on `mime_type`.
        - Format: `file_uri="https://..."`
    2. **Local file path**
        - Use the local file path directly as `file_uri`; the underlying layer automatically calls the Files API to complete the upload.
        - Format: `file_uri=f"file://{local_path}"`
    3. **Base64 Data URI**
        - Supports passing Base64-encoded data.
        - Reference: [Volcengine Ark: Image Understanding - Base64 Encoding](https://www.volcengine.com/docs/82379/1362931?lang=zh#477e51ce)

#### Example Code

**Note**: All example code below is based on the following `Agent` configuration and `main` function:

```python
import os
import asyncio
import uuid

from google.adk.events import Event
from google.genai import types
from google.genai.types import FileData

from veadk import Agent, Runner
from veadk.memory.short_term_memory import ShortTermMemory

agent = Agent(
    enable_responses=True
)
short_term_memory = ShortTermMemory()
runner = Runner(
    agent=agent,
    short_term_memory=short_term_memory,
)

async def main(message: types.Content):
    session_id = uuid.uuid4().hex
    await short_term_memory.session_service.create_session(
        app_name=runner.app_name, user_id=runner.user_id, session_id=session_id
    )

    async for event in runner.run_async(
        user_id=runner.user_id,
        session_id=session_id,
        new_message=message,
    ):
        if isinstance(event, Event) and event.is_final_response():
            if event.content and event.content.parts:
                if not event.content.parts[0].thought:
                    print(event.content.parts[0].text)
                elif len(event.content.parts) > 1:
                    print(event.content.parts[1].text)
```

#### Image Understanding

<Tabs items={['Local path', 'Files API', 'Image URL']}>
<Tab value="Local path">

Supports image files up to **512MB**.

This approach passes a local file path directly; the system automatically calls the Files API to upload it, then calls the Responses API for analysis.

```python
local_path = os.path.abspath("example-data.png")
message = types.UserContent(
    parts=[
        types.Part(text="Describe this image"),
        types.Part(
            file_data=FileData(
                file_uri=f"file://{local_path}",
                mime_type="image/png"
            )
        )
    ],
)
asyncio.run(
    main(message)
)
```

</Tab>
<Tab value="Files API">

Upload the file via the Volcengine Ark Files API. Supports image files up to **512MB**.

```python
from veadk.utils.misc import upload_to_files_api
local_path = os.path.abspath("example-data.png")
file_id = asyncio.run(upload_to_files_api(local_path))
message = types.UserContent(
    parts=[
        types.Part(text="Describe this image"),
        types.Part(
            file_data=FileData(
                file_uri=f"file_id://{file_id}",
                mime_type="image/png"
            )
        )
    ],
)
asyncio.run(
    main(message)
)
```

</Tab>
<Tab value="Image URL">

Supports image files up to **10MB**.

```python
image_url = "<your-url>"
message = types.UserContent(
    parts=[
        types.Part(text="Describe this image"),
        types.Part(
            file_data=FileData(
                file_uri=f"{image_url}",
                mime_type="image/png"
            )
        )
    ],
)
asyncio.run(
    main(message)
)
```

</Tab>
</Tabs>

#### Video Understanding

<Tabs items={['Files API', 'Video URL']}>
<Tab value="Files API">

Upload the file via the Volcengine Ark Files API. Supports video files up to **512MB**.

When passing a video file in `file_data`, you may optionally add the `video_metadata` field to specify the video frame rate (fps).

You can use the `fps` field to control how often frames are extracted from the video. The default is 1, i.e. one frame is extracted per second and fed to the model for visual understanding. Adjust the `fps` field to balance video length against model processing efficiency.

- When the video changes rapidly or you need to track frame changes (e.g. counting a character's actions), increase `fps` (up to 5) to prevent missing content due to too-sparse sampling.
- When the video changes slowly or frame changes are not important (e.g. analyzing a person's behavior), lower `fps` appropriately (down to 0.2) to balance video length and processing efficiency.

```python
from veadk.utils.misc import upload_to_files_api
local_path = os.path.abspath("example-video.mp4")
file_id = asyncio.run(upload_to_files_api(local_path, fps=0.3)) # optional `fps`
message = types.UserContent(
    parts=[
        types.Part(text="Describe this video"),
        types.Part(
            file_data=FileData(
                file_uri=f"file_id://{file_id}",
                mime_type="video/mp4"
            ),
            video_metadata={        # optional
                "fps": 0.3
            }
        )
    ],
)
asyncio.run(
    main(message)
)
```

</Tab>
<Tab value="Video URL">

Supports video files up to **50MB**.

```python
video_url = "<your-video-url>"
message = types.UserContent(
    parts=[
        types.Part(text="Describe this video"),
        types.Part(
            file_data=FileData(
                file_uri=f"{video_url}",
                mime_type="video/mp4"
            )
        )
    ],
)
asyncio.run(
    main(message)
)
```

</Tab>
</Tabs>

#### Document Understanding

Some models support processing PDF documents; the system understands the entire document's context through its vision capabilities. When a PDF document is passed in, the model paginates the file into multiple images, analyzes and interprets the text and image information within, and combines this information to complete the document-understanding task.

<Tabs items={['Files API', 'Document URL']}>
<Tab value="Files API">

Upload the file via the Volcengine Ark Files API. Supports documents up to **512MB**.

```python
from veadk.utils.misc import upload_to_files_api
local_path = os.path.abspath("example-pdf.pdf")
file_id = asyncio.run(upload_to_files_api(local_path))
message = types.UserContent(
    parts=[
        types.Part(text="Please summarize the content of the document"),
        types.Part(
            file_data=FileData(
                file_uri=f"file_id://{file_id}",
                mime_type="application/pdf"
            )
        )
    ],
)
asyncio.run(
    main(message)
)
```

</Tab>
<Tab value="Document URL">

Supports documents up to **50MB**.

```python
message2 = types.UserContent(
    parts=[
        types.Part(text="Please summarize this document"),
        types.Part(
            file_data=FileData(
                file_uri="<your-file-url>",
                mime_type="application/pdf"
            )
        )
    ],
)
asyncio.run(
    main(message2)
)
```

</Tab>
</Tabs>

## Context Caching

In Responses API mode, VeADK enables session caching by default. This mechanism automatically stores the initial context and dynamically updates it on each conversation turn. On subsequent requests, the system merges the cached content with the new input before sending it to the model for inference. This is especially useful for long-context scenarios such as multi-turn conversations and complex tool calls.

### Viewing Cache Information

You can inspect token usage and cache hits through the returned `Event.usage_metadata` field.

Here is an example `usage_metadata` from a conversation with two turns:

```json
{"cached_content_token_count":0,"candidates_token_count":87,"prompt_token_count":210,"total_token_count":297}
{"cached_content_token_count":297,"candidates_token_count":181,"prompt_token_count":314,"total_token_count":495}
```

**Field descriptions:**

- `cached_content_token_count`: Number of cached tokens hit (i.e. tokens read from the cache).
- `candidates_token_count`: Number of tokens generated by the model (output tokens).
- `prompt_token_count`: Total number of input tokens sent to the model (including both cached and uncached parts).
- `total_token_count`: Total tokens consumed (input + output).

**Caching mechanism notes:**

- Caching only affects input (prompt) tokens, not output (completion) tokens.
- The **cache hit rate** reflects the effectiveness of the caching strategy; the higher the hit rate, the greater the token cost savings.
  - Formula: `cache hit rate = (cached_content_token_count / prompt_token_count) × 100%`
- Input token cost-saving rate: used to quantify the cache benefit across the whole session. It is the core business-facing metric and supports session-level aggregation.

### Cost Savings Example

Based on the sample data above, the cache hit rate is computed as follows:

- **First turn**: 0% (initial state, no cache)
- **Second turn**: `297 / 314 * 100% ≈ 94.58%`

Input token cost-saving rate: `(0 + 297) / (210 + 314) ≈ 56.68%`

This means that with caching enabled, the session's **input token cache hit rate reached 56.68%**, greatly reducing the computation overhead of repeated content. [Volcengine Ark: Cached Token Billing](https://www.volcengine.com/docs/82379/1544106?lang=zh)

Note: The `cached_content_token_count` of turn N is not necessarily equal to the `total_token_count` of turn N-1; if thinking is enabled, the two are not equal.

## Notes

1. **Version requirement**: You must ensure `google-adk>=1.21.0`.
2. **Model support**: Make sure the model you use supports the Responses API (note: Doubao series models after version 0615 generally support it unless otherwise stated).
3. **Caching mechanism**: When VeADK enables the Responses API with a Volcengine Ark model, context (session) caching is enabled by default. However, if `output_schema` is set on the Agent, caching is automatically disabled because that field conflicts with the caching mechanism.

## References

1. [Volcengine Ark: Responses API Migration Guide](https://www.volcengine.com/docs/82379/1585128?lang=zh)
2. [Volcengine Ark: Context Caching](https://www.volcengine.com/docs/82379/1602228?lang=zh#3e69e743)
3. [Volcengine Ark: Cached Token Billing](https://www.volcengine.com/docs/82379/1544106?lang=zh)
4. [Volcengine Ark: Multimodal Understanding](https://www.volcengine.com/docs/82379/1958521?lang=zh)
