---
title: Grok API and OpenAI-Compatible Models Guide | Multi-Agent Framework
sidebarTitle: Grok & OpenAI-API-Compatible
description: "Configure xAI Grok and other OpenAI-compatible APIs with AG2. Covers available models, function calling, web search, and setup for any OpenAI-compatible endpoint."
---

[Grok](https://x.ai/) is a family of AI models developed by xAI. Since Grok follows OpenAI's API specification, it works seamlessly with AG2's existing OpenAI client infrastructure — no extra dependencies needed.

This guide shows how to configure Grok (and other OpenAI-compatible APIs) with AG2, including function calling.

## Available Grok Models

The table below lists popular Grok models available via the API (as of March 2026). See [xAI Models and Pricing](https://docs.x.ai/developers/models) for the full and up-to-date list.

| Model ID | Context | Input / Output (per 1M tokens) | Capabilities |
|----------|---------|-------------------------------|--------------|
| `grok-4-1-fast-reasoning` | 2M | $0.20 / $0.50 | Reasoning, functions, structured output, vision |
| `grok-4-1-fast-non-reasoning` | 2M | $0.20 / $0.50 | Functions, structured output, vision |
| `grok-4-0709` | 256K | $3.00 / $15.00 | Reasoning, functions, structured output |
| `grok-code-fast-1` | 256K | $0.20 / $1.50 | Reasoning, functions, structured output (code-optimized) |
| `grok-3` | 131K | $3.00 / $15.00 | Functions, structured output |
| `grok-3-mini` | 131K | $0.30 / $0.50 | Reasoning, functions, structured output |

## Requirements

To get started, ensure you meet the following requirements:

1. **Install the AG2 package:**
   - Run the following command to install the [AG2](https://pypi.org/project/ag2/) package:
   ```bash
   pip install ag2[openai]
   ```

   <Tip>
   If you have been using `autogen` or `ag2`, all you need to do is upgrade it using:
   ```bash
   pip install -U autogen[openai]
   ```
   or
   ```bash
   pip install -U ag2[openai]
   ```
   as `autogen` and `ag2` are aliases for the same PyPI package.
   </Tip>

2. **Obtain a Grok API Key:**
   - Sign up for a Grok account at [x.ai](https://x.ai/) to generate your API key.
   - Refer to the [official Grok documentation](https://docs.x.ai/) for more information about obtaining and using the API key.

## Configuration for Grok

### Basic Configuration

Here's how to configure AG2 to use Grok:

```python
import os
from autogen import LLMConfig

# Basic Grok configuration
llm_config = LLMConfig(
    {
        "model": "grok-4-1-fast-non-reasoning",
        "api_type": "openai",  # Grok is OpenAI-compatible
        "base_url": "https://api.x.ai/v1",
        "api_key": os.environ.get("XAI_API_KEY"),
    },
    temperature=0.7,
)
```

### Configuration with Web Search

Grok offers real-time web search (`web_search`) and X/Twitter search (`x_search`) tools through xAI's [Responses API](https://docs.x.ai/docs/guides/tools/search-tools). To use these with AG2, set `api_type` to `"responses"` and enable web search via `built_in_tools`. Requires AG2 >= 0.12:

```python
llm_config = LLMConfig(
    {
        "model": "grok-4-1-fast-non-reasoning",
        "api_type": "responses",
        "base_url": "https://api.x.ai/v1",
        "api_key": os.environ.get("XAI_API_KEY"),
        "built_in_tools": ["web_search"],
    },
    temperature=0.5,
)
```

## API Parameters

Grok supports standard OpenAI-compatible parameters. You can include additional parameters in your configuration. Some commonly used parameters include:

- **temperature** (number 0..1): Controls randomness in outputs
- **max_tokens** (integer): Maximum tokens in the response
- **top_p** (number 0..1): Nucleus sampling parameter

## Basic Conversation Example

Here's a simple example demonstrating a conversation with Grok:

```python
import os
from autogen import AssistantAgent, UserProxyAgent, LLMConfig

grok_config = LLMConfig(
    {
        "model": "grok-4-1-fast-non-reasoning",
        "api_type": "openai",
        "base_url": "https://api.x.ai/v1",
        "api_key": os.getenv("XAI_API_KEY"),
    },
    temperature=0.5,
    max_tokens=1000,
)

# Create agents
assistant = AssistantAgent(
    name="grok_assistant",
    system_message="You are a helpful AI assistant powered by Grok.",
    llm_config=grok_config,
)

user_proxy = UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=3,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
)

# Start conversation
user_proxy.initiate_chat(
    assistant,
    message="Explain the key differences between Python's asyncio and threading.",
    max_turns=2,
)
```

## Function Calling with Grok

Grok supports advanced function calling capabilities. Here's how to set it up properly with AG2:

<Warning>
**Critical Pattern:** Functions must be registered with the `ConversableAgent`, not passed to `LLMConfig.tools`. Passing function objects to `LLMConfig.tools` will cause a "TypeError: Object of type function is not JSON serializable" error.
</Warning>

```python
import os
from typing import Annotated
from autogen import AssistantAgent, UserProxyAgent, LLMConfig

# Configure Grok for function calling
function_config = LLMConfig(
    {
        "model": "grok-4-1-fast-non-reasoning",
        "api_key": os.getenv("XAI_API_KEY"),
        "base_url": "https://api.x.ai/v1",
        "api_type": "openai",
    },
    temperature=0.3,
    max_tokens=800,
)

# Create function-calling assistant
function_assistant = AssistantAgent(
    name="grok_function_assistant",
    system_message="You are a helpful assistant that can call functions to get weather information and perform calculations. Use the available tools when appropriate.",
    llm_config=function_config,
)

user_proxy = UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=3,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
)

# Define and register functions using the decorator pattern
@user_proxy.register_for_execution()
@function_assistant.register_for_llm(description="Get current weather for a city.")
def get_weather(city: Annotated[str, "The city name"]) -> str:
    # Mock function - in production, call a real weather API
    return f"The current weather in {city} is sunny with a temperature of 22 degrees C."

@user_proxy.register_for_execution()
@function_assistant.register_for_llm(description="Calculate a mathematical expression safely.")
def calculate_math(expression: Annotated[str, "Mathematical expression to evaluate"]) -> str:
    try:
        # Simple evaluation - use a safer parser in production
        result = eval(expression.replace("^", "**"))
        return f"The result of {expression} is {result}"
    except:
        return f"Could not evaluate the expression: {expression}"

# Test function calling
user_proxy.initiate_chat(
    function_assistant,
    message="What's the weather like in Tokyo? Also, can you calculate 15 * 23 + 7?",
    max_turns=2,
)
```

## Function Calling Features

Grok's function calling API supports several advanced features:

### Tool Choice Options

You can control how Grok uses functions with the `tool_choice` parameter:

```python
# Automatic function calling (default)
llm_config = LLMConfig({
    "model": "grok-4-1-fast-non-reasoning",
    "api_type": "openai",
    "base_url": "https://api.x.ai/v1",
    "api_key": os.environ.get("XAI_API_KEY"),
    "tool_choice": "auto",  # Model decides when to call functions
})

# Force function calling
llm_config = LLMConfig({
    # ... config
    "tool_choice": "required",  # Always call a function
})

# Disable function calling
llm_config = LLMConfig({
    # ... config
    "tool_choice": "none",  # Never call functions
})
```

### Parallel Function Calling

Grok supports calling multiple functions simultaneously, which is enabled by default:

```python
llm_config = LLMConfig({
    "model": "grok-4-1-fast-non-reasoning",
    # ... other config
    "parallel_tool_calls": True,  # Enable parallel function calls (default)
})
```

## Best Practices

### 1. Security
- Always use environment variables for API keys
- Never hardcode sensitive information in your code

### 2. Configuration
- Use the correct `api_type` for your provider (`"openai"` for most compatible services)
- Place common parameters like `temperature` at the top level of `LLMConfig`
- Use `extra_body` for provider-specific features not covered by standard parameters
- Use `extra_headers` for custom HTTP headers required by your server (e.g., authentication tokens, routing headers for [VLLM](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#extra-http-headers) or other OpenAI-compatible servers)

### 3. Function Registration
- Register functions with `@agent.register_for_llm()` and `@proxy.register_for_execution()` decorators
- Use type annotations and docstrings for better function calling performance
- Consider using the decorator pattern for complex registration scenarios

### 4. Error Handling
- Implement proper error handling for API calls
- Use secure evaluation methods for mathematical expressions
- Test function calling thoroughly before production use

## Troubleshooting

### Common Issues

**ValidationError: Extra inputs are not permitted**
- **Cause:** Placing parameters like `temperature` inside `config_list` entries
- **Solution:** Move parameters to the top level of `LLMConfig`

**TypeError: Object of type function is not JSON serializable**
- **Cause:** Passing function objects to `LLMConfig.tools`
- **Solution:** Use `@agent.register_for_llm()` and `@proxy.register_for_execution()` decorators instead

**Model not found warnings**
- **Cause:** AG2 doesn't recognize the model name for cost calculation
- **Solution:** Add custom pricing with the `price` parameter or ignore the warning

## Other OpenAI-Compatible Providers

The configuration pattern above works with any service that implements the OpenAI chat completions API — just change `base_url` and `api_key`:

```python
import os
from autogen import LLMConfig

llm_config = LLMConfig(
    {
        "model": "your-model-name",
        "api_type": "openai",
        "base_url": "https://your-provider.com/v1",  # swap this
        "api_key": os.environ.get("YOUR_API_KEY"),    # and this
    },
    temperature=0.7,
)
```

Some popular OpenAI-compatible providers:

| Provider | `base_url` | Example models |
|----------|-----------|----------------|
| [Together AI](https://www.together.ai/) | `https://api.together.xyz/v1` | `meta-llama/Llama-3-70b-chat-hf` |
| [Fireworks AI](https://fireworks.ai/) | `https://api.fireworks.ai/inference/v1` | `accounts/fireworks/models/llama-v3p1-70b-instruct` |
| [Groq](https://groq.com/) | `https://api.groq.com/openai/v1` | `llama-3.3-70b-versatile` |

For provider-specific features, use `extra_body` or `extra_headers` to pass additional parameters without changing the core configuration.

## Additional Resources

- [Grok API Documentation](https://docs.x.ai/)
- [x.ai Function Calling Guide](https://docs.x.ai/docs/guides/function-calling)
- [AG2 Documentation](https://ag2.ai/)
- [OpenAI API Specification](https://platform.openai.com/docs/api-reference)
