Metadata-Version: 2.4
Name: agent-framework-mlx
Version: 0.3.0
Summary: MLX integration for the Agent Framework
Project-URL: Homepage, https://github.com/filipw/agent-framework-mlx
Author-email: Filip W <contact@strathweb.com>
License-Expression: MIT
License-File: LICENSE
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.9
Requires-Dist: agent-framework-core
Requires-Dist: mlx-lm>=0.28.3
Requires-Dist: pydantic>=2.0.0
Provides-Extra: test
Requires-Dist: pytest; extra == 'test'
Requires-Dist: pytest-asyncio; extra == 'test'
Description-Content-Type: text/markdown

# Agent Framework MLX

This library provides an [Apple MLX LM](https://github.com/ml-explore/mlx-lm) backend for the [Agent Framework](https://github.com/microsoft/agent-framework). It allows you to run Language Models locally on macOS with Apple Silicon using the `mlx-lm` library.

## Features

- **Local Inference**: Run models directly on your Mac using Apple Silicon
- **Streaming Support**: Full support for streaming responses.
- **Configurable Generation**: Fine-tune generation parameters like temperature, top-p, repetition penalty, and more.
- **Message Preprocessing**: Hook into the pipeline to modify messages before they are converted to prompts.
- **Agent Framework Integration**: Seamlessly plugs into the Agent Framework's `BaseChatClient` interface.

## Installation

Ensure you have Python 3.9+ and are running on macOS with Apple Silicon.

```bash
pip install agent-framework-mlx
```

Or install from source:

```bash
git clone https://github.com/filipw/agent-framework-mlx.git
cd agent-framework-mlx
pip install -e .
```

## Usage

### Basic Example

```python
import asyncio
from agent_framework import ChatMessage, Role, ChatOptions
from agent_framework_mlx import MLXChatClient, MLXGenerationConfig

# Initialize the client
client = MLXChatClient(
    model_path="mlx-community/Phi-4-mini-instruct-4bit",
    generation_config=MLXGenerationConfig(
        temp=0.7,
        max_tokens=500
    )
)

# Create messages
messages = [
    ChatMessage(role=Role.SYSTEM, text="You are a helpful assistant."),
    ChatMessage(role=Role.USER, text="Why is the sky blue?")
]

# Get response
response = await client.get_response(messages=messages, chat_options=ChatOptions())
print(response.text)
```

You can also use the client as backbone for Agent Framework agents when building agentic workflows:

```python
from agent_framework import ChatAgent

# notice the client constructed in the previous example now backs the local agent
local_agent = ChatAgent(
    name="Local_MLX",
    instructions="You are a helpful assistant.",
    chat_client=client
)

remote_agent = ChatAgent(
    name="Cloud_LLM",
    instructions="You are a fallback expert. The previous assistant was unsure. Provide a complete answer.",
    chat_client=azure_client
)

builder = WorkflowBuilder()
builder.set_start_executor(local_agent)

builder.add_edge(
    source=local_agent,
    target=remote_agent,
    condition=should_fallback_to_cloud
)

workflow = builder.build()
```

### Streaming

```python
async for update in client.get_streaming_response(messages=messages, chat_options=ChatOptions()):
    print(update.text, end="", flush=True)
```

### Advanced Configuration

You can configure generation parameters globally via `MLXGenerationConfig` or per-request via `ChatOptions`.

```python
config = MLXGenerationConfig(
    temp=0.7,
    top_p=0.9,
    repetition_penalty=1.1,
    seed=42
)
```

### Message Preprocessing

You can intercept and modify messages before they are sent to the model. This is useful for injecting instructions or formatting content.

```python
def inject_instruction(messages):
    if messages:
        messages[-1]["content"] += "\nIMPORTANT: Be concise."
    return messages

client = MLXChatClient(
    model_path="...",
    message_preprocessor=inject_instruction
)
```

## Requirements

- macOS
- Apple Silicon
- Python 3.9+

## License

This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.