Metadata-Version: 2.5
Name: modmex-ai
Version: 0.5.0
Summary: Lightweight AI flows, agents, tools, and model adapters for event-driven Python systems.
Author: Modmex
License: MIT
License-File: LICENSE
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: <4.0,>=3.11
Requires-Dist: modmex<2.0.0,>=1.2.0
Provides-Extra: async
Requires-Dist: httpx<1.0,>=0.27; extra == 'async'
Provides-Extra: aws
Requires-Dist: boto3<2.0.0,>=1.24.66; extra == 'aws'
Provides-Extra: lambda
Requires-Dist: modmex-lambda<1.0.0,>=0.5.10; extra == 'lambda'
Provides-Extra: realtime
Requires-Dist: websockets<16.0,>=15.0; extra == 'realtime'
Description-Content-Type: text/markdown

# modmex-ai

Lightweight, provider-neutral agents, flows, tools, and voice runtimes for Python.

[![CI](https://img.shields.io/github/actions/workflow/status/modmex/modmex-ai/ci.yml?branch=main&logo=github&label=CI)](https://github.com/modmex/modmex-ai/actions/workflows/ci.yml)
[![Coverage](https://img.shields.io/codecov/c/github/modmex/modmex-ai?label=coverage)](https://codecov.io/gh/modmex/modmex-ai)
[![PyPI](https://img.shields.io/pypi/v/modmex-ai.svg)](https://pypi.org/project/modmex-ai/)
[![Python Versions](https://img.shields.io/pypi/pyversions/modmex-ai.svg)](https://pypi.org/project/modmex-ai/)
[![License](https://img.shields.io/github/license/modmex/modmex-ai.svg)](https://github.com/modmex/modmex-ai/blob/main/LICENSE)

`modmex-ai` is a small runtime for applications that need an agent to reason,
call tools, hand off work, and return a controlled result. It fits equally well
in HTTP APIs, CLIs, background workers, WebSocket servers, notebooks, desktop
applications, voice systems, and event-driven/serverless workloads.

Its core has no provider SDK dependency. OpenAI is the first complete adapter;
the core contracts remain reusable for another text, speech, or realtime
provider.

The basic execution model is intentionally simple:

```text
input -> Flow -> one or more Agents -> output, tools, handoffs, and events
```

## What it provides

- `Agent` for instructions, structured output, tools, guardrails, and streaming.
- `Flow` for multi-agent routing, handoffs, continuation, and sync/async execution.
- Sessions, snapshots, optimistic concurrency contracts, and approval checkpoints.
- Realtime and chained voice contracts that keep STT, reasoning, and TTS composable.
- Provider-neutral tracing, usage, evals, and host-owned observability.

Install the core package in any Python application:

```bash
pip install modmex-ai
```

Add only the optional transport needed by a deployment:

```bash
pip install 'modmex-ai[async]'      # native async HTTP
pip install 'modmex-ai[realtime]'  # WebSocket-based Realtime adapters
```

## Quick start

```python
from modmex import BaseModel
from modmex_ai import Agent, FakeModel, Flow


class TriageOutput(BaseModel):
    intent: str
    confidence: float


triage = Agent(
    name="triage",
    instructions="Classify the incoming message.",
    output_type=TriageOutput,
    handoffs=["support"],
)

flow = Flow(
    name="analyze-message",
    entrypoint=triage,
    agents=[],
    model=FakeModel(['{"intent":"support","confidence":0.91}']),
)

result = flow.run("I need help")
result.output.model_dump()
```

The same `Flow` can be called by a REST handler, a queue consumer, a CLI
command, or a long-lived WebSocket connection. Event-driven systems are a
natural fit, but they are not required.

Multi-agent delegation uses handoff function tools, not a field in the model
output. A handoff defaults to transfer_to_agent_name; its typed input is
validated and delivered to an on_handoff callback before the next agent
continues with the conversation.

## Multimodal inputs

Messages can combine typed text, document, and image parts without exposing a
provider payload in application code. `FileInput` and `ImageInput` accept a
URL, a provider file id, or inline bytes/Base64 as appropriate. The provider
adapter decides which source it supports.

The OpenAI Responses adapter maps these parts to the official `input_text`,
`input_file`, and `input_image` request items. For documents stored privately,
prefer a short-lived signed URL and create it only for the model request; do
not persist it in conversation state or publish it in events.

```python
from modmex_ai import FileInput, InputDetail, Message, TextInput
from modmex_ai.models import ModelRequest
from modmex_ai.providers.openai import OpenAIResponsesModel

request = ModelRequest(
    messages=[
        Message(
            role="user",
            content=[
                TextInput(text="Extract the organization name and expiration date."),
                FileInput(
                    url=presigned_file_url,
                    filename="agreement.pdf",
                    media_type="application/pdf",
                    detail=InputDetail.HIGH,
                ),
            ],
        )
    ]
)

response = OpenAIResponsesModel("gpt-5.6", api_key="...").complete(request)
print(response.output_text)
```

`OpenAIChatModel` deliberately rejects `FileInput` and `ImageInput` until that
adapter implements their distinct wire format. Use `OpenAIResponsesModel` for
multimodal input.

## Durable sessions and approvals

The core provides portable `SessionSnapshot` and `PersistedFlowState` models,
plus optimistic-concurrency store contracts. A host chooses the database and
owns the lifecycle; this keeps the framework usable in a web application, a
worker, or serverless infrastructure without importing a storage SDK into the
core.

Tools can require an externally verified approval. When a protected tool is
requested, a Flow raises `FlowSuspended` with the exact pending tool call and
a serializable checkpoint. Persist it, collect the signed approval in the
host, then resume the Flow without asking the model to decide again.

For Lambda-oriented hosts, install the optional connector integration and use
the DynamoDB stores from `modmex-ai`:

```bash
pip install 'modmex-ai[lambda]'
```

```python
from modmex_ai.persistence import DynamoDbDurableSessionStore
from modmex_lambda.connectors.dynamodb import Connector

store = DynamoDbDurableSessionStore(Connector("sessions-table"))
```

Other applications can implement the same `DurableSessionStore` and
`FlowStateStore` contracts with Postgres, Redis, or their existing storage.

## Realtime voice sessions

`VoiceAgentSession` is the common contract for voice sessions. There are two
implementations with distinct responsibilities:

- `OpenAIRealtimeSession` para speech-to-speech por WebSocket/SIP.
- `ChainedVoiceSession` para conservar el control explícito de
  STT → `Agent`/`Flow` → TTS.

Ambas exponen eventos normalizados, uso acumulado, herramientas y handoffs. El
adaptador de OpenAI conserva el payload específico del proveedor únicamente en
su frontera; el core usa `tool_call_id` para no confundir llamadas de tools con
identificadores de llamadas telefónicas.

Install the optional WebSocket transport only in workloads that need it:

```bash
pip install 'modmex-ai[realtime]'
```

The OpenAI adapter reuses `Agent` tools, handoffs, tool guardrails, context,
usage, and traces. It does not depend on the OpenAI Agents SDK. For a SIP call
that was already accepted by your server, connect the worker using its
`realtime_call_id`:

```python
import asyncio

from modmex_ai import Agent
from modmex_ai.providers.openai import (
    OpenAIRealtimeClient,
    OpenAIRealtimeSessionConfig,
)


async def run_voice_call(realtime_call_id: str) -> None:
    agent = Agent(
        name="support",
        instructions="Help the caller concisely and use tools when needed.",
    )
    client = OpenAIRealtimeClient()
    session = await client.connect(
        agent=agent,
        realtime_call_id=realtime_call_id,
        config=OpenAIRealtimeSessionConfig(voice="marin"),
    )
    async with session:
        await session.configure()
        await session.create_response(
            instructions="Greet the caller and ask how you can help."
        )
        async for event in session.events():
            if event.type == "response.done":
                print(session.usage.model_dump())


asyncio.run(run_voice_call("rtc_u1_example"))
```

When the model emits a function call, the session validates and executes the
registered Modmex-AI tool, sends its output to the socket, and asks Realtime to
continue. A `transfer_to_*` handoff updates the active agent's instructions and
tool surface in the same Realtime session.

For a SIP accept endpoint, build the accept body from the exact same session
configuration used after the WebSocket connects. That prevents model, voice,
instructions and tool definitions from drifting between both phases:

```python
accept_payload = OpenAIRealtimeSessionConfig(voice="marin").to_accept_payload(agent)
```

## Chained voice pipeline

Use a chained session when the application must choose its transcription and
speech providers, or needs text-agent behavior between audio turns. A `Flow`
is preferred when the conversation can hand off to another agent.

```python
session = ChainedVoiceSession(
    flow=conversation_flow,
    speech_to_text=transcriber,
    text_to_speech=synthesizer,
    context={"workspace_id": "workspace-1"},
)

audio_reply = await session.process_audio(caller_audio)
async for event in session.voice_events():
    # transcript_final, tool_finished, handoff_completed,
    # response_completed, session_ended, or error
    handle_voice_event(event)
```

The STT/TTS providers are small async protocols, so concrete provider SDKs stay
outside the core package. Close either session with a `VoiceTerminationReason`
to make lifecycle reporting explicit.

For bounded audio turns, the OpenAI adapters use the Audio API directly through
the lightweight HTTP client—no provider SDK is installed:

```python
from modmex_ai.providers.openai import (
    OpenAISpeechProvider,
    OpenAITranscriptionProvider,
)

session = ChainedVoiceSession(
    flow=conversation_flow,
    speech_to_text=OpenAITranscriptionProvider(model="gpt-4o-mini-transcribe"),
    text_to_speech=OpenAISpeechProvider(voice="marin", response_format="wav"),
)
```

To continue a chained conversation in another process, persist the host's
`Session` implementation and `session.continuation`. The continuation tracks
the active agent and provider state; the host owns durable history storage.
For a small self-contained host, `SessionSnapshot.from_session(session)` can
serialize an `InMemorySession` and later restore it with
`snapshot.to_memory_session()`.

`ChainedVoiceSession.process_audio_stream()` supports providers that implement
streaming STT and TTS contracts. It emits transcript deltas, waits for a final
transcript before invoking the text flow, then yields playable audio chunks.
The callable adapters make it possible to plug in a provider-specific function
without making that provider a dependency of `modmex-ai`.

## Native async models

Models can optionally expose `async def acomplete(request)`. `Agent.run_async`
and `Flow.run_async` use it directly, including async tools; existing sync
models retain their compatible worker-thread fallback. The built-in OpenAI
clients expose `acomplete()` as well. Install the optional native transport to
make them select `httpx` automatically:

```bash
pip install 'modmex-ai[async]'
```

Without that extra, the compatible stdlib transport runs in a worker and keeps
the event loop responsive.

## Realtime transcription

For live microphone or telephony media, use the transcription-only WebSocket
adapter instead of a speech-to-speech agent session. It emits `Transcription`
deltas as audio arrives and one final value after manual commit:

```python
from modmex_ai.providers.openai import OpenAIRealtimeTranscriptionProvider

transcriber = OpenAIRealtimeTranscriptionProvider()
async for transcript in transcriber.transcribe_stream(pcm_chunks, context=context):
    render(transcript.text, final=transcript.is_final)
```

The adapter uses 24 kHz PCM by default and requires `modmex-ai[realtime]`.
For completed audio turns, `OpenAITranscriptionProvider` remains the simpler
request-based alternative.

For a long-lived call, open one reusable session and commit each user turn:

```python
session = await transcriber.connect()
await session.append_audio(pcm_chunk)
await session.commit_turn()
async for transcript in session.events():
    correlate(transcript.item_id, transcript.text, transcript.is_final)
```

## Turn cancellation and provider retries

`ChainedVoiceSession` accepts `VoiceTurnOptions(max_provider_retries=...)`.
Retries apply only to STT/TTS provider I/O, never to the text flow, tools, or
handoffs. Call `cancel_current_turn()` to cooperatively stop a streaming turn
before its next provider chunk; hosts can still cancel the owning task when an
immediate interruption of a blocked transport is required.

After every completed, cancelled, timed-out, or failed bounded turn,
`ChainedVoiceSession.last_turn` contains a `VoiceTurnOutcome`. Its metrics
separate STT, agent, TTS, total latency, and provider retry count.
Pass a `VoiceTurnObserver` to export those outcomes to the host's own logs or
metrics backend without adding an observability dependency to this package.

## Model Context Protocol (MCP)

`MCPClient` is the low-level protocol client. `MCPClientConnection` adds
discovery, tool adaptation, caching and lifecycle management so an `Agent` can
consume one or more remote MCP servers through `mcp_clients`:

```python
from modmex_ai import Agent, MCPClientConnection, MCPClientManager

async with MCPClientManager([
    MCPClientConnection("https://example.com/mcp"),
]) as mcp_clients:
    agent = Agent(
        name="assistant",
        instructions="Use the available remote tools when appropriate.",
        mcp_clients=mcp_clients,
    )
    result = await agent.run_async("Help me with my request")
```

Use `client.list_tools()` directly only when the application needs to manage
the remote tool catalog itself. Resources and prompts remain explicit:

```python
connection = mcp_clients[0].client
menu = await connection.read_resource("pizza://menu")
prompt = await connection.get_prompt("assistant")
```

The client supports MCP `2026-07-28`, pagination, structured tool results,
modern metadata/headers and buffered SSE responses. It does not create
transport sessions. The MCP server transport and Lambda/API Gateway
integration are provided by `modmex-lambda`.

### Concurrent MCP discovery and tool calls

Async agent runs and `MCPClientManager` connect to MCP servers concurrently.
Each connection caches its tool catalog. Discovery and cursor pagination within
one server stay ordered.

Set `Agent(..., parallel_tool_calls=True)` to execute the tool calls returned in
one model response concurrently in `run_async` and `arun_stream`. The default is
`False`. Enable this for independent async tools, including MCP calls; synchronous
functions still run on the event loop. Tools share the run context, so avoid
parallel calls that depend on each other's results or mutate the same state.
Streaming progress and finish events arrive as tools produce them; results sent
back to the model retain the original call order. A failed or cancelled batch
cancels and awaits its remaining tasks. Synchronous runs remain sequential.


## Execution resilience and latency (0.5)

Existing agents retain fail-fast tool handling unless explicitly configured:

```python
from modmex_ai import Agent, ModelSettings, RunTimeoutError, execution_budget

agent = Agent(
    name="driver",
    instructions="Use tools to perform operations; never claim unconfirmed writes succeeded.",
    model=model,
    tools=tools,
    tool_error_policy="return_error",
    execution_timeout=25,
    settings=ModelSettings(reasoning_effort="low", max_tokens=500),
)
try:
    result = await agent.run_async("Report arrival")
except RunTimeoutError:
    # The application persists pending work and sends its own deterministic fallback.
    # A timed-out write may already have completed: verify before retrying.
    ...
```

`return_error` converts ToolExecutionError/ToolValidationError into sanitized function
outputs with `ok: false`, an error code, and `retryable: false`. The model can then
respond without exposing internal exception text. There are no automatic tool retries.
Approvals, remote MCP requests for human input, guardrail blocks, cancellation, unknown tools and execution deadlines still
propagate. With recovery enabled, remote MCP tools use their buffered call path even
in agent streaming; model streaming remains available, but MCP progress is not emitted.

Execution deadlines are optional and shared with nested agents and HTTP retries.
Wrap a whole Flow in `with execution_budget(seconds):` to share a single deadline across
handoffs. Async execution cancels cooperative operations when the budget expires.
Synchronous execution checks deadlines between operations and bounds HTTP timeouts;
it cannot forcibly interrupt arbitrary blocking Python functions or undo writes.
The same limitation applies to blocking synchronous functions called inside async code
and synchronous model fallbacks running in worker threads. Closing a stream early
is still the caller's responsibility. Do not hold a stream open while running unrelated
work in the same task: its execution budget remains active until it is closed.

Trace events include model response duration, first streamed event latency, message
count, UTF-8 context/schema sizes, and tool duration/outcome. Byte counts are not token
estimates. New metrics do not include prompt text, arguments or exception messages;
existing application tracing policies still apply. Non-streaming model failures produce
`model_error` with exception type and elapsed time. Attach existing Trace observers to
export these metrics; supply RetryConfig(on_retry=...) for sanitized retry events.

HTTP retries honor Retry-After (seconds or HTTP date), skip exhausted billing quota,
respect the active execution budget and support optional jitter_seconds. An explicitly
empty status_codes set disables retries. Streams are never replayed after output starts.
Transport/network timeouts retain their existing fail-fast policy. No HTTP connection
pooling was replaced: AsyncHttpClient already owns/reuses its transport explicitly.

`reasoning_effort` is omitted by default and maps to Responses `reasoning.effort` or
Chat Completions `reasoning_effort`. Model support and accepted effort levels vary;
choose compatible settings explicitly. The library validates known effort names but
does not maintain a model capability catalog or silently remove sampling parameters.
See https://developers.openai.com/api/docs/guides/reasoning.

These mechanisms do not send user messages, escalate business cases, change an agent's
permissions, or compact business context automatically. Those remain application policies.


## Incremental context compaction

Compaction is opt-in and provider-neutral. It is independent of Agent/Flow execution,
so hosts can compact the exact business context they assemble, without changing agents.

```python
from modmex_ai import ContextCompactor, CompactionSnapshot, execution_budget

compactor = ContextCompactor(
    model=summary_model,  # Host chooses the model and credentials.
    threshold=24_000,     # Serialized UTF-8 bytes by default, not tokens.
    keep_recent_turns=4,
    max_input_size=32_000,
    max_summary_bytes=6_000,
    preservation_instructions="Preserve load IDs, assignments and unresolved requests.",
)
with execution_budget(25):  # Shared budget for compaction AND the ensuing agent run.
    result = await compactor.compact_async(full_history, previous=saved_snapshot, trace=trace)
    # Persist result.snapshot using your repository and optimistic concurrency control.
    # Supply result.messages as agent input; do not add the previous summary again.
    # If still over your allowed budget, apply an explicit host policy before executing.
    response = await agent.run_async(result.messages)
```

Pass the full, append-only history every time, together with the last successful
CompactionSnapshot (serializable with model_dump()). The snapshot stores summary,
covered item count and a digest of the covered prefix. Changed/truncated history or a
snapshot from another history is rejected; the host must reset/rebuild it explicitly.
Already covered messages are replaced by the existing summary before checking the
threshold, and only new complete turns enter the next incremental summary request.
Persist the snapshot only for the matching history revision; concurrent turns require
host-side synchronization. The original history is never mutated or deleted.

System/developer messages are preserved verbatim, including those inside the covered
prefix. Summaries are injected as user-role historical data, never elevated to privileged
instructions. Recent user turns are kept verbatim; pending calls and parallel tool-call
blocks are never split. Supply tool history as SessionItem function_call/output (or
handoff_call/output) pairs with unique call IDs. Orphan/duplicate outputs are rejected;
legacy Message(role="tool") records require explicit conversion by the host.

A single bounded batch is compacted per call. Large turns are not truncated to fit the
summarizer. Results report unchanged, compacted, blocked or failed, sizes, duration,
usage and a safe reason. A compacted result can still exceed the threshold: the host
must check after_size and choose whether to compact again, defer or reject execution.
Supply a deterministic, monotonic measure=your_token_counter and thresholds in matching units for token budgets;
max_summary_bytes remains a byte cap. Empty, oversized, or non-reducing summaries never
replace the old snapshot. Provider failures retain the previous effective context and
unsummarized messages; cancellation and RunTimeoutError propagate. Usage from completed
summarizer responses is reported even if the summary is rejected. Failed provider calls
may incur usage unavailable to the library. Summarization is lossy: critical authorization
and transaction state must always be reloaded from authoritative application data.

The compactor does not change provider_state or server-managed conversations. Hosts
must rebuild/clear incompatible provider state when replacing raw history with a summary;
otherwise old history can be sent twice. Compaction does not enable automatic persistence,
age filtering, permissions or TMS-specific retention rules. Existing SessionSnapshot.compact
remains a separate legacy item-trimming utility and is not automatically replaced.
