# `ai.orchestrator` — Module Overview

> This document is partially auto-generated. Sections tagged `<!-- AUTO:id -->` are refreshed by the generator.
> Everything else is yours to edit freely and will never be overwritten.

<!-- AUTO:meta -->
## About This Document

This file is **partially auto-generated**. Sections wrapped in `<!-- AUTO:id -->` tags
are overwritten each time the generator runs. Everything else is yours to edit freely.

| Field | Value |
|-------|-------|
| Module | `matrx_ai/orchestrator` |
| Last generated | 2026-04-07 11:38 |
| Output file | `matrx_ai/orchestrator/MODULE_README.md` |
| Signature mode | `signatures` |

**To refresh auto-sections:**
```bash
python utils/code_context/generate_module_readme.py matrx_ai/orchestrator --mode signatures
```

**To add permanent notes:** Write anywhere outside the `<!-- AUTO:... -->` blocks.
<!-- /AUTO:meta -->

<!-- HUMAN-EDITABLE: This section is yours. Agents & Humans can edit this section freely — it will not be overwritten. -->

## Terminal-output invariant

A provider-level `finish_reason=stop` is not sufficient for a successful user
turn. A tool-less terminal response must contain non-whitespace answer text or
another user-visible typed output block. Reasoning-only responses are persisted
as failed and emit `empty_assistant_response`; reasoning that contains literal
`<tool_call>` / `<function=...>` markup emits `unparsed_tool_call`. This keeps
hidden reasoning from becoming a silent UI stop.

## Architecture

> **Fill this in.** Describe the execution flow and layer map for this module.
> See `utils/code_context/MODULE_README_SPEC.md` for the recommended format.
>
> Suggested structure:
>
> ### Layers
> | File | Role |
> |------|------|
> | `entry.py` | Public entry point — receives requests, returns results |
> | `engine.py` | Core dispatch logic |
> | `models.py` | Shared data types |
>
> ### Call Flow (happy path)
> ```
> entry_function() → engine.dispatch() → implementation()
> ```


<!-- AUTO:tree -->
## Directory Tree

> Auto-generated. 8 files across 1 directories.

```
matrx_ai/orchestrator/
├── MODULE_README.md
├── __init__.py
├── concurrent_engine.py
├── executor.py
├── parallel_executor.py
├── recovery_logic.py
├── requests.py
├── tracking.py
# excluded: 1 .md
```
<!-- /AUTO:tree -->

<!-- AUTO:signatures -->
## API Signatures

> Auto-generated via `output_mode="signatures"`. ~5-10% token cost vs full source.
> For full source, open the individual files directly.

```
---
Filepath: matrx_ai/orchestrator/concurrent_engine.py  [python]

  OnProgress = Callable[[int, int, 'ItemOutcome[Any, Any]'], Awaitable[None]]
  class WarmupStrategy(StrEnum):
  class EngineConfig(BaseModel):
      # fields: initial_concurrency: int = 10, max_concurrency: int | None = None, min_concurrency: int = 1, max_retries: int = 3, base_backoff: float = 2.0, max_backoff: float = 60.0, jitter: float = 1.0, rate_limit_cooldown: float = 30.0, window_size: int = 20, error_rate_threshold: float = 0.3, recovery_streak: int = 10, warmup: WarmupStrategy = WarmupStrategy.NONE, retry_sweep: bool = True, retry_sweep_concurrency: int = 2, retry_sweep_delay: float = 3.0
      def _set_defaults(self) -> EngineConfig
  class ItemOutcome(BaseModel):
      # fields: item: Any, result: Any | None = None, error: str | None = None, attempts: int = 1, elapsed: float = 0.0
      def success(self) -> bool
  class BatchResult(BaseModel):
      # fields: succeeded: list[ItemOutcome[T, R]] = list(), failed: list[ItemOutcome[T, R]] = list(), total: int = 0, elapsed_seconds: float = 0.0, total_retries: int = 0, rate_limit_events: int = 0, concurrency_changes: list[tuple[float, int]] = list(), was_cancelled: bool = False, retry_sweep_recovered: int = 0
      def succeeded_count(self) -> int
      def failed_count(self) -> int
      def failure_summary(self) -> list[dict[str, Any]]
      def print_report(self, *, label: str = 'Batch Engine Report') -> None
  class _AIMDController:
      def __init__(self, config: EngineConfig, start_time: float) -> None
      def _record_change(self, new_level: int) -> None
      def on_success(self) -> None
      def on_retryable_failure(self) -> None
      def on_rate_limit(self) -> None
  class _PendingItem:
      def __init__(self, item: T, attempt: int = 0) -> None
  class WorkerResultError(Exception):
      def __init__(self, message: str, *, result: Any = None) -> None
  class _WorkerResult:
      def __init__(self, item: T, *, result: R | None = None, exception: Exception | None = None, elapsed: float = 0.0, attempt: int = 0) -> None
      def success(self) -> bool
      def to_outcome(self) -> ItemOutcome[T, R]
  class ConcurrentEngine:
      def __init__(self, config: EngineConfig | None = None, on_progress: OnProgress | None = None, is_retryable: Callable[[Exception], bool] | None = None, is_result_success: Callable[[Any], bool] | None = None) -> None
      async def run(self, items: list[T], worker: Callable[[T], Awaitable[R]], cancel_event: asyncio.Event | None = None) -> BatchResult[T, R]
      async def _call_worker(self, item: T, worker: Callable[[T], Awaitable[R]], *, attempt: int) -> _WorkerResult[T, R]
  def _default_is_retryable(exc: Exception) -> bool
  def _is_rate_limit(exc: Exception) -> bool
  def _build_result() -> BatchResult[T, R]
  async def _notify_progress(outcome: ItemOutcome[T, R]) -> None
  async def _acquire_slot() -> None
  async def _release_slot() -> None
  async def _worker_loop() -> None
  async def _sweep_one(candidate: ItemOutcome[T, R]) -> None


---
Filepath: matrx_ai/orchestrator/__init__.py  [python]



---
Filepath: matrx_ai/orchestrator/recovery_logic.py  [python]

  def handle_finish_reason(response: UnifiedResponse, request: AIMatrixRequest, retry_attempt: int, max_retries: int, debug: bool = False) -> Literal['continue', 'truncated', 'retry', 'stop']


---
Filepath: matrx_ai/orchestrator/parallel_executor.py  [python]

  class ParallelResult:
      def success(self) -> bool
      def output(self) -> str | None
  async def parallel_execute(configs: list[UnifiedConfig], *, concurrency: int = 5, rate_limit_delay: float = 0.0, metadata: dict[str, Any] | None = None) -> list[ParallelResult]
  async def _run_one(idx: int, config: UnifiedConfig) -> None


---
Filepath: matrx_ai/orchestrator/requests.py  [python]

  class AIMatrixRequest:
      def __post_init__(self)
      def user_id(self) -> str
      def emitter(self) -> Emitter | None
      def total_usage(self) -> AggregatedUsage
      def timing_stats(self) -> dict[str, Any]
      def tool_call_stats(self) -> dict[str, Any]
      def add_usage(self, usage: TokenUsage | None) -> None
      def add_timing(self, timing: TimingUsage | None) -> None
      def add_tool_calls(self, tool_calls: ToolCallUsage | None) -> None
      def from_dict(cls, data: dict[str, Any], emitter: Emitter | None = None) -> AIMatrixRequest
      def add_response(cls, original_request: AIMatrixRequest, response: UnifiedResponse, tool_results: list[ToolResultContent] | None = None) -> AIMatrixRequest
      def to_dict(self) -> dict[str, Any]
  class CompletedRequest:
      def conversation_id(self) -> str
      def user_id(self) -> str
      def messages(self) -> MessageList
      def to_dict(self) -> dict[str, Any]
      def to_storage_dict(self) -> dict[str, Any]


---
Filepath: matrx_ai/orchestrator/tracking.py  [python]

  class ToolCallUsage:
      def aggregate(tool_call_list: list[ToolCallUsage]) -> dict[str, Any]
  class TimingUsage:
      def total_duration(self) -> float
      def processing_duration(self) -> float
      def aggregate(timings: list[TimingUsage]) -> dict[str, Any]


---
Filepath: matrx_ai/orchestrator/executor.py  [python]

  LOCAL_DEBUG = False
  async def execute_ai_request(config: UnifiedConfig, max_iterations: int = 20, max_retries_per_iteration: int = 2, metadata: dict[str, Any] | None = None) -> CompletedRequest
  async def handle_tool_calls(response: UnifiedResponse, request: AIMatrixRequest, iteration: int) -> tuple[list | None, ToolCallUsage | None, list[TokenUsage]]
  async def _finalize_and_persist(current_request: AIMatrixRequest, iteration: int, final_response: UnifiedResponse, metadata: dict[str, Any], trigger_position: int, pre_execution_message_count: int, conversation_id: str | None = None, debug: bool = False) -> CompletedRequest
  def _schedule_labeling_if_new(exec_ctx: Any, config: UnifiedConfig) -> None
  async def execute_until_complete(initial_request: AIMatrixRequest, client: UnifiedAIClient, max_iterations: int = 20, max_retries_per_iteration: int = 2) -> CompletedRequest
```
<!-- /AUTO:signatures -->

<!-- AUTO:call_graph -->
## Call Graph

> Auto-generated. Scoped to: `handle_tool_calls, executor, registry, guardrails`.
> Shows which functions call which. `async` prefix = caller is an async function.
> Method calls shown as `receiver.method()`. Private methods (`_`) excluded by default.

```
Function Call Graphs

# Call graph: ai.orchestrator.executor
  async ai.orchestrator.executor.execute_ai_request → ai.orchestrator.executor.get_app_context() (line 67)
  async ai.orchestrator.executor.execute_ai_request → ai.orchestrator.executor.AIMatrixRequest() (line 73)
  async ai.orchestrator.executor.execute_ai_request → ai.orchestrator.executor.execute_until_complete(request, UnifiedAIClient(), max_iterations, max_retries_per_iteration) (line 81)
  async ai.orchestrator.executor.execute_ai_request → ai.orchestrator.executor.UnifiedAIClient() (line 83)
  async ai.orchestrator.executor.handle_tool_calls → tool_calls.append(content) (line 106)
  async ai.orchestrator.executor.handle_tool_calls → ai.orchestrator.executor.handle_tool_calls_v2(raw_calls) (line 122)
  async ai.orchestrator.executor.handle_tool_calls → ai.orchestrator.executor.ToolResultContent() (line 127)
  async ai.orchestrator.executor.handle_tool_calls → cr.get('is_error', False) (line 139)
  async ai.orchestrator.executor.handle_tool_calls → ai.orchestrator.executor.ToolCallUsage() (line 144)
  async ai.orchestrator.executor._finalize_and_persist → ai.orchestrator.executor.CompletedRequest() (line 188)
  async ai.orchestrator.executor._finalize_and_persist → metadata.get('status') (line 207)
  async ai.orchestrator.executor._finalize_and_persist → ai.orchestrator.executor.update_conversation_status(conv_id, 'error') (line 209)
  async ai.orchestrator.executor._finalize_and_persist → ai.orchestrator.executor.update_user_request_status(req_id, 'failed') (line 211)
  async ai.orchestrator.executor._finalize_and_persist → metadata.get('error') (line 214)
  async ai.orchestrator.executor._finalize_and_persist → ai.orchestrator.executor.persist_completed_request(completed) (line 219)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.get_app_context() (line 254)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.ensure_conversation_exists() (line 262)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.create_pending_user_request() (line 268)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.TimingUsage() (line 283)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 283)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 299)
  async ai.orchestrator.executor.execute_until_complete → client.execute(current_request) (line 300)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 301)
  async ai.orchestrator.executor.execute_until_complete → current_request.add_usage(response.usage) (line 309)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.handle_finish_reason(response, current_request, retry_attempt, max_retries_per_iteration, debug) (line 312)
  async ai.orchestrator.executor.execute_until_complete → ...send_status_update() (line 326)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor._finalize_and_persist() (line 342)
  async ai.orchestrator.executor.execute_until_complete → ...lower() (line 374)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.classify_provider_error(provider, e) (line 382)
  async ai.orchestrator.executor.execute_until_complete → error_info.get_backoff_delay(retry_attempt) (line 389)
  async ai.orchestrator.executor.execute_until_complete → ...send_status_update() (line 403)
  async ai.orchestrator.executor.execute_until_complete → asyncio.sleep(backoff_delay) (line 415)
  async ai.orchestrator.executor.execute_until_complete → ...send_error() (line 433)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 454)
  async ai.orchestrator.executor.execute_until_complete → current_request.add_timing(current_timing) (line 455)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor._finalize_and_persist() (line 465)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.UnifiedResponse() (line 468)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 483)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.handle_tool_calls(response, current_request, iteration) (line 484)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 487)
  async ai.orchestrator.executor.execute_until_complete → current_request.add_usage(child_usage) (line 491)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 494)
  async ai.orchestrator.executor.execute_until_complete → current_request.add_timing(current_timing) (line 495)
  async ai.orchestrator.executor.execute_until_complete → current_request.add_tool_calls(tool_call_usage) (line 498)
  async ai.orchestrator.executor.execute_until_complete → AIMatrixRequest.add_response() (line 501)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor._finalize_and_persist() (line 534)
  async ai.orchestrator.executor.execute_until_complete → time.time() (line 559)
  async ai.orchestrator.executor.execute_until_complete → current_request.add_timing(current_timing) (line 560)
  async ai.orchestrator.executor.execute_until_complete → traceback.print_exc() (line 567)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor._finalize_and_persist() (line 587)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.UnifiedResponse() (line 590)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor._finalize_and_persist() (line 629)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.locals() (line 633)
  async ai.orchestrator.executor.execute_until_complete → ai.orchestrator.executor.UnifiedResponse() (line 634)
```
<!-- /AUTO:call_graph -->

<!-- AUTO:dependencies -->
## Dependencies

**External packages:** matrx_connect, matrx_utils, pydantic, rich
**Internal modules:** matrx_ai.agents, matrx_ai.config, matrx_ai.context, matrx_ai.db, matrx_ai.providers, matrx_ai.tools
<!-- /AUTO:dependencies -->

<!-- AUTO:config -->
## Generation Config

> Auto-managed. Contains the exact parameters used to generate this README.
> Used by parent modules to auto-refresh this file when it is stale.
> Do not edit manually — changes will be overwritten on the next run.

```json
{
  "subdirectory": "matrx_ai/orchestrator",
  "mode": "signatures",
  "scope": null,
  "project_noise": null,
  "include_call_graph": false,
  "entry_points": null,
  "call_graph_exclude": [
    "tests"
  ],
  "signatures_exclude": [
    "tests",
    "db",
    "local_data",
    "docs"
  ]
}
```
<!-- /AUTO:config -->

<!-- AUTO:callers -->
## Upstream Callers

> Auto-discovered by scanning all project files that import from this module.
> Set `ENTRY_POINTS` in `generate_readme.py` to pin specific functions.

| Caller | Calls |
|--------|-------|
| `matrx_ai/__init__.py` | `AIMatrixRequest()` |
| `matrx_ai/providers/unified_client.py` | `AIMatrixRequest()` |
| `matrx_ai/tests/prompts/prompt_to_config.py` | `AIMatrixRequest()` |
| `matrx_ai/tests/test_translations.py` | `AIMatrixRequest()` |
| `matrx_ai/agents/services/batch_categorize.py` | `BatchResult()` |
| `matrx_ai/__init__.py` | `CompletedRequest()` |
| `matrx_ai/agents/definition.py` | `CompletedRequest()` |
| `matrx_ai/db/persistence.py` | `CompletedRequest()` |
| `matrx_ai/agents/services/batch_categorize.py` | `ConcurrentEngine()` |
| `matrx_ai/agents/services/batch_categorize.py` | `EngineConfig()` |
| `matrx_ai/agents/services/batch_categorize.py` | `ItemOutcome()` |
| `matrx_ai/__init__.py` | `TimingUsage()` |
| `matrx_ai/__init__.py` | `ToolCallUsage()` |
| `matrx_ai/agents/services/batch_categorize.py` | `WarmupStrategy()` |
| `matrx_ai/agents/tests/test_categorization.py` | `WarmupStrategy()` |
| `matrx_ai/agents/definition.py` | `execute_ai_request()` |
| `matrx_ai/tests/execution_test.py` | `execute_ai_request()` |
| `matrx_ai/__init__.py` | `execute_until_complete()` |
| `matrx_ai/tests/prompts/agent_comparison.py` | `execute_until_complete()` |
| `matrx_ai/tests/prompts/prompt_to_config.py` | `execute_until_complete()` |
<!-- /AUTO:callers -->
