Metadata-Version: 2.4
Name: aid-runtime-python
Version: 0.1.70
Summary: The AID Framework Python runtime: model gateway with pinning and cost budgets, prompt registry, tool executor, OTel GenAI tracing, plain-async pipeline, and an eval harness that gates on statistical regression against a stored baseline.
Author: The AID Framework authors
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/aid-framework/aid-framework
Project-URL: Documentation, https://github.com/aid-framework/aid-framework/blob/main/docs/design.md
Project-URL: Issues, https://github.com/aid-framework/aid-framework/issues
Keywords: ai,codegen,llm,evals,opentelemetry,pydantic
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: httpx<1,>=0.27
Requires-Dist: opentelemetry-api<2,>=1.24
Requires-Dist: pydantic<3,>=2.7
Requires-Dist: pydantic-ai-slim<3,>=2
Provides-Extra: pgvector
Requires-Dist: psycopg[binary]<4,>=3.1; extra == "pgvector"
Provides-Extra: deepagents
Requires-Dist: deepagents==0.7.32; extra == "deepagents"
Provides-Extra: dev
Requires-Dist: mypy<3,>=1.11; extra == "dev"
Requires-Dist: opentelemetry-sdk<2,>=1.24; extra == "dev"
Requires-Dist: pytest<10,>=8; extra == "dev"
Requires-Dist: pytest-asyncio<2,>=0.24; extra == "dev"
Requires-Dist: ruff<1,>=0.6; extra == "dev"

# `aid-runtime-python`

The Python runtime for [AID Framework](https://github.com/aid-framework/aid-framework): the behavior layer that
apps generated by the [`py-fastapi`](https://github.com/aid-framework/aid-framework/tree/main/packages/generators/py-fastapi)
target import. Generated code is thin glue; this is where the behavior lives.

Distribution name **`aid-runtime-python`**, import package **`aid_runtime`**. The two differ deliberately: a
distribution name is conventionally hyphenated, and a Python import package cannot be. That asymmetry is asserted
by the test suite so it cannot drift.

## Install

```sh
pip install aid-runtime-python
```

Requires Python 3.11 or newer. Dependencies are `httpx`, `pydantic`, `pydantic-ai-slim` and
`opentelemetry-api`.

Two optional extras:

```sh
pip install "aid-runtime-python[pgvector]"   # the pgvector store adapter
pip install "aid-runtime-python[deepagents]" # the deepagents agent engine, pinned
```

## Quick start

```python
from aid_runtime import (
    Budget,
    CompletionRequest,
    CostProfile,
    ModelGateway,
    ModelSpec,
    OpenAICompatibleProvider,
    PinnedModel,
)

spec = ModelSpec(
    name="gpt-4o-2024-08-06",
    provider="openai-compatible",
    cost=CostProfile(input_per_1k_usd=0.0025, output_per_1k_usd=0.01),
    context_window=128_000,
    max_output_tokens=4_096,
)

gateway = ModelGateway(
    providers={"openai-compatible": OpenAICompatibleProvider(api_key=...)},
    aliases={"small": PinnedModel(alias="small", spec=spec)},
    budget=Budget(max_usd=5.0, scope="nightly"),
)

result = await gateway.generate("small", CompletionRequest.prompt("hello"))

result.text, result.usage, result.cost_usd, result.alias, result.attempts, result.fell_back
```

## What it provides

| Module | Provides |
| --- | --- |
| `gateway.py` | `ModelGateway` — alias resolution, capability negotiation, ordered fallback, cost accounting, GenAI spans |
| `providers/` | The `Provider` protocol, one HTTP adapter (`OpenAICompatibleProvider`), one deterministic fake (`FakeProvider`) |
| `prompts.py` | `PromptTemplate` / `PromptRegistry` — versioned prompts, typed rendering, few-shot assembly |
| `structured.py` | `build_agent` — structured output via Pydantic AI only |
| `tools.py` | `Tool` / `ToolExecutor` — declared definitions, argument validation, structured results |
| `pipeline.py` | `Pipeline` / `PipelineContext` — plain-async named steps over a frozen context |
| `agents/` | `AgentRuntime` — the loop, transfer vs. delegate, step/token/cost budgets with bounded degrade, memory, checkpointed human-in-the-loop |
| `retrieval/` | Connectors, chunkers, the `Embedder` / `Reranker` / `VectorStore` protocols, hybrid fusion, citations, retrieval metrics |
| `evals/` | Datasets, metrics, and the statistical baseline gate |
| `tracing.py` | OTel GenAI spans: `llm_span`, `agent_span`, `record_usage`, `record_cost`, `record_attributes`, `record_hitl`, `record_exception` |
| `cost.py` | `Budget`, `CostLedger`, `estimate_cost_usd`, `estimate_tokens` |
| `types.py` | The request/response/usage/spec models everything else is typed against |

194 names are exported. The layering is acyclic and one-directional: `errors` and `types` import nothing else in
the package, `gateway` is imported only by the package `__init__` and the eval runner, and no core module imports
a higher layer back.

## Model pinning

An **alias resolves to an exact provider model id**, including a pinned version. That is not cosmetic: a baseline
recorded against `gpt-4o` means nothing if the alias floats to `gpt-4o-2026-…` on the next run, so an **unknown
alias is an error** (`UnknownModelAliasError`) rather than a silent fall-through to a provider default.

- `resolve(alias)` returns the `PinnedModel` (alias + `spec`); `provider_for(alias)` its provider.
- `require(alias, capabilities)` asserts the capabilities *and* returns the pinned model, checking **both** sides —
  a model that supports tool calling behind an adapter that does not serialise tool definitions cannot tool-call —
  and raises `CapabilityMismatchError` naming what is missing.
- `generate(alias, request, *, fallbacks=(), budget=None)` tries the alias then each fallback **in the order
  given**, and reports the aliases actually attempted. Fallbacks are ordered, not conditional.
- Every charged call opens a span and updates the ledger. `CostLedger` tracks per-call `LedgerEntry` records and
  yields a `SpendSummary`; `Budget` (`max_usd`, `max_total_tokens`, `scope`) is enforced *before* the call, and
  `estimate_usage` / `estimate_cost_usd` price a request from a model's `CostProfile` when a caller wants to check
  before spending.

## The eval gate measures statistics, not equality

A model is **expected** to vary between runs, so a metric that compares two runs for equality is either always
red or measuring nothing. Non-determinism is not a bug to be fixed here; it is the property the gate is built to
be robust against.

- `Dataset` — versioned cases with inputs, expectations, tags and metadata; `from_records` / `from_json` /
  `from_path`, `to_json`, and lookups by id (`case`) and by tag (`by_tag`).
- Metrics — `ExactMatch`, `NormalizedContains`, `JsonSchemaValid`, and `LlmJudge`. `metric_names` and
  `default_metrics` cover the common set.
- `run_evaluation(dataset, runner, metrics, runs=5)` samples the runner **N times** and produces an `EvalRun` of
  per-run metric means, so a single unlucky run does not decide the verdict. `evaluate_once` is the single-pass
  primitive; a `DatasetRun` gives per-case outcomes and a rendered `summary()`.
- `EvalBaseline.from_run(run, model=..., note=...)` stores a baseline as JSON: per metric the sample count, mean
  and spread. A committed example ships at `runtimes/python/tests/fixtures/capitals-baseline.json`.
- `BaselineGate(thresholds).evaluate(run, baseline)` compares the current sample against the baseline using a
  one-sided normal test. Each `Threshold` carries a `metric`, an optional absolute `min_mean`, a
  `max_regression` allowance, and an `alpha` significance level. A metric passes by clearing `min_mean`, by being
  within `max_regression` of the baseline, or by regressing only within sampling noise.

Two deliberate limits:

- **`LlmJudge` is implemented and never CI-required.** A judge is itself a model call, and a gate that depends on
  one non-deterministic call inside the gate you are trying to trust is the problem this design avoids.
  `gateway_judge(gateway, alias)` builds the judge call when a caller wants it.
- **`GateConfigurationError` refuses a threshold on a model-derived metric**, because an N-run sample of a
  probabilistic judge cannot carry a fixed threshold honestly.

`cost` is the enforced budget. A latency p95 budget belongs with dashboards, not here.

## Retrieval

`retrieval/` is an IR-driven layer: `binding_from_spec(retriever, embedding=..., store=...)` folds the IR's
`retrievers`, `embeddings` and `stores` entries into one `RetrieverBinding`, which a `Retriever` then ingests and
searches through. The runtime consumes the names the IR emits, field for field.

- Three protocols, in the same style as `providers.Provider`: `Embedder`, `Reranker`, `VectorStore`.
- `FakeDeterministicEmbedder` maps text to a stable unit vector with a hash — keyless and offline, which is what
  lets a retrieval gate run in CI. `OpenAICompatibleEmbedder` is the one hosted adapter: a single POST to
  `/embeddings` over `httpx`, with the retry, timeout and error-chaining behavior of the HTTP provider.
- `LexicalReranker` is the only reranker. It is deterministic and lexical; there is no hosted rerank adapter.
- Chunk ids are **content hashes**, so re-ingesting the same input produces the same ids and a no-op diff. Ingest
  is held to idempotence rather than to the byte-identical codegen guarantee.
- **Pinning.** `embed_model` names an IR `embeddings` entry and resolves at codegen time to an exact `model_id`
  plus its recorded width; the resolved id, width and store `kind` travel into the generated code. An alias is
  never resolved at runtime, because a baseline recorded against a floating model means nothing.
- **Cost.** Embedding calls charge the existing `CostLedger` through the existing `Budget`, using the catalog's
  `CostProfile` for the pinned model. There is no retrieval-specific budgeting machinery.

`RetrievalMetric` (`RecallAtK`, `MeanReciprocalRank`, `CitationGrounding`) is deliberately a **separate protocol**
from the shipped `Metric`, which scores a single string prediction. Retrieval scores a *retrieved set*:
recall@k and MRR need the ranked ids, and citation grounding needs the passages behind the citations. Retrieval is
therefore sampled through `RetrievalGate.sample(...)`, producing a standard `EvalRun`, and then judged by the
**unchanged** `BaselineGate`. Retrieval metrics are deterministic, so they pass the gate's own eligibility check.

### pgvector

`PgVectorStore` is a thin adapter over `psycopg` and the `vector` extension.

- It is **locally provisioned and not covered by CI.** No CI job starts a database; the in-memory adapter carries
  the store tests, and both adapters share the same `VectorStore` protocol and `similarity` semantics.
  `compose.pgvector.yml` in the repository brings up a database for local use.
- The `[pgvector]` extra is **import-safe when absent**: importing `aid_runtime` or `aid_runtime.retrieval` never
  imports the driver, and loading the adapter raises a `ConfigurationError` naming the extra.

## Control flow is plain async

`Pipeline` is a list of named async (or plain) steps over a `PipelineContext` that is **frozen**, so a step cannot
mutate what an earlier step observed — it returns a new context, which keeps a failed run's state inspectable. A
step may return a context, a mapping, or `None`. Each step runs in its own span, a step raising anything aborts
the run with the original exception chained (`PipelineError.__cause__`), and `StopPipeline` is the supported early
exit.

**Pydantic AI is used for structured output and nothing else** (`structured.build_agent`). The orchestration
library stays out of the control-flow path deliberately, so it remains swappable, and **no graph or orchestration
dependency is taken at all**: the agent loop in `agents/` is this package's own.

Tracing follows the OTel GenAI semantic conventions: `gen_ai.*` attributes for the operation, model, provider,
request parameters and usage, with AID-specific values namespaced `aid.*` so the two never collide. A failing call
records its exception exactly once.

## The agent runtime

`AgentRuntime.from_ir(document, providers=..., ...)` reads the emitted IR and runs an agent to a stop condition:
model call, tool calls, observations, until a terminal answer, `max_steps`, or a budget breach. Model calls go
through `ModelGateway`, so fallback, cost accounting and tracing apply without a second implementation.

**Transfer and delegation are different, and the API says which.** `_transfer` (`handoffs`) hands the conversation
to the target and never resumes the caller, so `RunResult.agent_id` is the target and `path` grows. `_delegate`
(`delegates`) runs the target in an isolated `[system, user]` context and returns its answer as a single tool
observation, so `RunResult.agent_id` is the caller and `path` is unchanged. Both refuse to re-enter an agent
already on the path.

**A budget breach fails hard.** `BudgetExceededError` stops the run; the gateway preflight and the ledger both
enforce the ceiling. An agent declaring `on_budget_exhausted: "degrade"` (spec v0.2) instead steps exactly one rung
down its declared alias ladder — no retry loop, and the build-time gate rejects the declaration when no strictly
cheaper alias exists. Each degraded step is recorded in `StepRecord.degraded_from`, and a checkpoint carries its
current alias so a resumed run re-routes from where it stopped.

**Memory** is in-process plus one store-backed adapter. `thread` scope is the checkpointer unit: trimmed to its
`token_budget`, never written to a store. `user` and `global` scopes read and write the store and render as one
extra system message, so injection is visible in the request rather than hidden.

**Human-in-the-loop** pauses by returning a checkpoint (completed steps, pending call, current alias), and
`resume(checkpoint, decision)` continues it. The pause is recorded as an `aid.hitl` span event, not an attribute,
because a run can pause more than once.

## Limits, stated plainly

These are refused explicitly rather than approximated — each raises `FeatureDeferredError` or a
`ConfigurationError` naming the reason:

- `ModelGateway.stream` — the transport is request/response only, and no provider advertises
  `Capability.STREAMING`.
- Retrieval: non-file sources, `semantic` chunking, `incremental` refresh, retriever `filters`, a hosted
  reranker, and the `qdrant` / `pinecone` / `weaviate` / `custom` store kinds.
- Escalation routing and third-party orchestration frameworks.

## Development

From a checkout of the repository:

```sh
pip install -e ".[dev]"
pytest
ruff check .
ruff format --check .
mypy
```

`mypy` runs strict over `src` **and** `tests`, so the tests are type-checked too.

**Non-responsibility:** no app-specific prompts, tools, or rubrics. Those belong to the generated app's
`business/` tier — the seam this runtime is called from.

## See also

- [`@aid-framework/runtime`](https://www.npmjs.com/package/@aid-framework/runtime) — the same runtime in
  TypeScript, module for module
- [`@aid-framework/generator-py-fastapi`](https://www.npmjs.com/package/@aid-framework/generator-py-fastapi) — the
  generator that emits apps against this runtime

## License

[Apache-2.0](https://github.com/aid-framework/aid-framework/blob/main/LICENSE). Code generated by AID Framework is
not subject to this license.
