Metadata-Version: 2.4
Name: langchaint
Version: 0.5.0
Summary: It ain't langchain.
Keywords: llm,anthropic,openai,claude,async
Author: John Hopfensperger
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Dist: jsonschema>=4.25
Requires-Dist: pydantic>=2.13.4
Requires-Python: >=3.13
Project-URL: Homepage, https://github.com/s-banach/langchaint
Project-URL: Repository, https://github.com/s-banach/langchaint
Project-URL: Issues, https://github.com/s-banach/langchaint/issues
Description-Content-Type: text/markdown

# langchaint

Provider-neutral async LLM client over the official provider SDKs (anthropic >= 0.116.0, openai >= 2.45.0).
Alpha: the API is unstable and may change without notice.

## Design

**Generation only via binding.**
`LLM` has no generate methods.
`LLM.bind(...) -> BoundLLM[OutputT]` freezes everything that determines the cacheable prompt prefix: `system_prompt`, `tool_manager`, `inference_params`, `tool_choice`, `parallel_tool_calls`, `automatic_prompt_caching`.
`response_format` is frozen too, as the field that fixes the output type.
`automatic_prompt_caching` has no default: caching changes billing, so every `bind` states it.
A `TextPart` or `ImagePart` with `cache_breakpoint=True` places a prompt-cache boundary at exactly that part, under either `automatic_prompt_caching` value, so binding `False` and marking parts is the fully user-specified caching configuration.
`generate_many(conversations, warm_cache=True)` runs the first conversation to completion before admitting the rest, so a batch sharing a cached prefix pays one cache write instead of one per in-flight item; it is opt-in because it costs one item of serial latency.
`system_prompt` also binds as a sequence of `TextPart`s, so a boundary can sit inside the frozen prefix (stable instructions marked, semi-stable context after).
There are no per-call parameter overrides; changing parameters is `rebind(...)`, which returns a new `BoundLLM` with the SDK keyword arguments converted again.
`bind(response_format=Model)` returns `BoundLLM[Model]`; without `response_format` it returns `BoundLLM[str]`, selected by overload.
`rebind` carries the same overload, so `rebind(response_format=Model)` switches the output type to `BoundLLM[Model]`, `rebind(response_format=None)` switches it back to `BoundLLM[str]`, and leaving it out keeps the current type.
Every generate and stream method takes a conversation of `Message`s; a bare `str` is accepted as shorthand for a conversation of one `UserMessage` holding that text.

**The catalog is a constructor function per backend returning a ready LLM (anthropic adds a second for Bedrock).**
`from langchaint.openai import openai_model` then `openai_model("gpt-5.6-terra")`, and `from langchaint.anthropic import anthropic_model` then `anthropic_model("claude-sonnet-5")`, take the provider's own model identifier (a `Literal`, so typos fail the type check), look up the public prices, construct the adapter, and wrap it in an `LLM`.
`client=None` constructs the native first-party SDK client from environment credentials.
Anthropic Bedrock is a second sibling constructor, `anthropic_bedrock_model(model, aws_region=...)`: it names the same catalog model and reads the model's Bedrock surface (which of two SDK client classes) and wire model id from a table, so the application names neither the client class nor the Bedrock id.
Constructing an adapter directly covers models outside the catalog.

**The SDKs are optional dependencies, pinned directly by the application.**
The neutral core (`LLM`, the message tree, the error taxonomy) imports no SDK, so `import langchaint` needs neither package.
Each backend lives in its own subpackage that imports its SDK at module top: install `openai` or `anthropic` (pinning the SDK variant you need, such as `anthropic[bedrock]`) for the ones you use, and importing `langchaint.openai` without the openai package raises a `ModuleNotFoundError` naming the package to install. langchaint declares no extras of its own.
The import path is the boundary: only code that reaches for a backend requires its SDK, and a type checker following `langchaint.openai` never resolves anthropic's types.
OTel tracing follows the same pattern: `langchaint.tracing` imports only opentelemetry-api, install `opentelemetry-api`, and it stays off `import langchaint`.

**Adapters wrap SDK clients and delegate to the SDK.**
`AnthropicMessagesProvider(client=AsyncAnthropic(...))` and `OpenAIResponsesProvider(client=AsyncOpenAI(...))` call `messages.create/parse/stream` and `responses.create/parse/stream`; stream assembly and structured-output parsing are the SDK's, not hand-written.
Adapters store a `with_options(max_retries=0)` copy of the client, so the SDK never retries beneath the package's retry loop.
OpenAI support is the Responses API only: every supported OpenAI model speaks it, so there is no Chat Completions adapter.
The adapter always sends `store=False` because conversation state is the caller's conversation argument.
Anthropic Bedrock is two distinct SDK surfaces, the legacy `InvokeModel` client `AsyncAnthropicBedrock` and the Messages-API client `AsyncAnthropicBedrockMantle`, and the catalog models split across them, so `anthropic_bedrock_model` reads the client class and wire model id per model from `ANTHROPIC_BEDROCK`.
OpenAI on Bedrock is the bundled `AsyncBedrockOpenAI` passed as `client`; there is no Converse adapter.
Requests travel as typed frozen dataclasses whose optional fields are `X | Omit`, passed to the SDK as explicit keywords: no `**kwargs`, no hand-written wire TypedDicts, and the SDK overloads resolve without casts.

**Anthropic prompt caching is placed by the adapter.**
A bind-time `cache_control` marker goes on the system block (or the last tool when there is no system prompt), and a per-request marker goes on the last block of the last message, so the breakpoint follows a growing conversation.
Placement is manual because `messages.parse` lacks the top-level `cache_control` parameter; manual placement keeps create/parse/stream uniform.
Every marker's TTL is the adapter's `cache_ttl` (`"5m"` default, `"1h"` for entries that must survive longer gaps at 2x write cost), a keyword of `anthropic_model` and `anthropic_bedrock_model`.

**Usage counters partition, and it carries the adapter-priced cost.**
`Usage` stores `input_tokens_cache_read`, `input_tokens_cache_write`, `input_tokens_cache_none`, `output_tokens`, and `output_tokens_reasoning` (the reasoning share of output tokens); the three input counters are a disjoint partition and `input_tokens_total` is derived.
The counters are validated non-negative, which is what guards the openai path, where `input_tokens_cache_none` is derived by subtraction from the provider's all-inclusive `input_tokens`.
`Usage` also carries `cost_in_usd`, computed inside the adapter from raw provider counts against a `PricingTable`: it is stored, never derived from the counters, because providers split counters the normalized `Usage` collapses (anthropic bills 5-minute and 1-hour cache writes at different rates, and an adapter that sees 1-hour writes without `cache_write_1h_usd_per_million_tokens` raises `AbortBatchError` rather than misbill). Cost riding inside `Usage` means the two are born together and one `sum()` folds both; total across several results with `Usage.sum_of(...)`.
Provider-specific detail the partition drops (the 5m/1h write split, `server_tool_use`, `service_tier`) stays readable on `usage_raw`, the raw SDK usage object carried by reference beside every `Usage`.
For a per-category cost split, each backend has `cost_breakdown(usage_raw, pricing)` (`langchaint.anthropic.cost_breakdown` / `langchaint.openai.cost_breakdown`): it extracts the counts from the raw SDK usage (only there does the 5m/1h write split survive) and prices them through the same neutral `price` call that produced the stored `cost_in_usd`, so the returned `CostBreakdown`'s `total_cost_in_usd` equals it.
`CostBreakdown` carries one cost per category, the derived `input_tokens_cost_in_usd` and `total_cost_in_usd`, and the priced `counts`, from which an application computes its own caching counterfactual against whatever baseline it chooses; cache writes bill above the uncached rate (1.25x at 5 minutes, 2x at 1 hour), so savings are negative on the turn that writes the cache and land on later read turns, meaning any savings number is only meaningful aggregated over a conversation.

**Success is a `Response`, failure is a `GenerationError`.**
A generate that succeeds returns a frozen `Response[OutputT]` with every field present; one that ends terminally raises (or, in a batch, returns) a `GenerationError`.
Its three leaves are the terminal per-item outcomes: `RetriesExhaustedError` (transient budget spent), `RefusalError` (the model refused on the structured path), and `MaxCompletionTokensExceededError` (the structured response hit the token cap before its JSON parsed).
On the base, `stop_reason` is `StopReason | None`: `"refusal"` or `"max_tokens"` on those two leaves, `None` on `RetriesExhaustedError`, whose attempts never reached a completed turn.
Both hold `attempt_records`, one `AttemptRecord` per request sent: raw `time.monotonic()` start and end readings bracketing the send only (slot waits and backoff sleeps excluded, so rate limiting is distinguishable from slow requests), the attempt's `TransientError` (None on the attempt that succeeded or a rejected 200), the attempt's `usage` (carrying `cost_in_usd`; `ZERO_USAGE` for a transport failure that billed nothing), and `usage_raw` (the raw SDK usage, None when no wire payload existed).
On a `Response` every record but the last failed; on a `GenerationError` the records describe the terminal outcome.
`attempts` is derived from the records on both.
Both also carry `model`, `provider_name`, and `elapsed_seconds` (first request to completion, waits included, so stored rather than derived), so the module-level `to_row(result)` flattens either to the same scalar keys and a mixed list of successes and failures is one table.
`usage` is the paid total across every attempt, folded from the records, on both `Response` and `GenerationError`: it is the number to bill on (its `cost_in_usd` is the money the call spent), so a refusal or truncation reports its real cost while a retry-exhausted item whose attempts billed nothing reports zero; `error_text` carries the failure reason.
On a `Response`, `stop_reason`, `assistant_message`, and `raw` are always present; `usage_successful_attempt` is the single kept answer's own usage (the one matching `output` and `raw.usage`), equal to `usage` unless a billed 200 was retried (an empty structured parse retried as transient), in which case `usage` is larger by the retried attempt's billing.
`raw` is the SDK's own response model held by reference; it is never dumped to a dict, because dumping deep-copies every response body per request.

**One RateLimiter owns retrying and pacing.**
`RateLimiter` holds `max_attempts`, `backoff_base_seconds`, `backoff_max_seconds`, and `max_in_flight`; it is stateful and shareable, so one instance passed to several `LLM`s is one shared budget for the account they hit.
Its slot gates every request start on every path (first attempts, retries, batch items, stream openings); backoff sleeps outside the slot.
There is deliberately no `requests_per_minute`: an in-flight bound self-adjusts throughput along request duration, while a client-side rate number models one dimension of the provider's multi-dimensional limit and goes stale with the account tier.
A rate-limit error (`Provider.classify` returning `"rate_limit"`, or any error naming a server-stated retry-after) pauses admission for everyone sharing the limiter: for the server-stated wait when one was sent (parsed from the `retry-after-ms` / `retry-after` headers onto `TransientError.retry_after_seconds`, capped at 60 seconds), else for the failing task's backoff delay.
After the pause, admission stays limited to one probe request at a time until the probe succeeds; other transient errors (timeouts, 5xx) pause nobody, because they say nothing about the account's quota.

**Retry stays in the package, classification in the adapter.**
Adapters raise `TransientError`, `AbortBatchError`, or a `GenerationError` leaf directly where they know the answer: rate limits and overload (transient), a bad request (abort), and an empty structured parse, which splits three ways (a refusal is `RefusalError`, a token-cap truncation is `MaxCompletionTokensExceededError`, anything else is `TransientError`).
The retry loop retries only `TransientError`; it honors the rest without asking.
Unrecognized exceptions go through `Provider.classify`, which returns `"rate_limit"`, `"transient"`, or `"abort"`, and anything it does not recognize is abort.
`generate_one` raises `RetriesExhaustedError` for transient exhaustion, `RefusalError` or `MaxCompletionTokensExceededError` on the structured path, and `AbortBatchError` for an abort classification; the first three share the `GenerationError` base a caller can catch at once.
`generate_many` returns `list[Response[OutputT] | GenerationError]`, so a terminal per-item failure is a row in its slot rather than a raise, while an `AbortBatchError` still cancels the siblings and raises; results stay order-aligned.

**Streaming is a handle.**
`stream_one` returns a `StreamHandle`: an async iterator of `StreamItem = str | ToolCall`, an idempotent `await handle.final()` returning the assembled `Response`, and an async context manager so abandoning a stream closes the connection.
Text chunks are the provider SDK's own strings passed through without a wrapper class or copy, and each `ToolCall` is yielded once, complete, when its block closes.
There are deliberately no tool-call delta items (a consumer cannot act on partial argument JSON, and both SDKs accumulate the arguments and hand over the finished call) and no usage or stop items (`usage` (carrying `cost_in_usd`) and `stop_reason` live on `final()`'s `Response`, and a bare-`str` stop reason could not share a union with bare-`str` text chunks).
Connection failures before the first yielded item retry under the `RateLimiter`; after the first yielded item nothing retries, because replaying chunks the caller already consumed would duplicate output.
An open stream holds one `RateLimiter` slot from opening until it closes or exhausts, so long-lived streams count against `max_in_flight`.

**Tools are explicit, dispatch is owned.**
A `PydanticTool` is an async function taking one pydantic model and returning str or a sequence of content parts (text and images the model then sees), plus an explicit name and description; the args model is the schema source, so there is no signature introspection and no docstring scraping.
Tool content is model-facing, so it is exactly `MessageContent` (`str | Sequence[Part]`, the model-facing message body, aliased in `messages.py`): a function with a typed result serializes it to that form itself.
A function may instead return a `ToolOutputExplicit` wrapping that content plus `is_error` and `app_data` (the app-facing channel, which does carry a typed `BaseModel` or `Mapping` the model never sees).
`PydanticTool.validate_and_run` validates raw call JSON against `args_model` and runs the function; it lives on `PydanticTool` because there the args type parameter is concrete, so the validated arguments reach the function fully typed.
`PydanticTool.dispatch` returns `DispatchHandled[AppDataT] | DispatchInvalidToolArgs`, so on the handled outcome a caller that dispatched a known tool reads `app_data` back at its concrete type with no `isinstance`.
For a tool whose schema is a raw JSON schema and not a pydantic model (an MCP tool discovered at run time), `JSONSchemaTool` carries the JSON schema as `args_schema` (sent to the provider unchanged) and a function taking the parsed arguments as a `dict[str, object]` (the type dispatch builds; a `Mapping[str, object]` annotation is also accepted by contravariance); `dispatch` validates the arguments against `args_schema` with `jsonschema` (first that they are a JSON object, then the schema's field-level rules), so the function only ever sees valid arguments, a `JSONSchemaTool` failure renders through the same formatter as a `PydanticTool` failure, and wrapping an MCP tool needs no hand-written pydantic model or validation code per tool. `PydanticTool` and `JSONSchemaTool` share the `Tool` protocol (a `name`, a `schema()`, a `dispatch()`), so one `ToolManager` holds a mix of the two, and an application adds its own tool form by implementing `Tool`.
`ToolManager` serializes the tools provider-neutrally, resolves the called name, and returns each outcome as a `DispatchOutcome`, a three-arm union over a heterogeneous tool set (where `app_data` erases to `BaseModel | Mapping[str, object] | None`): `DispatchHandled` carries the model-facing `ToolMessage` the caller appends to the conversation, paired with the function's `app_data` (data the model never sees); `DispatchInvalidToolArgs` carries a default `is_error` `ToolMessage` plus the neutral `InvalidToolArgsDetail` tuple as a required `details` field, so a caller that authors its own reply reads the per-failure paths and messages with no narrowing crutch; `DispatchUnknownTool` carries a default `is_error` `ToolMessage` naming the held tools plus the off-list `called_name`.
Every arm carries `tool_message`, so a caller that only appends the reply reads `result.tool_message` with no `match`.
An off-list tool name becomes a `DispatchUnknownTool` outcome rather than a raise, just like argument JSON the model got wrong becomes a `DispatchInvalidToolArgs` outcome: both are model data the model can correct (a provider can emit a name outside the sent schemas, and a rebind can strand an earlier turn's `tool_call`).
`ToolManager.dispatch_many` dispatches one assistant turn's several tool calls concurrently and returns the outcomes in call order.
A tool function exception is still a defect and still raises, but only after every sibling settles: the raise is a `DispatchExceptionGroup` (an `ExceptionGroup`, so every traceback prints and `except*` works) whose `completed_outcomes` carries the settled calls' outcomes, so `app_data` a completed sibling produced (a billing record for money the tool spent) survives the raise.
Cancellation propagates bare, never grouped, and only after the cancelled sibling dispatches finish unwinding; defects co-occurring with it chain as its `__cause__` in a `DispatchExceptionGroup`, so they still surface.
`tool_choice` is the neutral vocabulary `"auto" | "required" | SpecificToolChoice | "none"` (anthropic's `"any"` maps from `"required"`); `parallel_tool_calls: bool` maps to anthropic `disable_parallel_tool_use` and openai `parallel_tool_calls`.

**The ReAct loop is a recipe, not vendored code.**
There is no agent class.
The loop is ~15 lines and is correct because the hard parts (retries, pacing, classification, validation) live below `generate_one` and `dispatch`, so a hand-copied loop cannot be subtly wrong.
The non-streaming loop runs over `generate_one`; the streaming loop runs over `stream_one`, printing text chunks as they arrive and dispatching the completed `ToolCall`s between turns, reading `dispatch(call).tool_message` for the ToolMessage it appends and matching `DispatchHandled` for the function's model-invisible `app_data` or `DispatchInvalidToolArgs` for the `details`.
Owning the loop is what lets a caller enforce a budget mid-run, stream tokens, or swap the binding on a tool result, none of which a fixed agent surface exposes.

## Layout

    CLAUDE.md                design, naming, and review rules
    README.md                this file
    src/langchaint/
        messages.py          message and part models, StopReason
        usage.py             Usage counters
        pricing.py           PriceableCounts, CostBreakdown, price
        inference_params.py  InferenceParams
        exceptions.py        error vocabulary: TransientError,
                             AbortBatchError, AttemptRecord,
                             GenerationError and its leaves
        response.py          Response[OutputT], to_row
        rate_limiter.py      RateLimiter
        tools.py             Tool, PydanticTool, JSONSchemaTool,
                             ToolSchema, ToolManager
        provider.py          Binding, ToolChoice, PricingTable,
                             StreamItem, ProviderResult,
                             ProviderStream, BoundProvider, Provider
        streaming.py         StreamHandle
        llm.py               LLM, BoundLLM, the retry loop
        anthropic/           the anthropic backend (needs the anthropic
                             SDK): __init__ has anthropic_model,
                             ANTHROPIC_PRICING, AnthropicModelName,
                             cost_breakdown; messages_provider.py has
                             the adapter AnthropicMessagesProvider
        openai/              the openai backend (needs the openai SDK):
                             __init__ has openai_model, OPENAI_PRICING,
                             OpenAIModelName, cost_breakdown;
                             responses_provider.py has the adapter
                             OpenAIResponsesProvider
        tracing/             OTel span tracing (needs opentelemetry-api):
                             __init__ has
                             TracedLLM, TracedBoundLLM, TracedStreamHandle

## Verification

Run `scripts/CI.sh`; it runs `pyrefly check`, `ruff check`, and `pytest` through `uv run`, so the tools resolve from the locked dev dependency group, not a hand-activated `.venv`.
All must pass with zero errors.
The tests are offline: they feed constructed SDK objects into the adapter helpers and drive `BoundLLM`/`StreamHandle` with stub providers, so they need no API keys.
