Metadata-Version: 2.4
Name: vmi-llm-sdk
Version: 0.2.0
Summary: VMI LLM SDK — prompts, memory (vector + connected DB), transactions, multi-provider LLM calls
Author-email: VMI <kartik.p@talowiz.com>
License-Expression: MIT
Project-URL: Repository, https://bitbucket.org/shubhanshu_talowiz/vmi-libraries
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.31.0
Provides-Extra: openai
Requires-Dist: openai>=1.30.0; extra == "openai"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.30.0; extra == "anthropic"
Provides-Extra: gemini
Requires-Dist: google-generativeai>=0.7.0; extra == "gemini"
Provides-Extra: all
Requires-Dist: openai>=1.30.0; extra == "all"
Requires-Dist: anthropic>=0.30.0; extra == "all"
Requires-Dist: google-generativeai>=0.7.0; extra == "all"
Provides-Extra: test-app
Requires-Dist: python-dotenv>=1.0.0; extra == "test-app"
Dynamic: license-file

# VMI LLM SDK (Python)

A configuration-driven SDK for building AI features on top of the VMI platform.
It integrates two VMI backend services and calls LLM providers directly with
your own API keys:

| Service | Auth | Used for |
| --- | --- | --- |
| **vmi-backend** | `X-Access-Token` header (or JWT) | prompt definitions, 2-phase call logging, connected-DB context, tagged memory data |
| **vmi-orchestrator** (vector DB) | `Authorization: Bearer` | semantic memory search/write, sessions, transactions |
| **LLM providers** (OpenAI, Anthropic/Claude, Gemini, DeepSeek) | your own API keys | the actual model call |

This README documents **every public function, method and class** in the
SDK — what it does, every parameter it accepts, what it returns, what it
raises, and a runnable example. Use the table of contents to jump straight
to what you need.

---

## Table of contents

- [Install](#install)
- [Quick start](#quick-start)
- [Configuration — `VmiConfig`](#configuration--vmiconfig)
- [Module-level API (`llm_sdk`)](#module-level-api-llm_sdk)
- [`VmiClient` — the entry point](#vmiclient--the-entry-point)
  - [Health](#health)
  - [Memory functions](#memory-functions)
  - [LLM functions](#llm-functions)
  - [Transaction functions](#transaction-functions)
  - [Session functions](#session-functions)
- [Data models](#data-models)
- [Exceptions](#exceptions)
- [Internals reference](#internals-reference)
  - [`MemoryService`](#memoryservice-llm_sdkmemory_servicepy)
  - [`LLMService`](#llmservice-llm_sdkllm_servicepy)
  - [`VectorDBClient`](#vectordbclient-llm_sdkvector_clientpy)
  - [`BackendClient`](#backendclient-llm_sdkbackend_clientpy)
  - [`HttpClient`](#httpclient-llm_sdkhttp_clientpy)
  - [`PromptCache`](#promptcache-llm_sdkprompt_cachepy)
  - [`resolvePrompt`](#resolveprompt-llm_sdkprompt_resolverpy)
  - [Providers](#providers-llm_sdkproviders)
- [Transactions](#transactions)
- [Sessions](#sessions)
- [Architecture](#architecture)
- [Test bench (web UI)](#test-bench-web-ui)
- [Tests](#tests)

---

## Install

```bash
pip install -e .            # core (requests only)
pip install -e ".[all]"     # + openai, anthropic, google-generativeai
```

Provider keys can be supplied via `VmiConfig` fields or the matching
environment variable: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`,
`GOOGLE_API_KEY`, `DEEPSEEK_API_KEY`. DeepSeek reuses the OpenAI SDK against
a different `base_url`, so it needs no separate package.

---

## Quick start

```python
from llm_sdk import VmiClient, VmiConfig

# backendUrl / vectorDbUrl are NOT constructor arguments — they are
# hardcoded inside llm_sdk/config.py and always point at the fixed
# vmi-backend / vmi-orchestrator deployment this SDK talks to. You only
# supply credentials and identity.
client = VmiClient(VmiConfig(
    accessToken="<company access token>",
    vectorDbToken="<vector db token>",
    companyCode="WPCORP4812",
    background="You help engineers with the ACME order platform.",
))
```

Or the module-level API, which keeps a single default client for you:

```python
import llm_sdk
llm_sdk.init(config)
llm_sdk.callLLM(promptId="PROMPT-XXXX", inputParams={"question": "..."})
```

---

## Configuration — `VmiConfig`

`llm_sdk.config.VmiConfig` is a plain dataclass holding every setting the SDK
needs. Every field has a sensible default (`""`, `0`, etc.) except where
noted, so you only set what your use case requires — but two "require"
methods enforce the minimum needed for a given operation (see below).

> **`backendUrl` and `vectorDbUrl` are hardcoded, not configurable.** They
> are module-level constants (`_BACKEND_URL`, `_VECTOR_DB_URL`) baked into
> `llm_sdk/config.py`, wired onto `VmiConfig` as `init=False` fields. Every
> consumer of this library talks to the same fixed vmi-backend /
> vmi-orchestrator deployment — these are infrastructure details, not
> per-consumer configuration. **Passing `backendUrl=` or `vectorDbUrl=` to
> `VmiConfig(...)` raises a `TypeError`** (they aren't constructor
> parameters at all); to point at a different deployment you edit the
> constants in `config.py` itself. You only ever supply credentials
> (`accessToken`/`jwtToken`/`vectorDbToken`) and identity (`companyCode`/`tenantId`).

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `backendUrl` | `str` | *(hardcoded, not settable)* | Base URL of vmi-backend. Fixed in `llm_sdk/config.py`; not a constructor argument. Used for prompts, call logging, connected-DB context, tagged memory data. |
| `accessToken` | `str` | `""` | Company access token, sent as `X-Access-Token`. Required for vmi-backend's public endpoints (prompts, call logging, memory-data, `extract-open`). |
| `jwtToken` | `str` | `""` | Alternative/additional auth: sent as `Authorization: Bearer <jwtToken>` together with `X-Tenant-ID`. Required for the JWT-protected endpoints (`/database-knowledge/extract`, `/memory-data/fetch`). |
| `tenantId` | `str` | `""` | Tenant id sent as `X-Tenant-ID` alongside `jwtToken`. If left blank and `companyCode` is set, it is auto-filled from `companyCode` (and vice versa) in `__post_init__`. |
| `vectorDbUrl` | `str` | *(hardcoded, not settable)* | Base URL of vmi-orchestrator (the vector DB). Fixed in `llm_sdk/config.py`; not a constructor argument. Used for all memory operations (`fetchMemory`, `addMemory`, sessions, transactions). |
| `vectorDbToken` | `str` | `""` | Token sent as `Authorization: Bearer <vectorDbToken>` to vmi-orchestrator. |
| `openaiApiKey` | `str` | `os.environ["OPENAI_API_KEY"]` | OpenAI key. Falls back to the environment variable if not set explicitly. Also used as the summarizer model's key. |
| `anthropicApiKey` | `str` | `os.environ["ANTHROPIC_API_KEY"]` | Claude/Anthropic key. |
| `googleApiKey` | `str` | `os.environ["GOOGLE_API_KEY"]` | Gemini key. |
| `deepseekApiKey` | `str` | `os.environ["DEEPSEEK_API_KEY"]` | DeepSeek key (used with the OpenAI-compatible client). |
| `companyCode` | `str` | `""` | Identifies the calling company to vmi-backend; sent on `initiateCall`/`completeCall` and connected-DB requests. Same identifier as `tenantId` — whichever one you set fills the other. |
| `environmentCode` | `str` | `""` | Environment identifier (e.g. `"prod"`, `"staging"`) sent on call-logging requests. |
| `companyProjectCode` | `str` | `""` | Default memory isolation scope. When set, every vector write is tagged with `metadata.companyProjectCode` and every search is filtered by it, so multiple projects sharing one orchestrator never see each other's memories. Per-call `companyProjectCode` arguments to `fetchMemory` override this default. |
| `background` | `str` | `""` | Default "purpose" text injected into the system prompt when `callLLM(passBackground=True)` is called without an explicit string. |
| `summarizerModel` | `str` | `"gpt-4o-mini"` | Fixed internal OpenAI model used to condense fetched memory locally before the main LLM call, when the orchestrator doesn't return its own summary. |
| `defaultCollection` | `str` | `"vectors"` | Vector-DB collection used when a call doesn't specify one. |
| `defaultTopK` | `int` | `3` | Default number of results returned by `fetchMemory`/`search` when `topK` isn't specified. |
| `connectionTimeoutSeconds` | `float` | `10.0` | TCP connect timeout for every HTTP call the SDK makes. |
| `readTimeoutSeconds` | `float` | `60.0` | Read timeout for every HTTP call. |
| `maxRetries` | `int` | `2` | Number of retry attempts on transient HTTP failures (network errors, 502/503/504, and other configured retryable statuses). |
| `promptCacheTtlSeconds` | `float` | `300.0` | How long a prompt definition fetched from vmi-backend is cached before being re-fetched. |

### `VmiConfig.__post_init__()`

Runs automatically after construction. If only one of `companyCode` /
`tenantId` is set, it copies that value into the other, since they identify
the same tenant to the two different backends.

### `VmiConfig.requireBackend() -> None`

Validates that the config has enough to talk to vmi-backend: `backendUrl`
must be set (it always is, since it's hardcoded), and at least one of
`accessToken` or `jwtToken` must be set. Raises `VmiConfigError` if the
credential is missing. Called internally before any backend-dependent
operation (`callLLM` with a `promptId`, `fetchMemoryFromConnectedDB`,
`fetchPrompt`).

### `VmiConfig.requireVectorDb() -> None`

Validates that `vectorDbUrl` is set — always true in practice since it's
hardcoded; this check exists mainly as a defensive guard. Called internally
before any vector-DB operation (`fetchMemory`, `addMemory`, `addMemories`,
`deleteMemory`, `getMemory`, sessions, transactions, `ping`).

---

## Module-level API (`llm_sdk`)

For scripts and simple apps that only need one client, `llm_sdk` exposes a
default-client-backed functional API so you don't have to thread a
`VmiClient` instance through your code.

### `llm_sdk.init(config: VmiConfig) -> VmiClient`

Creates a new `VmiClient` from `config` and stores it as the module's default
client (used by every function below). Returns the created client in case
you want to keep a reference to it too. Calling `init` again replaces the
previous default client.

```python
import llm_sdk
client = llm_sdk.init(VmiConfig(vectorDbToken="..."))
```

### `llm_sdk.getClient() -> VmiClient`

Returns the default client set by `init`.

- **Raises:** `VmiConfigError` if `init()` was never called.

### `llm_sdk.fetchMemory(query: str, **kwargs) -> MemoryData`

Thin wrapper: calls `getClient().fetchMemory(query, **kwargs)`. See
[`VmiClient.fetchMemory`](#fetchmemory) for the full parameter list.

### `llm_sdk.addMemory(text: str, **kwargs) -> dict[str, Any]`

Thin wrapper: calls `getClient().addMemory(text, **kwargs)`. See
[`VmiClient.addMemory`](#addmemory).

### `llm_sdk.fetchMemoryFromConnectedDB(promptText: str, **kwargs) -> ConnectedDBContext`

Thin wrapper: calls `getClient().fetchMemoryFromConnectedDB(promptText, **kwargs)`.
See [`VmiClient.fetchMemoryFromConnectedDB`](#fetchmemoryfromconnecteddb).

### `llm_sdk.callLLM(**kwargs) -> LLMCallResult`

Thin wrapper: calls `getClient().callLLM(**kwargs)`. See
[`VmiClient.callLLM`](#callllm).

---

## `VmiClient` — the entry point

`llm_sdk.client.VmiClient` is the object-oriented entry point of the SDK.
Construct one per configuration/tenant; it wires up the backend client,
vector-DB client, memory service, prompt cache and LLM service for you.

```python
from llm_sdk import VmiClient, VmiConfig

# backendUrl / vectorDbUrl are hardcoded (see VmiConfig above) — only
# credentials and identity are passed in here.
client = VmiClient(VmiConfig(
    accessToken="...",
    vectorDbToken="...",
    companyCode="WPCORP4812",
))
```

### `VmiClient.__init__(config: VmiConfig)`

Builds the client. Since `config.backendUrl` is always set (it's hardcoded),
this always creates a `BackendClient` and a `PromptCache` on top of it;
backend-dependent methods (`fetchMemoryFromConnectedDB`, `callLLM` with
`promptId`, `fetchPrompt`) will still raise `VmiConfigError` at call time if
you haven't supplied `accessToken` or `jwtToken`. The vector-DB client is
likewise always created (`config.vectorDbUrl` is hardcoded too); the lazy
`config.requireVectorDb()` check on each vector-DB call is a defensive
no-op that only exists to fail loudly if that constant were ever cleared.

- **Parameters:** `config` (`VmiConfig`) — the SDK configuration described above.

### Health

#### `VmiClient.ping() -> dict[str, Any]`

Liveness check against vmi-orchestrator (`GET /health`, unauthenticated,
outside `/api/v1`).

- **Returns:** `{"status": "ok"}` (or whatever JSON body the orchestrator returns) when it's reachable.
- **Raises:** `VmiConfigError` if `vectorDbUrl` isn't configured; `VmiApiError` if the orchestrator is unreachable or returns a non-2xx status after retries.

```python
client.ping()  # {"status": "ok"}
```

### Memory functions

#### `fetchMemory`

```python
VmiClient.fetchMemory(query: str, **kwargs) -> MemoryData
```

Semantic memory search against vmi-orchestrator's `/api/v1/vectors/search`.
Delegates to `MemoryService.fetchMemory` (documented in full in
[Internals reference](#memoryservice-llm_sdkmemory_servicepy)); this is the
signature you actually call:

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `query` | `str` | — | The natural-language text to search for (embedded and matched against stored vectors). |
| `storage` | `str` | `"longterm"` | `"longterm"` or `"session"`. Session storage requires `sessionId`. |
| `sessionId` | `Optional[str]` | `None` | Session to search within, when `storage="session"`. |
| `collection` | `Optional[str]` | `None` | Vector collection name; falls back to `config.defaultCollection`. |
| `topK` | `Optional[int]` | `None` | Max number of results; falls back to `config.defaultTopK`. |
| `filters` | `Optional[dict]` | `None` | Metadata filters passed straight through to the orchestrator (e.g. `{"userId": "user_1"}`). |
| `tags` | `Optional[dict[str, str]]` | `None` | Tag filters, ANDed together (e.g. `{"projectCode": "PRJ1"}`). |
| `summarize` | `bool` | `True` | Ask the orchestrator to also return a server-side summary of the hits. |
| `companyProjectCode` | `Optional[str]` | `None` | Overrides `config.companyProjectCode` for this call, scoping the search to one project. |
| `rerank` | `bool` | `False` | Fetch a wider candidate pool and re-rank it with a cross-encoder (longterm storage only). |
| `rerankTopN` | `Optional[int]` | `None` | Size of the candidate pool before reranking (orchestrator default is 20). |
| `rerankMode` | `Optional[str]` | `None` | Reranking strategy name, passed through to the orchestrator. |
| `contentType` | `Optional[str]` | `None` | One of `CODING \| CHAT \| VIDEO \| DOCUMENT \| GENERAL` — selects the orchestrator's retrieval strategy for that content type. |
| `transaction` | `Optional[VmiTransaction]` | `None` | If given, the search is scoped to the transaction's `tx_id`, so it can see staged/uncommitted writes. |

- **Returns:** `MemoryData` — see [Data models](#data-models).
- **Raises:** `VmiConfigError` if `vectorDbUrl` isn't set; `VmiApiError` on unrecoverable API failures.
- **Resilience:** if `summarize=True` and the orchestrator returns a server error (≥500), the call is retried once with `summarize=False` rather than failing outright — the server-side summarizer is the flakiest part of search. Callers that need a summary (like `callLLM`) fall back to local summarization when this happens.

```python
memory = client.fetchMemory(
    "what did we decide about caching?",
    storage="longterm",
    topK=5,
    filters={"userId": "user_1"},
    tags={"projectCode": "PRJ1"},
    summarize=True,
)
memory.memoryText   # numbered plain-text block, ready to paste into an LLM prompt
memory.dataNodes    # [DataNode(id, text, score, metadata), ...]
memory.summary      # server-side summary (present when summarize=True and it succeeded)
```

#### `addMemory`

```python
VmiClient.addMemory(text: str, **kwargs) -> dict[str, Any]
```

Writes one memory entry to vector storage (`POST /api/v1/vectors/write`).

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `text` | `str` | — | The text to embed and store. |
| `vectorId` | `Optional[str]` | `None` | Explicit id for the entry; auto-generated (`mem_<uuid4>`) if omitted. |
| `metadata` | `Optional[dict]` | `None` | Arbitrary metadata stored alongside the vector (used later for `filters` in `fetchMemory`). |
| `tags` | `Optional[dict[str, str]]` | `None` | Tags stored alongside the vector (used later for `tags` AND-filters in `fetchMemory`). |
| `storage` | `str` | `"longterm"` | `"longterm"` or `"session"`. |
| `sessionId` | `Optional[str]` | `None` | Session to write into, when `storage="session"`. |
| `collection` | `Optional[str]` | `None` | Vector collection; falls back to `config.defaultCollection`. |
| `indexId` | `Optional[str]` | `None` | Optional index selector passed through to the orchestrator. |
| `isTransactional` | `bool` | `False` | Stage the write server-side without a client-managed transaction object. |
| `transaction` | `Optional[VmiTransaction]` | `None` | If given, implies `isTransactional=True` and binds the write to this transaction's `tx_id` (commit/rollback as a group). |

- **Returns:** the raw JSON response from the orchestrator's write endpoint (a `dict`).
- **Raises:** `VmiConfigError` if `vectorDbUrl` isn't set; `VmiApiError` on failure.

```python
client.addMemory(
    "The user prefers Python over Java",
    metadata={"type": "preference"},
    tags={"userId": "user_1"},
    isTransactional=True,
)
```

#### `addMemories`

```python
VmiClient.addMemories(items: list[dict[str, Any]], **kwargs) -> dict[str, Any]
```

Batch write (`POST /api/v1/vectors/batch`) — writes many memory entries in
one HTTP call.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `items` | `list[dict]` | — | Each item is `{"text": ..., "vector_id"? / "vectorId"?, "metadata"?, "tags"?}`. `text` is required per item; if `metadata` doesn't already contain `"text"`, it's added automatically (the batch endpoint, unlike single `write`, does not copy the text into metadata server-side, and without it the entries would be unsearchable/unreadable in results). |
| `storage` | `str` | `"longterm"` | `"longterm"` or `"session"`. |
| `sessionId` | `Optional[str]` | `None` | Session to write into. |
| `collection` | `Optional[str]` | `None` | Vector collection; falls back to `config.defaultCollection`. |
| `isTransactional` | `bool` | `False` | Stage the whole batch server-side. |
| `transaction` | `Optional[VmiTransaction]` | `None` | Applies the transaction's `tx_id` to the batch and to every item in it. |

- **Returns:** the raw JSON response from the batch endpoint (a `dict`).
- **Raises:** `VmiConfigError` if `vectorDbUrl` isn't set; `VmiApiError` on failure.

```python
client.addMemories([
    {"text": "User likes dark mode", "metadata": {"type": "preference"}},
    {"text": "User's timezone is IST", "tags": {"userId": "user_1"}},
])
```

#### `deleteMemory`

```python
VmiClient.deleteMemory(vectorId: str, **kwargs) -> dict[str, Any]
```

Deletes one memory entry by id (`DELETE /api/v1/vectors/delete`).

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `vectorId` | `str` | — | The id of the entry to delete. |
| `storage` | `str` | `"longterm"` | `"longterm"` or `"session"`. |
| `sessionId` | `Optional[str]` | `None` | Session the entry lives in, when `storage="session"`. |
| `collection` | `Optional[str]` | `None` | Vector collection; falls back to `config.defaultCollection`. |
| `isTransactional` | `bool` | `False` | Stage the delete server-side. |
| `transaction` | `Optional[VmiTransaction]` | `None` | Binds the delete to this transaction's `tx_id`. |

- **Returns:** the raw JSON response from the delete endpoint (a `dict`).
- **Raises:** `VmiConfigError` if `vectorDbUrl` isn't set; `VmiApiError` on failure (e.g. unknown `vectorId`).

```python
client.deleteMemory("mem_abc123", storage="longterm")
```

#### `getMemory`

```python
VmiClient.getMemory(vectorId: str, **kwargs) -> dict[str, Any]
```

Fetches a single memory entry by id (`GET /api/v1/vectors/{vectorId}`).

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `vectorId` | `str` | — | The id of the entry to fetch. |
| `storage` | `str` | `"longterm"` | `"longterm"` or `"session"`. |
| `collection` | `Optional[str]` | `None` | Vector collection; falls back to `config.defaultCollection`. |
| `sessionId` | `Optional[str]` | `None` | Session the entry lives in, when `storage="session"`. |

- **Returns:** the raw JSON response describing that entry (a `dict`).
- **Raises:** `VmiConfigError` if `vectorDbUrl` isn't set; `VmiApiError` if the id doesn't exist or the call fails.

```python
entry = client.getMemory("mem_abc123")
```

#### `fetchMemoryFromConnectedDB`

```python
VmiClient.fetchMemoryFromConnectedDB(promptText: str, **kwargs) -> ConnectedDBContext
```

Searches the company's registered/connected databases for context relevant
to `promptText` and returns a consolidated summary
(`POST /api/v1/database-knowledge/extract-open`, or `/extract` under JWT
auth).

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `promptText` | `str` | — | The text to find relevant database context for (typically the user's message). |
| `connectedDatabaseCodes` | `Optional[list[str]]` | `None` | Restrict the search to specific registered database codes; searches all connected databases if omitted. |
| `entityIdentifier` | `Optional[dict[str, str]]` | `None` | Identifies a specific entity to scope the search to, e.g. `{"type": "user", "id": "user_123"}`. |
| `topK` | `int` | `5` | Max number of facts/results to retrieve. |
| `purpose` | `Optional[str]` | `None` | Free-text purpose/intent hint passed to the extraction backend. |
| `identifierText` | `Optional[str]` | `None` | Free-text identifier hint (used when `entityIdentifier` isn't structured). |
| `transaction` | `Optional[VmiTransaction]` | `None` | Passes the transaction's `tx_id` in the request body. |

- **Returns:** `ConnectedDBContext` — see [Data models](#data-models).
- **Raises:** `VmiConfigError` if `backendUrl`/auth aren't configured; `VmiApiError` on failure.

```python
context = client.fetchMemoryFromConnectedDB(
    "recent orders for this customer",
    connectedDatabaseCodes=["DB-ORDERS"],
    entityIdentifier={"type": "user", "id": "user_123"},
)
context.consolidatedSummary
```

### LLM functions

#### `callLLM`

```python
VmiClient.callLLM(**kwargs) -> LLMCallResult
```

Executes an LLM call through the full pipeline that mirrors vmi-backend's
`CompanyProjectStepService.createStep` (memory retrieval → connected-DB
context → system-prompt assembly → provider call → call logging → memory
write-back). This is the SDK's flagship function — see the [pipeline
diagram](#llmservice-llm_sdkllm_servicepy) below for exactly what happens on
each call.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `promptId` | `Optional[str]` | `None` | Id of a prompt definition stored in vmi-backend. Mutually exclusive with `rawPrompt` — pass exactly one. When used, the vendor/model default to the prompt definition's `aiVendorCode`/`aiVendorVersionCode` unless `provider`/`model` override them. |
| `rawPrompt` | `Optional[str]` | `None` | An inline prompt template (with `{{placeholders}}`) instead of a stored prompt. Requires `provider` and `model` to be set explicitly. |
| `inputParams` | `Optional[dict[str, Any]]` | `None` | Values substituted into `{{name}}` placeholders in the prompt text (see [`resolvePrompt`](#resolveprompt-llm_sdkprompt_resolverpy)). Non-string values are JSON-encoded. |
| `provider` | `Optional[str]` | `None` | Vendor code: `OPENAI \| CLAUDE \| GEMINI \| DEEPSEEK` (case-insensitive). Overrides the prompt definition's vendor when both are given. |
| `model` | `Optional[str]` | `None` | Model/version code, e.g. `"gpt-4o"`, `"claude-sonnet-5"`. Overrides the prompt definition's model when both are given. |
| `passMemory` | `bool` | `False` | When `True`, fetches relevant memory for the user message and injects a summarized version into the system prompt. Non-fatal — a fetch/summarize failure degrades to no memory context, never blocks the call. |
| `saveInMemory` | `bool` | `False` | When `True`, instructs the model (via the system prompt) to return `{"output": ..., "summary": ...}` JSON, and writes `summary` back to vector storage after the call succeeds. |
| `connectedDB` | `bool` | `False` | When `True`, fetches connected-database context for the user message and injects its consolidated summary into the system prompt. Non-fatal. |
| `connectedDatabaseCodes` | `Optional[list[str]]` | `None` | Restricts the connected-DB fetch (when `connectedDB=True`) to specific database codes. |
| `passBackground` | `bool \| str` | `False` | When `True`, injects `config.background` as the purpose/role grounding at the top of the system prompt. Pass a `str` instead to override the configured background for just this call. |
| `storage` | `str` | `"longterm"` | Memory storage backend used for `passMemory` fetches and `saveInMemory` writes: `"longterm"` or `"session"`. |
| `sessionId` | `Optional[str]` | `None` | Session id, required when `storage="session"`. See the "first message" note below. |
| `collection` | `Optional[str]` | `None` | Vector collection for memory operations; falls back to `config.defaultCollection`. |
| `memoryTopK` | `Optional[int]` | `None` | `topK` passed to the internal `fetchMemory` call. |
| `memoryFilters` | `Optional[dict]` | `None` | `filters` passed to the internal `fetchMemory` call. |
| `memoryTags` | `Optional[dict[str, str]]` | `None` | `tags` passed to the internal `fetchMemory` call. |
| `temperature` | `float` | `0.7` | Sampling temperature passed to the provider. |
| `maxTokens` | `int` | `2048` | Max output tokens requested from the provider. |
| `transaction` | `Optional[VmiTransaction]` | `None` | Propagated to every child call this pipeline makes (memory fetch, connected-DB fetch, call-log initiate/complete, memory write-back), so the whole call — including its memory writes — can be staged and committed/rolled back atomically. |

- **Returns:** `LLMCallResult` — see [Data models](#data-models).
- **Raises:** `VmiConfigError` if neither/both of `promptId`/`rawPrompt` are given, or `rawPrompt` is given without `provider`+`model`, or backend config is missing when `promptId` is used; `ProviderError` if the underlying vendor call fails (memory/connected-DB/call-logging failures are swallowed and logged instead, per the non-fatal-periphery rule — only the LLM call itself can fail the request).
- **Session first-message rule:** mirrors `createProjectStep`'s `previousStepCount` check — the *first* message sent to a given `sessionId` (tracked per `VmiClient` instance, in-memory) skips the `passMemory` fetch entirely, since a brand-new session has no memory yet.
- **Output parsing:** when `saveInMemory=True`, the system prompt asks for `{"output": "...", "summary": "..."}` JSON. If the model doesn't comply, the raw text is used as `output` and `summary` is `None` — the call never fails just because the model ignored the format instruction.

```python
result = client.callLLM(
    promptId="PROMPT-XXXX",          # OR rawPrompt="..." + provider= + model=
    inputParams={"question": "..."}, # resolves {{question}} in the prompt

    passMemory=True,      # fetchMemory + gpt-4o-mini summarization → system prompt
    saveInMemory=True,    # LLM returns {"output","summary"}; summary written back
    connectedDB=True,     # inject connected-DB consolidated summary
    passBackground=True,  # inject purpose grounding (config.background or a str)

    storage="longterm", sessionId=None,
    temperature=0.7, maxTokens=2048,
)
result.output        # main response text
result.summary       # memory summary (present when saveInMemory=True)
result.callCode      # vmi-backend call-tracking code
result.totalTokens, result.durationMs
```

Pipeline executed per call (see [`LLMService`](#llmservice-llm_sdkllm_servicepy)
for the full breakdown of each step):

```
resolve prompt (promptId → backend, TTL-cached | rawPrompt)
  → passMemory:    fetchMemory → summarize (gpt-4o-mini, non-fatal)
  → connectedDB:   extract-open → consolidatedSummary (non-fatal)
  → system prompt: background + summarized memory + DB context (+ JSON format if saveInMemory)
  → initiate call log (backend, non-fatal)
  → provider call (OPENAI | CLAUDE | GEMINI | DEEPSEEK)
  → complete call log (tokens, duration, status, memoryData)
  → saveInMemory:  write summary to vector DB
```

#### `fetchPrompt`

```python
VmiClient.fetchPrompt(promptId: str) -> PromptDefinition
```

Fetches (and TTL-caches) a prompt definition from vmi-backend
(`GET /api/v1/public/ai-prompts/{promptId}`) without executing it.

| Parameter | Type | Description |
| --- | --- | --- |
| `promptId` | `str` | The prompt's `visibleId`. |

- **Returns:** `PromptDefinition` — see [Data models](#data-models).
- **Raises:** `VmiConfigError` if `backendUrl`/auth aren't configured; `PromptNotFoundError` if the id doesn't resolve.
- **Caching:** results are cached per `promptId` for `config.promptCacheTtlSeconds` (default 300s); repeated calls within the TTL don't hit the network.

```python
prompt = client.fetchPrompt("PROMPT-XXXX")
prompt.promptContent, prompt.aiVendorCode, prompt.aiVendorVersionCode
```

#### `invalidatePromptCache`

```python
VmiClient.invalidatePromptCache(promptId: Optional[str] = None) -> None
```

Forces the next `fetchPrompt`/`callLLM(promptId=...)` call to re-fetch from
vmi-backend instead of using the cached definition.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `promptId` | `Optional[str]` | `None` | Invalidate only this prompt's cache entry. If `None`, clears the entire cache. |

- **Returns:** `None`. No-op if the client has no backend configured.

```python
client.invalidatePromptCache("PROMPT-XXXX")   # one entry
client.invalidatePromptCache()                # everything
```

### Transaction functions

All vector-DB write/search operations in this SDK can be scoped to a
transaction. See [Transactions](#transactions) below for the concept; this
section documents the individual methods.

#### `beginTransaction`

```python
VmiClient.beginTransaction(
    backend: str = "longterm",
    collection: Optional[str] = None,
    sessionId: Optional[str] = None,
) -> VmiTransaction
```

Opens a new transaction on vmi-orchestrator (`POST /api/v1/tx/begin`).

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `backend` | `str` | `"longterm"` | Which storage backend the transaction applies to: `"longterm"` or `"session"`. |
| `collection` | `Optional[str]` | `None` | Vector collection; falls back to `config.defaultCollection`. |
| `sessionId` | `Optional[str]` | `None` | Session the transaction is scoped to, when relevant. |

- **Returns:** `VmiTransaction` — see [Data models](#data-models). Pass it as the `transaction=` argument to any other SDK function.
- **Raises:** `VmiConfigError` if `vectorDbUrl` isn't set; `TransactionError` if the orchestrator rejects the request or returns no `tx_id`.

```python
tx = client.beginTransaction()
```

#### `commitTransaction`

```python
VmiClient.commitTransaction(transaction: VmiTransaction) -> dict[str, Any]
```

Commits all writes staged under `transaction` (`POST /api/v1/tx/{txId}/commit`).

- **Parameters:** `transaction` (`VmiTransaction`) — the transaction to commit.
- **Returns:** the raw JSON response from the commit endpoint.
- **Raises:** `TransactionError` if the commit fails.

#### `rollbackTransaction`

```python
VmiClient.rollbackTransaction(transaction: VmiTransaction) -> dict[str, Any]
```

Discards all writes staged under `transaction`
(`POST /api/v1/tx/{txId}/rollback`).

- **Parameters:** `transaction` (`VmiTransaction`) — the transaction to roll back.
- **Returns:** the raw JSON response from the rollback endpoint.
- **Raises:** `TransactionError` if the rollback fails.

#### `transactionStatus`

```python
VmiClient.transactionStatus(transaction: VmiTransaction) -> dict[str, Any]
```

Reads the current status of a transaction (`GET /api/v1/tx/{txId}/status`)
— e.g. whether it's open, committed or rolled back.

- **Parameters:** `transaction` (`VmiTransaction`).
- **Returns:** the raw JSON status body.

#### `transaction` (context manager)

```python
with VmiClient.transaction(
    backend: str = "longterm",
    collection: Optional[str] = None,
    sessionId: Optional[str] = None,
) as tx:
    ...
```

Convenience wrapper around `beginTransaction`/`commitTransaction`/
`rollbackTransaction`: begins a transaction, yields it, commits on a clean
exit, and rolls back automatically (re-raising the original exception) if
the `with` block raises.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `backend` | `str` | `"longterm"` | Same as `beginTransaction`. |
| `collection` | `Optional[str]` | `None` | Same as `beginTransaction`. |
| `sessionId` | `Optional[str]` | `None` | Same as `beginTransaction`. |

- **Yields:** `VmiTransaction`.

```python
with client.transaction() as tx:                 # commit on exit, rollback on error
    client.addMemory("staged fact", transaction=tx)
    client.callLLM(rawPrompt="...", provider="OPENAI", model="gpt-4o",
                   saveInMemory=True, transaction=tx)

# equivalent, done manually:
tx = client.beginTransaction()
client.transactionStatus(tx)
client.commitTransaction(tx)      # or client.rollbackTransaction(tx)
```

### Session functions

#### `createSession`

```python
VmiClient.createSession(sessionId: str) -> None
```

Creates a session on vmi-orchestrator (`POST /api/v1/session/create`).

| Parameter | Type | Description |
| --- | --- | --- |
| `sessionId` | `str` | The id to create. |

- **Returns:** `None`.
- **⚠️ Not idempotent:** creating a session that already exists **replaces
  it and wipes its stored vectors**. Call this once per genuinely new
  session — writes and searches with `storage="session"` auto-create a
  missing session on demand, so calling `createSession` up front is
  optional and only useful when you want to guarantee a clean slate.

```python
client.createSession("sess_001")
```

#### `closeSession`

```python
VmiClient.closeSession(sessionId: str, flush: bool = True) -> dict[str, Any]
```

Closes a session (`POST /api/v1/session/close`).

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `sessionId` | `str` | — | The session to close. |
| `flush` | `bool` | `True` | When `True`, flushes the session's memory into longterm storage before closing so it isn't lost. |

- **Returns:** the raw JSON response from the close endpoint.

```python
client.fetchMemory("...", storage="session", sessionId="sess_001")
client.closeSession("sess_001", flush=True)   # flush session memory to longterm
```

---

## Data models

All models live in `llm_sdk/models.py` and are **frozen dataclasses**
(immutable — set once at construction, never mutated afterward) unless noted.

### `MemorySource` (str Enum)

Identifies where memory came from. Values: `SESSION = "session"`,
`LONG_TERM = "longterm"`, `CONNECTED_DB = "connected_db"`.

### `VmiTransaction`

An open vector-DB transaction handle. Pass it as `transaction=` to
`fetchMemory`/`addMemory`/`callLLM`/etc. so every child API call carries its
`tx_id` and is staged rather than applied immediately.

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `txId` | `str` | — | The transaction id returned by the orchestrator. |
| `backend` | `str` | `"longterm"` | `"longterm"` or `"session"`. |
| `collection` | `str` | `"vectors"` | The vector collection the transaction applies to. |
| `sessionId` | `Optional[str]` | `None` | Associated session, if any. |

### `PromptDefinition`

A prompt fetched from vmi-backend.

| Field | Type | Description |
| --- | --- | --- |
| `visibleId` | `str` | The prompt's public id. |
| `title` | `str` | Human-readable title. |
| `description` | `str` | Human-readable description. |
| `promptContent` | `str` | The template text, containing `{{placeholder}}` variables. |
| `aiVendorCode` | `str` | Default vendor for this prompt: `OPENAI \| CLAUDE \| GEMINI \| DEEPSEEK`. |
| `aiVendorVersionCode` | `str` | Default model code, e.g. `"gpt-4o"`. |
| `isLive` | `bool` | Whether the prompt is currently active/published in vmi-backend. |

### `DataNode`

One individual hit from a vector search.

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `id` | `str` | — | The vector's id. |
| `text` | `str` | — | The extracted text content of the hit (prefers `chunkBody` for code chunks, then `text`/`content`/`metadata.text`). |
| `score` | `Optional[float]` | `None` | Similarity score, when returned by the orchestrator. |
| `metadata` | `dict[str, Any]` | `{}` | The full metadata stored alongside the vector. |

### `MemoryData`

Result of `fetchMemory`.

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `query` | `str` | — | The original query text. |
| `memoryText` | `str` | — | A numbered, plain-text block (`"[1] ...\n[2] ..."`) ready to paste directly into an LLM prompt as context. |
| `dataNodes` | `list[DataNode]` | `[]` | The individual hits behind `memoryText`. |
| `summary` | `Optional[str]` | `None` | Server-side (orchestrator) summary of the hits, present when `summarize=True` succeeded. |
| `raw` | `dict[str, Any]` | `{}` | The full unprocessed response from the orchestrator. |

### `ConnectedDBContext`

Result of `fetchMemoryFromConnectedDB`.

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `promptText` | `str` | — | The text the search was run for. |
| `consolidatedSummary` | `str` | — | The main, human-readable summary of what was found — this is what you typically inject into a prompt. |
| `entityResults` | `list[dict[str, Any]]` | `[]` | Per-entity raw result rows. |
| `totalFactsFound` | `int` | `0` | Total number of facts retrieved across all searched databases. |
| `databasesSearched` | `int` | `0` | How many connected databases were queried. |
| `purpose` | `Optional[str]` | `None` | Echoes the `purpose` argument, if the backend returned one. |
| `raw` | `dict[str, Any]` | `{}` | The full unprocessed response data. |

### `LLMCallResult`

Result of `callLLM`.

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `output` | `str` | — | The model's main response text (extracted from the `{"output", "summary"}` JSON when `saveInMemory=True`, otherwise the raw response text). |
| `summary` | `Optional[str]` | — | The 2-3 sentence memory summary, present only when `saveInMemory=True` and the model complied with the requested JSON format. |
| `provider` | `str` | — | The resolved vendor code actually used (`OPENAI \| CLAUDE \| GEMINI \| DEEPSEEK`). |
| `model` | `str` | — | The resolved model/version code actually used. |
| `inputTokens` | `int` | — | Prompt/input tokens consumed, per the provider's usage report. |
| `outputTokens` | `int` | — | Completion/output tokens consumed. |
| `durationMs` | `int` | — | Wall-clock time the provider call took, in milliseconds. |
| `callCode` | `Optional[str]` | `None` | vmi-backend's call-tracking code (only set when a `promptId` was used and call logging succeeded). |
| `systemPrompt` | `str` | `""` | The exact system prompt sent to the model (background + summarized memory + DB context [+ JSON-format instructions]) — useful for debugging/inspection. |
| `memoryText` | `str` | `""` | The raw (pre-summarization) memory text that was fetched, when `passMemory=True`. |
| `connectedDBSummary` | `str` | `""` | The connected-DB consolidated summary that was injected, when `connectedDB=True`. |
| `raw` | `dict[str, Any]` | `{}` | The full raw response object from the underlying provider SDK call. |

**Property:**

- `LLMCallResult.totalTokens -> int` — computed as `inputTokens + outputTokens`.

### `ProviderResult`

The contract every provider client (`OpenAIProvider`, `AnthropicProvider`,
`GeminiProvider`) must return from `call(...)`.

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `text` | `str` | — | The raw text response from the model. |
| `inputTokens` | `int` | — | Input/prompt tokens used. |
| `outputTokens` | `int` | — | Output/completion tokens used. |
| `raw` | `dict[str, Any]` | `{}` | The vendor SDK's raw response object (or a compact dict for vendors like Gemini that don't serialize cleanly). |

---

## Exceptions

All in `llm_sdk/exceptions.py`. Every exception the SDK raises derives from
`VmiError`, so `except VmiError:` catches everything the SDK can throw.

| Exception | Extends | Extra attributes | Raised when |
| --- | --- | --- | --- |
| `VmiError` | `Exception` | — | Base class; never raised directly. |
| `VmiConfigError` | `VmiError` | — | The SDK is misconfigured: a required token/key is missing (e.g. calling `fetchMemoryFromConnectedDB` without `accessToken`/`jwtToken`, or `callLLM` with both/neither of `promptId`/`rawPrompt`). Not raised for the URLs — those are hardcoded and always present. |
| `VmiApiError` | `VmiError` | `statusCode: int \| None`, `responseBody: str \| None` | A vmi-backend or vmi-orchestrator HTTP call failed (non-2xx status after retries, or a network error). |
| `PromptNotFoundError` | `VmiApiError` | (inherited) | `fetchPrompt`/`callLLM(promptId=...)` referenced a `promptId` that vmi-backend doesn't recognize (HTTP 404, or an empty `data` payload). |
| `ProviderError` | `VmiError` | `provider: str \| None`, `errorCode: str \| None` | The underlying LLM vendor call failed — bad/missing API key, vendor API error, unknown `aiVendorCode`, or a required provider package (`anthropic`, `google-generativeai`) isn't installed. |
| `TransactionError` | `VmiApiError` | (inherited) | `beginTransaction`/`commitTransaction`/`rollbackTransaction` failed, or `tx/begin` returned no `tx_id`. |

```python
from llm_sdk import VmiError, VmiConfigError, ProviderError

try:
    client.callLLM(rawPrompt="Hi", provider="OPENAI", model="gpt-4o")
except ProviderError as e:
    print(e.provider, e.errorCode, e)
except VmiError as e:
    print("SDK error:", e)
```

---

## Internals reference

You normally interact only with `VmiClient` / the module-level functions
above. This section documents the internal service and client classes they
delegate to, for anyone extending the SDK, debugging a call, or writing
tests against a lower layer directly.

### `MemoryService` (`llm_sdk/memory_service.py`)

Implements the memory-related business logic on top of `VectorDBClient` and
`BackendClient`. `VmiClient.fetchMemory`/`addMemory`/`fetchMemoryFromConnectedDB`
delegate straight to this class — its methods and parameters are exactly
those documented under [Memory functions](#memory-functions) above,
constructed as:

```python
MemoryService(vectorClient: VectorDBClient, backendClient: Optional[BackendClient])
```

- **`fetchMemory(...)  -> MemoryData`** — runs `VectorDBClient.search(...)`, converts
  the raw `results` list into `MemoryData.memoryText` (via
  `VectorDBClient.toMemoryText`) and `MemoryData.dataNodes` (via
  `VectorDBClient.toDataNodes`), and retries once without `summarize` on a
  server-side summarizer failure.
- **`addMemory(...) -> dict`** — thin pass-through to `VectorDBClient.write(...)`.
- **`fetchMemoryFromConnectedDB(...) -> ConnectedDBContext`** — pass-through to
  `BackendClient.extractConnectedDBContext(...)`; raises `VmiConfigError` if
  no `backendClient` was configured (i.e. `VmiConfig.backendUrl` is unset).

### `LLMService` (`llm_sdk/llm_service.py`)

Implements `callLLM`'s orchestration — a faithful Python port of
vmi-backend's `CompanyProjectStepService.createStep`. Constructed as:

```python
LLMService(config, providerRegistry, memoryService, backendClient, promptCache)
```

Public methods:

- **`callLLM(**kwargs) -> LLMCallResult`** — documented in full under
  [`VmiClient.callLLM`](#callllm). Internally it runs, in order:
  1. **Resolve the prompt.** `promptId` → `_getPromptDefinition` (cached) →
     `resolvePrompt` substitutes `inputParams` into `{{placeholders}}`, and
     vendor/model default to the definition unless overridden. `rawPrompt` →
     `resolvePrompt` directly, vendor/model must be supplied explicitly.
  2. **Memory context** (`passMemory=True`). Skipped entirely on the first
     message of a brand-new session (`storage="session"`, tracked per
     client instance) — mirrors `createProjectStep`'s `previousStepCount`
     check. Otherwise calls `MemoryService.fetchMemory`, then prefers the
     orchestrator's own `summary`, falling back to `_summarizeMemory` (a
     local call to the fixed `summarizerModel`, default `gpt-4o-mini`) when
     no summary was returned. Any exception here is caught and logged —
     memory context is optional, never fatal.
  3. **Connected-DB context** (`connectedDB=True`). Calls
     `MemoryService.fetchMemoryFromConnectedDB`; exceptions are caught and
     logged, non-fatal.
  4. **Background/purpose grounding** (`passBackground`). Uses the string
     directly if one was passed, otherwise `config.background`.
  5. **Assemble the system prompt** via `_buildSystemPrompt` (see below).
  6. **Initiate call logging** (`BackendClient.initiateCall`) — only when a
     `promptId` was used and a backend client exists; never blocks the LLM
     call even on failure (returns `None` instead of raising).
  7. **Call the provider** (`ProviderRegistry.getProvider(vendorCode).call(...)`),
     timing the call. A failure here **is** propagated (this is the one step
     allowed to fail the whole request) — but the failure is still logged
     to vmi-backend first (`completeCall(status="ERROR", ...)`) before
     re-raising.
  8. **Parse the output** via `_parseOutput` — extracts `{"output","summary"}`
     JSON when `saveInMemory=True` was requested; falls back to treating the
     whole response as plain `output` text if the model didn't comply.
  9. **Complete call logging** (`BackendClient.completeCall`) with token
     counts, duration, status and the summary.
  10. **Write the summary back to memory** (`saveInMemory=True` and a
      `summary` was extracted) via `MemoryService.addMemory`, tagged with
      `metadata={"type": "summary", "promptId": ..., "callCode": ...}`.
      Failure here is caught and logged, non-fatal.
- **`getPrompt(promptId: str) -> PromptDefinition`** — same as
  `VmiClient.fetchPrompt`; delegates to the internal cached loader.

Private helpers (useful to understand when debugging a `callLLM` call):

- **`_getPromptDefinition(promptId) -> PromptDefinition`** — reads through
  the `PromptCache`; raises `VmiConfigError` if no cache/backend was
  configured.
- **`_summarizeMemory(memoryText: str) -> str`** — sends `memoryText` to
  `config.summarizerModel` (via a throwaway `OpenAIProvider`) with a fixed
  system prompt asking for a 3-5 sentence plain-text condensation.
  `temperature=0.3`, `maxTokens=512`. Returns `""` on empty input or on any
  failure (never blocks the main call).
- **`_buildSystemPrompt(background, summarizedMemory, connectedDBSummary, requireSummary) -> str`**
  *(static)* — assembles the final system prompt text in this order:
  1. Role line — `"You are an AI assistant. Your purpose: {background}"` or
     the generic `"You are an AI assistant."` if no background was given.
  2. `"Relevant context from previous interactions:\n{summarizedMemory}"` (if non-empty).
  3. `"Relevant data from connected databases:\n{connectedDBSummary}"` (if non-empty).
  4. The strict `{"output": "...", "summary": "..."}` JSON-format instruction
     (only when `requireSummary=True`, i.e. `saveInMemory=True`).

  Note: the user's actual message is deliberately **not** included here —
  it's passed separately as the provider's `userMessage`, keeping a clean
  role/context/user split.
- **`_parseOutput(text, expectSummary) -> tuple[str, Optional[str]]`**
  *(static)* — if `expectSummary=False`, returns `(text, None)` unchanged.
  Otherwise searches `text` for a `{...}` JSON block (via regex), parses it,
  and returns `(parsed["output"], parsed.get("summary"))` when valid;
  degrades to `(text, None)` with a warning log if parsing fails or the
  expected keys are missing.

### `VectorDBClient` (`llm_sdk/vector_client.py`)

The low-level HTTP client for vmi-orchestrator. A Python port of
vmi-backend's `llmutils/VectorDBClient`, extended with transaction and
session endpoints. Constructed as `VectorDBClient(config: VmiConfig)`.

- **`health() -> dict`** — `GET /health` (unauthenticated, outside `/api/v1`).
- **`createSession(sessionId: str) -> None`** — `POST /api/v1/session/create`.
  **Not idempotent** — see the [`createSession`](#createsession) warning above.
- **`closeSession(sessionId: str, flush: bool = True) -> dict`** — `POST /api/v1/session/close`.
- **`beginTransaction(backend="longterm", collection=None, sessionId=None) -> VmiTransaction`**
  — `POST /api/v1/tx/begin`; raises `TransactionError` on failure or a
  missing `tx_id` in the response.
- **`commitTransaction(transaction) -> dict`** — `POST /api/v1/tx/{txId}/commit`.
- **`rollbackTransaction(transaction) -> dict`** — `POST /api/v1/tx/{txId}/rollback`.
- **`transactionStatus(transaction) -> dict`** — `GET /api/v1/tx/{txId}/status`.
- **`listOpenTransactions() -> dict`** — `GET /api/v1/tx/open`; lists all
  currently-open transactions (not exposed on `VmiClient` — call it directly
  on `client._vectorClient` if needed, or via your own `VectorDBClient`).
- **`search(queryText, storage="longterm", sessionId=None, collection=None, topK=None, filters=None, tags=None, summarize=True, indexId=None, companyProjectCode=None, rerank=False, rerankTopN=None, rerankMode=None, contentType=None, transaction=None) -> dict`**
  — `POST /api/v1/vectors/search`. Builds the request body from all the
  optional filters/tags/rerank/contentType parameters, includes `tx_id` when
  a transaction is given, and is read-safe to retry on transient 500s
  *unless* `summarize=True` (a deterministic summarizer failure would just
  delay `MemoryService`'s fallback). Returns
  `{"results": [...], "count": n, "summary"?: str}`.
- **`write(text, vectorId=None, metadata=None, tags=None, storage="longterm", sessionId=None, collection=None, indexId=None, isTransactional=False, transaction=None, update=False) -> dict`**
  — `POST /api/v1/vectors/write`, or `PUT /api/v1/vectors/update` when
  `update=True`. Auto-generates `vector_id` as `mem_<uuid4>` if not given.
  A `transaction` implies `is_transactional=True`.
- **`batchWrite(items, storage="longterm", sessionId=None, collection=None, isTransactional=False, transaction=None) -> dict`**
  — `POST /api/v1/vectors/batch`. See [`addMemories`](#addmemories) above
  for the item shape and the `metadata["text"]` backfill behavior.
- **`delete(vectorId, storage="longterm", sessionId=None, collection=None, isTransactional=False, transaction=None) -> dict`**
  — `DELETE /api/v1/vectors/delete`.
- **`fetch(vectorId, storage="longterm", collection=None, sessionId=None) -> dict`**
  — `GET /api/v1/vectors/{vectorId}`.
- **`toDataNodes(results: list[dict]) -> list[DataNode]`** *(static)* —
  converts raw orchestrator result rows into `DataNode` objects.
- **`toMemoryText(results: list[dict]) -> str`** *(static)* — converts raw
  result rows into the numbered plain-text block used as `MemoryData.memoryText`.
  Keeps only the best-scoring chunk per `artifactId` (results arrive sorted
  by score) so multi-chunk hits from the same source file/document don't
  flood the context.
- **`_extractText(row: dict) -> str`** *(static, internal)* — for chunk
  results (`metadata.isChunk is True`), prefers `metadata.chunkBody` (the
  full code/text) over `embed_text` (signature + docstring only); otherwise
  falls back to `row["text"]`, `row["content"]`, or `metadata["text"]`, in
  that order.
- **`_sessionExists(sessionId, collection="vectors") -> bool`** *(internal)*
  — non-destructive probe: fetches a made-up vector id scoped to the
  session; a `404` means the session exists (the vector just doesn't), a
  `500` means the session itself is missing. There's no dedicated
  session-status endpoint, so this is the only safe way to check.
- **`_callWithSessionRecovery(call, storage, sessionId)`** *(internal)* —
  wraps a vector operation; on a ≥500 error against `storage="session"`, it
  checks `_sessionExists` and — only if the session is genuinely
  missing/expired — creates it and retries the call once. This is the only
  point where the SDK auto-creates a session, precisely because
  `createSession` up front would risk wiping an existing one.

### `BackendClient` (`llm_sdk/backend_client.py`)

The low-level HTTP client for vmi-backend. Supports two auth modes sent
together whenever both are configured: `X-Access-Token` for public
endpoints, and `Authorization: Bearer <jwt>` + `X-Tenant-ID` for
JWT-protected ones. Constructed as `BackendClient(config: VmiConfig)`.
Responses are all wrapped server-side as `{"success": bool, "message": str, "data": ...}`;
`_data(response, context)` unwraps this and raises `VmiApiError` when
`success` is falsy.

- **`getPrompt(promptId: str) -> PromptDefinition`** — `GET /api/v1/public/ai-prompts/{promptId}`.
  Raises `PromptNotFoundError` on HTTP 404 or an empty `data` payload.
- **`initiateCall(aiPromptCode, promptInText, aiVendorCode, aiVendorVersionCode, inputObjectJson=None, txId=None) -> Optional[str]`**
  — `POST /api/v1/public/ai-prompt-calls/initiate`, phase 1 of call
  logging. Returns the `callCode` to pass to `completeCall`, or `None` if
  the request fails — **never raises**, since call logging must never block
  the actual LLM call.
- **`completeCall(callCode, status, promptOutText, totalInputTokens, totalOutputTokens, durationInMilliseconds, errorMessage=None, errorCode=None, memoryData=None, txId=None) -> None`**
  — `POST /api/v1/public/ai-prompt-calls/complete`, phase 2. Also
  **non-fatal**: logs a warning on failure instead of raising.
- **`extractConnectedDBContext(promptText, connectedDatabaseCodes=None, entityIdentifier=None, topK=5, purpose=None, identifierText=None, txId=None) -> ConnectedDBContext`**
  — posts a `PromptContextRequest` to `/api/v1/database-knowledge/extract-open`
  (when `config.accessToken` is set) or `/extract` (JWT auth otherwise).
- **`pushMemoryData(dataType, jsonData, searchableTag=None, uniqueId=None, isForTesting=False) -> dict`**
  — `POST /api/v1/public/memory-data`. Pushes arbitrary tagged JSON data
  (not vector-embedded) for later retrieval by tag. Not wrapped by
  `VmiClient` — call it via your own `BackendClient` instance, or reach into
  `client._backendClient.pushMemoryData(...)` if needed.

### `HttpClient` (`llm_sdk/http_client.py`)

Thin `requests`-based HTTP layer shared by `BackendClient` and
`VectorDBClient`. Centralizes timeouts, retries and error mapping.
Constructed as:

```python
HttpClient(baseUrl, headers=None, connectionTimeout=10.0, readTimeout=60.0, maxRetries=2)
```

- **`request(method, path, json=None, params=None, headers=None, retryableStatuses=None) -> dict`**
  — issues one HTTP request, retrying up to `maxRetries` times on network
  errors or a response status in `retryableStatuses | {502, 503, 504}`
  (exponential backoff, capped at 4 seconds). Raises `VmiApiError` (with
  `statusCode`/`responseBody` set) on a non-2xx response after retries are
  exhausted, or after all retry attempts hit a network error. Returns `{}`
  for an empty body, or `{"raw": <text>}` if the body isn't valid JSON.
- **`get(path, **kwargs)` / `post(path, **kwargs)` / `put(path, **kwargs)` / `delete(path, **kwargs)`**
  — convenience wrappers around `request()` with the method fixed.
- **Tracing:** set `logging.getLogger("llm_sdk.api").setLevel(logging.INFO)`
  to log every request/response (bodies compact-truncated at 600 chars, so
  embeddings and long texts don't flood your logs).

### `PromptCache` (`llm_sdk/prompt_cache.py`)

A small, thread-safe TTL cache for `PromptDefinition`s, replacing an older
socket-push mechanism — prompts are pulled on first use and re-fetched after
`ttlSeconds`.

```python
PromptCache(loader: Callable[[str], PromptDefinition], ttlSeconds: float = 300.0)
```

- **`get(promptId: str) -> PromptDefinition`** — returns the cached
  definition if it's younger than `ttlSeconds`; otherwise calls `loader(promptId)`,
  caches the result with the current timestamp, and returns it.
- **`invalidate(promptId: Optional[str] = None) -> None`** — removes one
  cached entry, or clears the whole cache if `promptId` is `None`.

### `resolvePrompt` (`llm_sdk/prompt_resolver.py`)

```python
resolvePrompt(promptContent: str, inputParams: dict[str, Any] | None = None) -> str
```

Substitutes `{{name}}` placeholders in `promptContent` with values from
`inputParams`.

| Parameter | Type | Description |
| --- | --- | --- |
| `promptContent` | `str` | The template text, e.g. `"Answer this: {{question}}"`. |
| `inputParams` | `Optional[dict[str, Any]]` | Values to substitute. Non-string values (numbers, lists, dicts, booleans) are JSON-encoded before substitution; string values are inserted verbatim. |

- **Returns:** the template with all *known* placeholders replaced.
  Placeholders whose key isn't present in `inputParams` are left untouched
  (`{{unknownVar}}` stays as-is), so partially-resolved prompts remain
  visible in call logs for debugging rather than silently vanishing.

```python
resolvePrompt("Hello {{name}}, your order {{orderId}} is {{status}}.",
              {"name": "Asha", "orderId": 4821, "status": "shipped"})
# "Hello Asha, your order 4821 is shipped."
```

### Providers (`llm_sdk/providers/`)

#### `ProviderClient` (abstract base, `providers/base.py`)

The contract every vendor implementation follows. **Providers do model
invocation only** — no memory, telemetry, or logging; those concerns live
entirely in `LLMService`.

```python
class ProviderClient(ABC):
    def __init__(self, apiKey: str): ...

    @abstractmethod
    def call(self, model: str, systemPrompt: str, userMessage: str,
              temperature: float = 0.7, maxTokens: int = 2048) -> ProviderResult: ...
```

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `model` | `str` | — | Vendor-specific model id, e.g. `"gpt-4o"`, `"claude-sonnet-5"`, `"gemini-1.5-pro"`. |
| `systemPrompt` | `str` | — | The full system/instruction text. |
| `userMessage` | `str` | — | The user's message/prompt. |
| `temperature` | `float` | `0.7` | Sampling temperature. |
| `maxTokens` | `int` | `2048` | Max tokens to generate. |

- **Returns:** `ProviderResult`.
- **`_requireApiKey(providerName: str) -> str`** *(protected helper)* — returns
  the configured API key, or raises `ProviderError` if it's empty.

#### `OpenAIProvider` (`providers/openai_provider.py`)

```python
OpenAIProvider(apiKey: str, baseUrl: str | None = None, name: str = "OPENAI")
```

Calls the OpenAI Chat Completions API. Also reused for any
OpenAI-compatible vendor by passing a different `baseUrl`/`name` — this is
how DeepSeek support works (`baseUrl="https://api.deepseek.com"`).

- **`call(model, systemPrompt, userMessage, temperature=0.7, maxTokens=2048) -> ProviderResult`**
  — sends a two-message chat completion (`system` + `user`). Handles
  reasoning-style models (`gpt-5*`, `o1/o3/o4...`) that reject `max_tokens`
  and/or non-default `temperature`: on the corresponding `BadRequestError`,
  it automatically swaps `max_tokens` → `max_completion_tokens`, or drops
  `temperature`, and retries (up to 3 attempts total) — there's no reliable
  way to detect this by model name alone across OpenAI-compatible vendors.
  Raises `ProviderError` on any other failure.

#### `AnthropicProvider` (`providers/anthropic_provider.py`)

```python
AnthropicProvider(apiKey: str)
```

Calls the Anthropic Messages API.

- **`call(model, systemPrompt, userMessage, temperature=0.7, maxTokens=2048) -> ProviderResult`**
  — sends `system` as a top-level parameter and the user message as the
  single message in `messages`. Concatenates all `text`-type content
  blocks in the response into `ProviderResult.text`. Raises `ProviderError`
  if the `anthropic` package isn't installed, or if the API call fails.

#### `GeminiProvider` (`providers/gemini_provider.py`)

```python
GeminiProvider(apiKey: str)
```

Calls Google's Gemini API via `google-generativeai`.

- **`call(model, systemPrompt, userMessage, temperature=0.7, maxTokens=2048) -> ProviderResult`**
  — configures the client with `system_instruction=systemPrompt` and
  `generation_config={"temperature", "max_output_tokens"}`, then generates
  content from `userMessage`. Raises `ProviderError` if
  `google-generativeai` isn't installed, or if the API call fails.

#### `ProviderRegistry` (`providers/registry.py`)

```python
ProviderRegistry(config: VmiConfig)
```

Maps vmi-backend's `aiVendorCode` values to the right provider client, with
per-vendor instance caching.

- **`getProvider(aiVendorCode: str) -> ProviderClient`** — case-insensitive
  lookup. `OPENAI` → `OpenAIProvider`; `CLAUDE` or `ANTHROPIC` →
  `AnthropicProvider`; `GEMINI` or `GOOGLE` → `GeminiProvider`; `DEEPSEEK` →
  `OpenAIProvider` pointed at `https://api.deepseek.com`. Each provider
  instance is created once per vendor code and reused for subsequent calls.
  Raises `ProviderError` for any other code, listing the supported ones.

---

## Transactions

Every SDK function that touches the vector DB accepts a `transaction=`
argument. When set, the transaction's `tx_id` is included in the API hits of
**all child calls** (search, write, extract, initiate, complete), so a
whole `callLLM` — including any memory writes it performs — can be staged
and committed or rolled back as a single atomic unit.

```python
with client.transaction() as tx:                 # commit on exit, rollback on error
    client.addMemory("staged fact", transaction=tx)
    client.callLLM(rawPrompt="...", provider="OPENAI", model="gpt-4o",
                   saveInMemory=True, transaction=tx)

# or manually:
tx = client.beginTransaction()
client.transactionStatus(tx)
client.commitTransaction(tx)      # / client.rollbackTransaction(tx)
```

See [Transaction functions](#transaction-functions) for the full parameter
reference of each method involved.

## Sessions

```python
client.createSession("sess_001")
client.fetchMemory("...", storage="session", sessionId="sess_001")
client.closeSession("sess_001", flush=True)   # flush session memory to longterm
```

See [Session functions](#session-functions) for full parameter details and
the important non-idempotency warning on `createSession`.

---

## Architecture

```
llm_sdk/
├── client.py           VmiClient facade (entry point)
├── config.py            VmiConfig
├── models.py             typed results (MemoryData, LLMCallResult, VmiTransaction, ...)
├── exceptions.py       VmiError hierarchy
├── http_client.py      retries / timeouts / error mapping
├── backend_client.py   vmi-backend API client
├── vector_client.py    vmi-orchestrator API client (port of Java VectorDBClient)
├── memory_service.py   fetchMemory / addMemory / fetchMemoryFromConnectedDB
├── llm_service.py      callLLM orchestration (port of createProjectStep)
├── prompt_cache.py     TTL cache for prompt definitions
├── prompt_resolver.py  {{variable}} substitution
└── providers/          OpenAI, Anthropic, Gemini, DeepSeek (+ registry)
```

Design rules (unchanged from v1):

* **Providers do model invocation only** — no memory, telemetry or logging inside them.
* **Non-fatal periphery** — memory fetch, summarization, connected-DB context and
  call logging degrade gracefully; only the LLM call itself can fail the request.
* **Immutability** — all result models are frozen dataclasses.
* **Provider abstraction** — vendor codes match vmi-backend's `LLMClientFactory`
  (`OPENAI`, `CLAUDE`, `GEMINI`, `DEEPSEEK`).

---

## Test bench (web UI)

```bash
python -m sdk_test_app.cli          # opens http://127.0.0.1:7799
```

A browser console with four scenarios — **Coding Assistant**, **Personal
Educator**, **Chatbot**, **Enquiry System** — each a memory-backed chat with
its own background and storage mode. For every message the inspector shows
the whole pipeline: what was retrieved from memory, the exact system prompt,
the model output, the summary pushed back to memory, connected-DB context,
and the raw API wire trace. A **Function lab** exposes every SDK function
individually (memory CRUD, connected DB, backend prompts, transactions,
sessions), a coverage board tracks which functions have passed/failed, and
**Run full test** exercises all of them in order with a pass/fail report.

Configuration via `sdk_test_app/.env`. Note there's no backend/vector-DB URL
variable — those are hardcoded in `llm_sdk/config.py`, not read from `.env`:

```
VMI_ACCESS_TOKEN=...
VMI_VECTOR_TOKEN=ABCDEFG1234
VMI_COMPANY_CODE=WPCORP4812
VMI_BACKGROUND=You are a helpful assistant.
OPENAI_API_KEY=sk-...
```

## Tests

```bash
pip install pytest
python -m pytest tests/ -q
```
