tdfs4ds.agent.embedding¶

tdfs4ds.agent.embedding ¶

Embedding model factory for the tdfs4ds consumer agent.

Uses dedicated EMBEDDING_MODEL_* configuration when set, otherwise falls back to INSTRUCT_MODEL_* variables:

'openai' -> langchain_openai.OpenAIEmbeddings
'bedrock' -> langchain_aws.BedrockEmbeddings (amazon.titan-embed-text-v2:0)
'mistralai' -> langchain_mistralai.MistralAIEmbeddings

Typical LiteLLM setup (vLLM for instruct, TEI for embeddings behind a proxy)::

tdfs4ds.INSTRUCT_MODEL_URL  = 'https://my-proxy/v1'       # chat completions
tdfs4ds.EMBEDDING_MODEL_URL = 'https://my-proxy/v1/e5'    # embeddings (custom sub-path)

AWS Bedrock setup requires AWS credentials and region configuration::

tdfs4ds.EMBEDDING_MODEL_PROVIDER = 'bedrock'
# Set AWS_REGION or AWS_DEFAULT_REGION environment variable
# AWS credentials can be provided via AWS_PROFILE, IAM role, or ~/.aws/credentials

logger `module-attribute` ¶

logger = getLogger('tdfs4ds')

_proxy_http_client `module-attribute` ¶

_proxy_http_client = _build_http_client

_canonical_provider ¶

_canonical_provider(provider)

aws -> bedrock, lower-cased (see utils.llm_provider).

_resolve_proxy ¶

_resolve_proxy(*, for_embedding: bool = False) -> 'str | None'

Return the configured outbound-proxy URL for LLM/embedding HTTP calls.

Embedding calls use EMBEDDING_MODEL_PROXY and fall back to INSTRUCT_MODEL_PROXY; instruct calls use INSTRUCT_MODEL_PROXY only. Returns None (direct connection) when nothing is set.

_resolve_tls ¶

_resolve_tls(*, for_embedding: bool = False)

Return (verify_ssl, ca_bundle) for the instruct model's HTTP calls.

TLS options are instruct-only: an embedding client is never affected by them, whatever the instruct model is configured to do. (True, None) means "use httpx's defaults" — i.e. nothing to customise.

_build_verify_argument ¶

_build_verify_argument(verify_ssl, ca_bundle)

Translate the TLS settings into httpx's verify argument.

False disables verification outright; a CA bundle becomes an SSLContext (httpx deprecates verify=<str>), built eagerly so an unreadable or malformed file fails here — naming the variable — instead of as an opaque [X509] PEM lib error on the first request. A bad bundle is never downgraded to default verification: that would weaken TLS invisibly.

_build_http_client ¶

_build_http_client(*, for_embedding: bool = False, is_async: bool = False, auth=None)

Build the httpx client for OpenAI-family calls, or None.

Returns a client when any of the outbound proxy, TLS verification or CA bundle needs customising, and None otherwise — which lets the OpenAI-family constructors use their own default client. That None is the backward-compatibility guarantee: with nothing configured, no http_client argument is passed at all, exactly as before these options existed.

The proxy URL's scheme selects the protocol (http:// or socks5://; SOCKS5 requires the httpx[socks] extra). Proxy and TLS settings compose into one client rather than competing.

Only OpenAI-family providers accept an http_client; Bedrock/GCP use their own SDK mechanisms.

http_get ¶

http_get(url, *, headers=None, timeout=30.0, for_embedding=False)

GET url through the same proxy and TLS settings as the model calls.

list_instruct_models / list_embedding_models talk to the very same endpoint as the chat and embedding clients, but used a bare httpx.get — so behind an internal certificate authority they failed with a TLS error while the model itself worked, and INSTRUCT_MODEL_PROXY did not apply to them at all. Anything in this package that reaches the model endpoint over plain HTTP should come through here.

Returns the httpx.Response (raise_for_status is the caller's).

_inject_provider_env ¶

_inject_provider_env(*, for_embedding: bool = False) -> None

Inject tdfs4ds config variables into the process environment so that LangChain / boto3 / google-cloud clients pick them up automatically.

The active provider is read from

EMBEDDING_MODEL_PROVIDER (when for_embedding=True) → falls back to INSTRUCT_MODEL_PROVIDER
INSTRUCT_MODEL_PROVIDER (when for_embedding=False)

Resolution order for setting values

EMBEDDING_MODEL_ → INSTRUCT_MODEL_ (when for_embedding=True)
INSTRUCT_MODEL_* only (when for_embedding=False)

get_embeddings ¶

get_embeddings() -> Embeddings

Return a LangChain Embeddings instance based on the active provider.

Resolution order for each setting: - EMBEDDING_MODEL_PROVIDER -> falls back to INSTRUCT_MODEL_PROVIDER - EMBEDDING_MODEL_API_KEY -> falls back to INSTRUCT_MODEL_API_KEY - EMBEDDING_MODEL_URL -> falls back to INSTRUCT_MODEL_URL - EMBEDDING_MODEL_MODEL -> used as-is (default: text-embedding-3-small)

Injects tdfs4ds config variables into environment so that boto3, Azure, and Google Cloud clients pick up credentials and endpoints automatically.

Returns:

Name	Type	Description
`Embeddings`	`Embeddings`	A configured LangChain embedding model.

Raises:

Type	Description
`ValueError`	If the resolved provider is not supported.

get_vector_store_embedding ¶

get_vector_store_embedding()

Return the embedding object for the active vector-store backend.

Selection rule

tdfs4ds.TERADATA_AI_API_TYPE set -> teradatagenai.TeradataAI(...) (in-database embedding; used by the 'teradata' backend).
otherwise -> get_embeddings() (LangChain Embeddings; works with vLLM / TEI / OpenAI-compatible setups and is also accepted as embedding= by TeradataVectorStore).

The ChromaDB backend always uses get_embeddings() regardless of this setting — TeradataAI is a Vantage-side construct and is only meaningful when the vector store also lives in Vantage.

list_embedding_models ¶

list_embedding_models(sub_paths: list[str] | None = None) -> 'pd.DataFrame'

List available embedding models from the configured provider.

For LiteLLM setups with multiple TEI backends (e.g. /v1/e5, /v1/code), pass the sub-path names to query each one::

list_embedding_models(sub_paths=['e5', 'code'])

Each sub-path is appended to INSTRUCT_MODEL_URL and queried for /models. Results are merged into a single DataFrame with a SUB_PATH column.

When sub_paths is None, the function derives the sub-path from EMBEDDING_MODEL_URL (if set) or queries INSTRUCT_MODEL_URL directly.

Parameters:

Name	Type	Description	Default
`sub_paths`	`list[str] \| None`	List of LiteLLM route prefixes to query (e.g. `['e5', 'code']`).	`None`

Returns:

Type	Description
`'pd.DataFrame'`	DataFrame with columns: MODEL_ID, OWNED_BY, SUB_PATH (OpenAI-compatible) or
`'pd.DataFrame'`	MODEL_ID, PROVIDER, MODEL_NAME (Bedrock).

Raises:

Type	Description
`ValueError`	If the provider is not recognized or not configured.

tdfs4ds.agent.embedding¶