Skip to content

fs-agent

Launch the consumer agent chatbot — embedding setup, Chroma index build, Gradio UI launch.

name: "Feature Store: Agent" description: "Launch the tdfs4ds consumer agent chatbot — configure embedding models, build the vector index from the business dictionary, test the agent, and start the Gradio UI. Invoke when the user wants a conversational interface for business consumers to ask natural-language questions about features, datasets, definitions, freshness, lineage, or how features are calculated. Supports Chroma (local/server) and Teradata Enterprise Vector Store as vector index backends." argument-hint: "[embedding-provider][vector-backend: chroma|teradata] [chroma-mode: local|server]" allowed-tools: "Read, Edit, Write, Bash, Glob, Grep"

Generate Python code to set up and launch the tdfs4ds consumer agent — a conversational chatbot that lets business users ask natural-language questions about features, datasets, definitions, data freshness, lineage, and how features are calculated. The agent runs as a LangGraph state machine over an embedding-backed vector index of the business dictionary.

User context: $ARGUMENTS

Step Skill Why
Before fs-setup Establish the Teradata connection and run tdfs4ds.setup() once if catalogs are not yet provisioned.
Before fs-document Populate the business dictionary — the agent answers only from documented objects. Run document_process / document_dataset_incremental / document_feature_store_incremental first.
Before fs-vector-store Dedicated skill for configuring/switching/troubleshooting the vector index backend (Chroma vs Teradata Enterprise Vector Store) in isolation.
After fs-inspect If the agent says "no features found", inspect the feature catalog and the follow-up table to confirm what is actually registered.

Demos: demos/notebooks/G2 - Consumer agent.ipynb, G3 - Admin app.ipynb.

Prerequisites

The agent answers only from the business dictionary — three temporal tables populated by:

  • document_process / document_process_incremental (see fs-document)
  • document_dataset_incremental
  • document_feature_store_incremental
  • upload_business_dictionary_objects / upload_business_dictionary_columns / upload_business_dictionary_sections

If the dictionary is empty, run documentation first — the agent will not invent answers.

Ask for any missing values before generating code:

  • Embedding provider — 'openai', 'vllm', 'azure', or 'bedrock'. Determines how vectors are computed.
  • Vector backend — 'chroma' (default, external store) or 'teradata' (keeps the index inside Vantage; requires langchain_teradata).
  • Chroma mode — 'local' (persist to disk under CHROMA_PATH) or 'server' (connect to a running Chroma instance via CHROMA_HOST/CHROMA_PORT). Only relevant when VECTOR_STORE_BACKEND='chroma'.

Pattern

import tdfs4ds                                  # reads the TDFS4DS_* settings of .env
tdfs4ds.reconnect_teradata(raise_error=True)    # teradataml context from TDFS4DS_TD_HOST / _USERNAME / _PASSWORD
tdfs4ds.connect(database='<DATABASE>', create_if_missing=True)
import teradataml as tdml
tdfs4ds.select_data_domain('<DATA_DOMAIN>')   # or set TDFS4DS_DATA_DOMAIN in .env

# ---------------------------------------------------------------------------
# Instruct model — intent classification + answer synthesis
# Preferred: set TDFS4DS_INSTRUCT_MODEL_* in .env — auto-loaded at import.
# ---------------------------------------------------------------------------
tdfs4ds.INSTRUCT_MODEL_PROVIDER = '<PROVIDER>'                 # 'openai' | 'vllm' | 'azure' | 'bedrock'
tdfs4ds.INSTRUCT_MODEL_URL      = '<MODEL_URL>'      # None for default OpenAI / Bedrock
# API key: keep it in .env as TDFS4DS_INSTRUCT_MODEL_API_KEY (never in code)
tdfs4ds.INSTRUCT_MODEL_MODEL    = '<MODEL_NAME>'

# ---------------------------------------------------------------------------
# Embedding model — for the Chroma vector index
# Falls back to INSTRUCT_MODEL_* if EMBEDDING_MODEL_* is not set.
# ---------------------------------------------------------------------------
tdfs4ds.EMBEDDING_MODEL_PROVIDER = '<EMBEDDING_PROVIDER>'      # 'openai' | 'vllm' | 'azure' | 'bedrock'
tdfs4ds.EMBEDDING_MODEL_URL      = '<MODEL_URL>'    # e.g. 'https://api.example.com/v1/e5'
# API key: keep it in .env as TDFS4DS_EMBEDDING_MODEL_API_KEY (never in code)
tdfs4ds.EMBEDDING_MODEL_MODEL    = '<EMBEDDING_MODEL>'         # e.g. 'text-embedding-3-small'
tdfs4ds.EMBEDDING_MODEL_DIM      = 1536

# Agent answer verbosity
tdfs4ds.AGENT_ANSWER_STYLE = 'standard'                        # 'concise' | 'standard' | 'verbose'
tdfs4ds.AGENT_LIST_MAX_ITEMS = 40                              # above this, list answers become an overview + "narrow it down" (0 = list all)

# ---------------------------------------------------------------------------
# Vector store — Option A: Chroma (default)
# ---------------------------------------------------------------------------
tdfs4ds.VECTOR_STORE_BACKEND = 'chroma'                        # default
tdfs4ds.CHROMA_MODE = 'local'                                  # 'local' | 'server'
tdfs4ds.CHROMA_PATH = './tdfs4ds_chroma'                       # disk path (local mode)
# tdfs4ds.CHROMA_HOST = 'localhost'                            # server mode only
# tdfs4ds.CHROMA_PORT = 8000                                   # server mode only

# ---------------------------------------------------------------------------
# Vector store — Option B: Teradata Enterprise Vector Store
#   Keeps all three index collections (objects, sections, columns) inside Vantage.
#   Requires: pip install langchain_teradata
# ---------------------------------------------------------------------------
# tdfs4ds.VECTOR_STORE_BACKEND         = 'teradata'
# tdfs4ds.TERADATA_VECTOR_STORE_PREFIX = 'tdfs4ds'   # creates tdfs4ds_objects, tdfs4ds_sections, tdfs4ds_columns
#
# Optional: TeradataAI in-database embedding (omit to reuse EMBEDDING_MODEL_* above)
# tdfs4ds.TERADATA_AI_API_TYPE   = 'azure'            # 'aws'|'azure'|'gcp'|'nim'|'hugging_face'|'onnx'
# tdfs4ds.TERADATA_AI_MODEL_NAME = 'text-embedding-3-small'
# tdfs4ds.TERADATA_AI_REGION     = None               # aws only
# tdfs4ds.TERADATA_AI_API_BASE   = None               # nim only
# tdfs4ds.TERADATA_AI_ACCESS_KEY = None               # or TDFS4DS_TERADATA_AI_ACCESS_KEY env var

# ---------------------------------------------------------------------------
# 1. Verify the business dictionary has content (sanity check)
# ---------------------------------------------------------------------------
objects = tdml.execute_sql(
    f"""
    CURRENT VALIDTIME
    SELECT OBJECT_NAME, OBJECT_TYPE,
           BUSINESS_DESCRIPTION (VARCHAR(100)) AS BUSINESS_DESCRIPTION
    FROM {tdfs4ds.SCHEMA}.{tdfs4ds.DOCUMENTATION_BUSINESS_DICTIONARY_OBJECTS}
    ORDER BY OBJECT_TYPE, OBJECT_NAME
    """
).fetchall()

print(f"Business dictionary: {len(objects)} object(s)")
for row in objects[:10]:
    print(f"  [{row[1]}] {row[0]:40s}  {(row[2] or '')[:60]}")

# ---------------------------------------------------------------------------
# 2. Build (or rebuild) the Chroma vector index
#    force_rebuild=False = incremental — only new/changed docs are embedded.
# ---------------------------------------------------------------------------
from tdfs4ds.agent import build_vector_index

build_vector_index(force_rebuild=False)

# ---------------------------------------------------------------------------
# 3. Smoke-test the agent programmatically
# ---------------------------------------------------------------------------
from tdfs4ds.agent import consumer_agent

print(consumer_agent("What features are available?", thread_id="test-1"))

# ---------------------------------------------------------------------------
# 4. Launch the Gradio chatbot (one-command equivalent: launch_chatbot_with_index)
# ---------------------------------------------------------------------------
from tdfs4ds.agent import launch_chatbot

demo = launch_chatbot(
    title="Feature Store Assistant",
    description=(
        "Ask questions about available features, datasets, definitions, "
        "data freshness, lineage, or how a feature is calculated — in French or English."
    ),
    port=7860,
    # public_base_url='https://my-jupyterhub.example.com',  # if behind a proxy
)

# tdml.remove_context()   # leave open while the chatbot is running

Chart rendering

When teradata-mcp-server plot_* tools return Chart.js JSON, the chatbot automatically converts it to a Plotly figure and renders it: - Inline in the chat bubble (the Chatbot component uses sanitize_html=False) - In a collapsible "Visualizations" panel below the chat

Only the Chart.js path is handled — chart payloads are recovered from ToolMessage.artifact (structured_content). Generic ui:// MCP-UI resources are not rendered.

Agent intents (consumer-agent skills)

The agent's skill set is data-driven — each skill is defined by a ca-* SKILL.md file in tdfs4ds/skills/. Removing a file removes that intent from the agent; the available skills are loaded at graph construction time.

Skill file Intent Trigger example
ca-search SEARCH "What features measure customer spending?"
ca-definition DEFINITION "What is total_amount?"
ca-usage USAGE "How do I use avg_amount in a model?"
ca-freshness FRESHNESS "Is total_amount up to date?"
ca-summary SUMMARY "Give me a snapshot of the feature store"
ca-lineage LINEAGE "Where does total_amount come from?"
ca-explain EXPLAIN "How is total_amount calculated?"
ca-dataset DATASET "Which dataset exposes total_amount?"
ca-build-dataset BUILD_DATASET "Build a dataset for churn analysis"
ca-data-query DATA_QUERY "Show me 20 rows from DS_FRAUD" (MCP-gated: only active when mcp_enabled=True and MCP_SERVER_URL is set)
(built-in) RECAP "What features were we discussing?"
(built-in) OUT_OF_SCOPE "What is the weather today?"

No MCP server available? Start the Community teradata-mcp-server locally with the tdfs4ds credentials (TDFS4DS_TD_HOST / _USERNAME / _PASSWORD, TD2 or LDAP — not a token logon). It sets MCP_SERVER_URL for you:

# %pip install teradata-mcp-server   (not a tdfs4ds dependency; pipx works too)
tdfs4ds.start_mcp_server()                    # 127.0.0.1:8001, profile 'eda'
tdfs4ds.launch_app("tdfs4ds.env", mcp=True)   # or: the admin app and its MCP server together

MCP settings (TDFS4DS_* in .env):

  • MCP_SERVER_EDITION: 'community' (default; the server holds its own database credentials, tdfs4ds sends none) or 'enterprise' (needs caller credentials: MCP_USERNAME / MCP_PASSWORD, falling back to the TD_* ones, or MCP_SERVER_HEADERS for a bearer token; a different tool set).
  • MCP_MAX_TOOLS (default 32): above it the tools bound to the model are ranked by the question (an Enterprise server exposes over 100, whose schemas overflow the context). On a context-window error the agent shows the model a one-line index of the tools and retries once with the ones it picks (MCP_TOOL_DISCOVERY, MCP_FALLBACK_MAX_TOOLS).
  • MCP_SERVER_CA_BUNDLE for a server behind an internal CA (verification stays on).
  • tdfs4ds.check_configuration() lists the tools found and runs one tool call through the model.

launch_app() checks the database settings first (host, username, password or a JWT token, TDFS4DS_TD_ENCRYPT), resolved as the app reads them, and tests the logon. Missing or refused: it asks (type them, use another .env found, or start without them) and offers to save typed values in a .env (default ~/.tdfs4ds/.env; the password confirmed separately). tdfs4ds.connection_settings(env_file) shows what the app would use, secrets hidden; ask=False never asks. When TDFS4DS_SCHEMA is not set it lists the feature stores visible with that logon and asks which to open (or start without one: Parameters), and it checks that the query band can be set and DBQL read (a note only: without it the Execution log shows no CPU / I/O). In the app, Parameters › Execution log: query band and history switches QUERY_BAND / EXECUTION_RESOURCES.

No feature store selected (python -m tdfs4ds.serve --env <file> asks nothing): the connection comes from TDFS4DS_TD_*, the store from TDFS4DS_SCHEMA, a separate setting. Before 0.3.3.28 a missing TDFS4DS_SCHEMA made every panel fail with Error 3707 "expected something like a name … between the 'FROM' keyword and '.'" (the SQL read FROM .FS_FEATURE_CATALOG), and Parameters showed the only store found as if it were chosen. The status bar's Feature store — is the tell. Fix: Parameters › choose the store › Apply stores, or set TDFS4DS_SCHEMA in the --env file and restart. Since 0.3.3.28 a logon whose default database holds a complete store (Connect page Default database, or TDFS4DS_TD_DATABASE) selects it, the startup log warns when none is set, and every panel says "No feature store is selected" instead of running SQL.

To restrict the agent to a subset of skills, pass the skills= parameter:

consumer_agent("What features are available?", skills=["SEARCH", "SUMMARY"])

To discover installed consumer-agent skills programmatically:

import tdfs4ds
tdfs4ds.consumer_agent_skill_catalog()   # {skill_name: {intent, description, ...}}

Multi-turn memory

The agent uses LangGraph with a process-level MemorySaver checkpointer (singleton — never created per call). Pass the same thread_id to maintain context:

consumer_agent("What datasets are available?",          thread_id="session-1")
consumer_agent("Tell me more about the first one",      thread_id="session-1")
consumer_agent("How is the first feature calculated?", thread_id="session-1")
consumer_agent("When was it last updated?",             thread_id="session-1")  # entity + feature remembered

Remembered state spans:

  • resolved_data_domain — domain that owns the current feature
  • resolved_object_name — last explicitly named feature / dataset
  • resolved_feature_triplet — (feature_name, entity_name, process_id, view_name)
  • resolved_entity_name, resolved_feature_list — entity + feature group in focus
  • resolved_column_sources — column→source-table map from the last EXPLAIN answer (enables drilling into any column with "what is <col>?")

Vector index details

build_vector_index populates three Chroma collections from the business dictionary:

  • objects — table / view / dataset overviews
  • sections — OVERVIEW, ENTITY, FEATURE_THEMES, BUSINESS_QUESTIONS, INTENDED_AUDIENCE (datasets)
  • columns — per-column business descriptions

Documents are embedded in batches of 8 to respect TEI / OpenAI rate limits. force_rebuild=False is incremental — re-run it after every documentation pass.

Rules

  • Follow-ups on a list (0.3.5.18): after a listing answer ("what datasets are available?"), a question that points without naming ("is it up to date?", "give me a sample") uses the listed items: one is answered directly, several are offered as numbered choices (reply with the number). To test a conversation, keep one thread_id for the whole exchange; a new thread starts with nothing listed.

  • The business dictionary must be populated before build_vector_index. Run fs-document first or call document_feature_store_incremental(upload=True) to cover the entire store in one pass.

  • All TDFS4DS_* config can come from .env or tdfs4ds.json; explicit assignment always wins.
  • For 'bedrock', no URL / API_KEY is needed — boto3 default credentials are used.
  • The agent never invents answers — if the dictionary lacks coverage, fix the documentation, then re-run build_vector_index(force_rebuild=False).
  • The graph is a process-level singleton: do not wrap the agent in a fresh StateGraph(...).compile(MemorySaver()) per call — every cross-turn state would be lost. Use consumer_agent(...) directly.
  • launch_chatbot_with_index(port=..., force_rebuild=False) is the single-call equivalent of steps 2–4.