Skip to content

fs-setup

Generate the connection boilerplate and run tdfs4ds.setup() / connect(). Invoked at the very start of every session.

Source file

The full skill specification (frontmatter + body) lives in tdfs4ds/skills/fs-setup/SKILL.md. The version below is included verbatim.


name: "Feature Store: Setup" description: "Generate code to connect to Teradata and initialise the tdfs4ds temporal feature store. Invoke when the user needs connection boilerplate, first-time setup, or wants to configure global feature store variables." argument-hint: "[schema][data-domain]" allowed-tools: "Read, Edit, Write, Bash, Glob, Grep"


Generate Python code to connect to a Teradata Vantage database and initialise the tdfs4ds temporal feature store.

User context: $ARGUMENTS

Next Skill When
→ fs-upload Engineer the first feature engineering view and register it.
→ fs-inspect Verify catalogs were provisioned and the data domain is active.
→ fs-document Once a process is registered, document it and score its EXPLAIN plan.

Demos: demos/notebooks/00 - Setup a temporal feature store.ipynb.

Ask for any missing values before generating code: - Schema / database name — Teradata database hosting the feature store (e.g. 'demo_db'). - Data domain — project namespace isolating features across teams (e.g. 'Customer Transaction Analytics').

Emit the following pattern, substituting real values for every <PLACEHOLDER>:

# .env (never hard-code credentials):
#   TDFS4DS_TD_HOST=<host>
#   TDFS4DS_TD_USERNAME=<user>          TDFS4DS_TD_PASSWORD=<password>
#   (token logon instead: TDFS4DS_TD_LOGDATA='token=<jwt>', TDFS4DS_TD_LOGMECH=JWT)
#   TDFS4DS_TD_DATABASE=<DATABASE>      # optional default database of the session
#   TDFS4DS_DATA_DOMAIN=..., TDFS4DS_INSTRUCT_MODEL_*, ...   (every other setting, prefixed TDFS4DS_)
import tdfs4ds                                  # loads the TDFS4DS_* settings of .env
tdfs4ds.reconnect_teradata(raise_error=True)    # opens the teradataml context from TDFS4DS_TD_*

# Not sure which database holds the store? List the ones this session can see:
# tdfs4ds.find_feature_stores()   # DATABASE_NAME, VIEW_DATABASE, COMPLETE, MISSING,
#                                 # READABLE, N_PROCESSES, N_DATA_DOMAINS, ERROR
tdfs4ds.connect(database='<DATABASE>')
import teradataml as tdml

# Optional: keep VIEWS in a separate database from TABLES (governance policy).
# Omit database_view (or leave TDFS4DS_SCHEMA_VIEW unset) to co-locate them — the
# default, identical to before. When set, the view database must be granted
# SELECT (WITH GRANT OPTION) on the table database.
# tdfs4ds.connect(database='<DATABASE>', database_view='<VIEW_DB>')

# Optional overrides — can also be set via TDFS4DS_* in .env or in tdfs4ds.json
# tdfs4ds.FEATURE_STORE_TIME = None     # valid time of the values a run writes (and of catalog reads).
#                                       # None = real time. Prefer driving runs with a TimeManager
#                                       # (run(pid, time_manager=tm), roll_out) - see fs-rollout.
# tdfs4ds.set_domain_time_manager(tm)   # 0.3.6.0: one clock for the data domain (optional table
#                                       # FS_DATA_DOMAIN_SETTINGS, created by setup/connect) - see fs-rollout
# tdfs4ds.DISPLAY_LOGS       = True
# tdfs4ds.DEBUG_MODE         = False
# tdfs4ds.STORE_FEATURE      = 'MERGE'  # keep 'MERGE': 'UPDATE_INSERT' is broken

# Vector store backend for the consumer agent (optional — default is Chroma)
# tdfs4ds.VECTOR_STORE_BACKEND         = 'chroma'     # 'chroma' (default) | 'teradata' (keeps index in Vantage)
# tdfs4ds.TERADATA_VECTOR_STORE_PREFIX = 'tdfs4ds'    # prefix for 3 named stores in Vantage

# TeradataAI in-database embedding (only used when VECTOR_STORE_BACKEND='teradata')
# When None, reuses the standard get_embeddings() (vLLM/TEI/OpenAI-compatible).
# tdfs4ds.TERADATA_AI_API_TYPE   = None   # 'aws'|'azure'|'gcp'|'nim'|'hugging_face'|'onnx'
# tdfs4ds.TERADATA_AI_MODEL_NAME = None
# tdfs4ds.TERADATA_AI_REGION     = None   # aws only
# tdfs4ds.TERADATA_AI_API_BASE   = None   # nim only
# tdfs4ds.TERADATA_AI_ACCESS_KEY = None   # or TDFS4DS_TERADATA_AI_ACCESS_KEY env var

# First-time / administrator only (idempotent): deploy catalog tables and views.
# Add database_view='<VIEW_DB>' to place every view in a separate database from the tables.
tdfs4ds.setup()

# Create the data domain (idempotent — warns if already present).
# Analytics notebooks activate it with select_data_domain instead.
tdfs4ds.create_data_domain('<DATA_DOMAIN>')

Pattern — write the configuration, then prove it

Do not hand-write a .env on Teradata AI Studio: the platform publishes most of it, and the values that must be asked for are the ones easiest to get subtly wrong.

import tdfs4ds

tdfs4ds.detect_ai_studio()                       # what the platform exposes
tdfs4ds.generate_ai_studio_env('.env', pin=True) # derive, ask, CHECK, then write

It asks only what it cannot derive (how to authenticate, which route LLM calls take, the API key and model), stages the result into the session, runs the full check against it, and refuses to write a file whose checks fail. On a token logon it also attempts the database logon first — the one credential the configuration check deliberately leaves alone, and the one this function writes.

If %gen_token is available, pass its token: it is a Vantage credential, where the session's OAUTH_ACCESS_TOKEN may not be.

%gen_token
tdfs4ds.generate_ai_studio_env('.env', token=user_token, llm='ingress')

Anywhere else, answer a few questions and let the wizard update the file you already have: tdfs4ds.configure_interactively('.env'). It edits .env line by line (comments and unrelated variables survive), lists the models the endpoint serves (boto3 for Bedrock), asks for an API key only when a keyless attempt is rejected, offers Chroma if the Teradata Vector Store is unavailable, and writes secrets only after an explicit yes. To snapshot a session that already works instead: tdfs4ds.export_config(fmt='env', source='module').

Verify before building anything — this needs no feature store, so run it first:

report = tdfs4ds.check_configuration()          # + selftest=True to prove indexing WRITES

Nine checks, each naming the setting to fix: config load, Vector Store auth (with the token's remaining lifetime), Vector Store service, LLM transport/TLS, instruct model, JSON constraints and tool calling, embedding model, MCP server, and an opt-in write-path self-test. tdfs4ds.describe_config() answers the narrower question "is this variable actually set?" without printing any secret.

Rules

  • On Teradata AI Studio, start a notebook with one call (0.3.5.17):

    import tdfs4ds
    tdfs4ds.open_session()        # the nearest .env: this folder, its parents, then ~/.tdfs4ds
    
    It loads the nearest .env (and says which), takes the platform's current token (or token=user_token after %gen_token) and puts it wherever the file uses a token (database logon, Vector Store, MCP bearer), opens the teradataml context, connects to TDFS4DS_TD_DATABASE and renews the token in the background. connect=False stops after the logon (then tdfs4ds.setup(database=...) for a new store). End the notebook with tdml.remove_context().

  • Open the session with tdfs4ds.reconnect_teradata(raise_error=True): it reads TDFS4DS_TD_* (password or token logon), and the package's automatic reconnection reuses the same variables, so a dropped connection comes back as the same user. tdml.create_context(...) by hand also works; open it before any call that reads the database.

  • Use tdfs4ds.create_data_domain(name) in the setup notebook (creates the domain, idempotent). Use tdfs4ds.select_data_domain(name) in all analytics/feature notebooks (activates an existing domain; raises ValueError if not found). Never set tdfs4ds.DATA_DOMAIN directly.
  • Never hard-code credentials; always use a .env file or OS environment variables.
  • tdfs4ds auto-loads TDFS4DS_* variables from .env and tdfs4ds.json at import time. Call tdfs4ds.load_config(path=..., dotenv_path=...) to reload or point at a custom file.
  • tdfs4ds.setup(database=...) is idempotent but is an administrator action — note this in a comment. Never write if_exists='replace': it drops the feature, process and dataset catalogs of that database before recreating them, so every feature registered there is lost, other users' included. The default 'fail' creates only what is missing.
  • When the user does not know the database, call tdfs4ds.find_feature_stores() before connect(): one read of DBC.TablesV (only what the session may see) plus one count per store. COMPLETE=False means connect() would return False (MISSING names the tables); READABLE=False means the catalog view refused the count (ERROR has the whole message). complete_only=True keeps the stores connect() accepts; details=False skips the counts.
  • connect() works on a store the user may only read: refreshing the catalog views needs the DROP right, and when it is refused the existing views are kept with a warning (0.3.2.4; before, it raised).
  • To separate views from tables, pass database_view='<VIEW_DB>' to setup()/connect() (or set TDFS4DS_SCHEMA_VIEW). Default (None) co-locates views with tables in database — no behaviour change. The view database needs SELECT ... WITH GRANT OPTION on the table database, and an unqualified process/dataset name then defaults to the view database.