Skip to content

Running the Consumer Agent with AWS Bedrock (no Docker)

This guide covers launching the tdfs4ds consumer agent chatbot on a remote machine accessed via VS Code SSH, using AWS Bedrock for both the instruct and embedding models. No Docker or separate Chroma server is required.

Prerequisites

  • Python environment with network access to your Teradata Vantage instance
  • AWS Bedrock access with a temporary Bearer token (generated from the AWS console)
  • VS Code with the Remote – SSH extension and port forwarding available

Step 1 — Install the package

pip install tdfs4ds
pip install langchain-aws boto3

langchain-aws and boto3 are required for Bedrock support and are not always pulled in automatically on all platforms.

Step 2 — One-time prerequisite setup

These steps only need to be run once when setting up a feature store that already has registered processes and datasets but has not yet been initialised for the agent. Skip any step that has already been completed.

3a — Connect with create_if_missing=True

This creates the documentation and lineage tables if they do not yet exist (idempotent):

import teradataml as tdml
import tdfs4ds

tdml.create_context(
    host="<teradata-host>",
    username="<user>",
    password="<password>",
)

tdfs4ds.SCHEMA      = "<your_schema>"
tdfs4ds.DATA_DOMAIN = "<your_data_domain>"

tdfs4ds.connect(create_if_missing=True)

3b — Upload the business dictionary (optional)

If you have object-level or column-level descriptions to seed the agent with, upload them now. All three functions accept a pandas DataFrame and merge into the temporal business dictionary tables.

import pandas as pd

# Object-level descriptions (tables, views, datasets)
objects_df = pd.DataFrame([
    {
        "DATABASE_NAME":        "<your_schema>",
        "OBJECT_NAME":          "MY_FEATURE_VIEW",
        "OBJECT_TYPE":          "V",          # 'V' = view, 'T' = table, 'D' = dataset
        "BUSINESS_DESCRIPTION": "Describes ...",
    },
    # add more rows as needed
])
tdfs4ds.upload_business_dictionary_objects(objects_df)

# Column-level descriptions
columns_df = pd.DataFrame([
    {
        "DATABASE_NAME":        "<your_schema>",
        "TABLE_NAME":           "MY_FEATURE_VIEW",
        "COLUMN_NAME":          "MY_FEATURE",
        "BUSINESS_DESCRIPTION": "Describes ...",
    },
])
tdfs4ds.upload_business_dictionary_columns(columns_df)

3c — Rebuild the lineage graph

Derives all dependency edges from every registered process view and dataset and persists them into FS_LINEAGE_GRAPH. This is required before generating documentation:

tdfs4ds.rebuild_lineage_graph(all_data_domains=True)

3d — Rebuild documentation

Walks the persisted lineage graph bottom-up and generates LLM documentation for every process view and dataset that has not yet been documented:

import os
os.environ["AWS_DEFAULT_REGION"] = "us-east-1"   # replace with your Bedrock region

tdfs4ds.INSTRUCT_MODEL_PROVIDER = "bedrock"
tdfs4ds.INSTRUCT_MODEL_REGION   = "us-east-1"
tdfs4ds.INSTRUCT_MODEL_API_KEY  = "<your-temporary-bearer-token>"
tdfs4ds.INSTRUCT_MODEL_MODEL    = "anthropic.claude-3-5-sonnet-20240620-v1:0"

tdfs4ds.rebuild_documentation(force_update=False, all_data_domains=True)

force_update=False skips objects that already have stored documentation. Set it to True to regenerate everything from scratch.

Step 4 — Write a launch script

Create a file launch_agent.py (or use a notebook cell):

import os
import teradataml as tdml
import tdfs4ds

os.environ["AWS_DEFAULT_REGION"] = "us-east-1"   # replace with your Bedrock region

# ── 1. Connect to Teradata ──────────────────────────────────────────────────
tdml.create_context(
    host="<teradata-host>",
    username="<user>",
    password="<password>",
)

# ── 2. Target schema and data domain ────────────────────────────────────────
tdfs4ds.SCHEMA      = "<your_schema>"
tdfs4ds.DATA_DOMAIN = "<your_data_domain>"

# ── 3. Instruct model (Bedrock) ─────────────────────────────────────────────
tdfs4ds.INSTRUCT_MODEL_PROVIDER = "bedrock"
tdfs4ds.INSTRUCT_MODEL_REGION   = "us-east-1"   # must match AWS_DEFAULT_REGION
tdfs4ds.INSTRUCT_MODEL_API_KEY  = "<your-temporary-bearer-token>"  # from AWS console
tdfs4ds.INSTRUCT_MODEL_MODEL    = "anthropic.claude-3-5-sonnet-20240620-v1:0"

# ── 4. Embedding model (Bedrock Titan) ──────────────────────────────────────
# EMBEDDING_MODEL_API_KEY falls back to INSTRUCT_MODEL_API_KEY automatically.
tdfs4ds.EMBEDDING_MODEL_PROVIDER = "bedrock"
tdfs4ds.EMBEDDING_MODEL_REGION   = "us-east-1"
tdfs4ds.EMBEDDING_MODEL_MODEL    = "amazon.titan-embed-text-v2:0"
tdfs4ds.EMBEDDING_MODEL_DIM      = 1024   # Titan v2 native dimension

# ── 5. Local Chroma vector index ────────────────────────────────────────────
tdfs4ds.CHROMA_PATH = "./tdfs4ds_chroma"   # persisted locally on the remote machine

# ── 6. Launch ───────────────────────────────────────────────────────────────
tdfs4ds.launch_chatbot_with_index(
    port=7860,
    inline=False,   # set to True only inside a Jupyter notebook
    share=False,    # VS Code port forwarding handles external access
)

Step 5 — Forward the port in VS Code

  1. Open the Ports panel in VS Code (bottom bar → Ports tab, or View → Ports).
  2. Click Forward a Port and enter 7860.
  3. Open http://localhost:7860 in your local browser.

The chatbot is now accessible from your local machine via the SSH tunnel — no public URL or share=True needed.

Notes

Temporary credentials expire. Bearer tokens from the AWS console are short-lived. When they expire, Bedrock calls will fail with an authentication error. Restart the script with a fresh token — there is no automatic refresh.

Titan v2 embedding dimensions. amazon.titan-embed-text-v2:0 supports dimensions 256, 512, and 1024 (default). If you previously built a Chroma index with a different model or dimension (e.g. 1536 from OpenAI text-embedding-3-small), delete the ./tdfs4ds_chroma folder before the first run to avoid a dimension mismatch error.

Feature store already initialised? If the feature store tables already exist in SCHEMA you can skip tdfs4ds.setup(). If they do not exist yet, call it once before launching:

tdfs4ds.setup()

inline=True only works inside a Jupyter notebook. For a plain .py script use inline=False.