The admin app from a notebook (SageMaker, JupyterHub, AI Studio)¶

tdfs4ds.launch_app() starts the admin app from a notebook cell and shows a link to open it.

import tdfs4ds
tdfs4ds.launch_app("/home/ec2-user/SageMaker/tdfs4ds.env")   # a button: "Open the tdfs4ds app"
tdfs4ds.app_status()                                          # running? its URL, log and connection
tdfs4ds.stop_app()

How it works¶

A separate process. The app runs as python -m tdfs4ds.serve in its own process. The notebook's own database connection is untouched, and closing the notebook does not stop the app; stop_app() does.
Reached through Jupyter. Inside a Jupyter notebook the app is reached through Jupyter's own proxy, at <jupyter base>/proxy/absolute/8000/.
Nothing is exposed on the network, and access is protected by the Jupyter login (on SageMaker, the console's "Open Jupyter").
The app's path prefix (TDFS4DS_SERVE_ROOT_PATH) is set to match. The Jupyter base comes from the Jupyter server's runtime file: / on a SageMaker notebook instance, /user/<name>/ on JupyterHub, /jupyterlab/default/ in a SageMaker Studio JupyterLab space.
When the prefix found is still wrong (every page answers {"detail":"Not Found"}), give it: launch_app(root_path="/jupyterlab/default/proxy/absolute/8000").
Outside Jupyter, the link is http://localhost:8000/. The same goes for a notebook whose kernel no Jupyter server runs, such as a VS Code notebook on a local kernel (0.3.6.12): there is no proxy to go through. through_jupyter=True / False forces either form.
Only the app. The standalone Gradio chat and the docs server are not started; the app has its own chat.
The .env file:
It holds the connection and the models. The file given to launch_app wins over the environment.
Without an argument, the app reads the environment and the first of ./.env and ~/.tdfs4ds/.env (the environment winning).
The connection is checked before starting (see below).

Missing connection settings: asked, then saved if you want¶

Before starting the app, launch_app checks that it will have a database host, username, password (or a JWT token: TDFS4DS_TD_LOGDATA) and TDFS4DS_TD_ENCRYPT, resolved exactly as the app reads them, and tests the logon (one connection with the raw driver; the notebook's own connection is not touched).

tdfs4ds.connection_settings()            # what the app would use, where each value comes from (password hidden)
tdfs4ds.launch_app()                     # asks for what is missing
tdfs4ds.launch_app("~/projects/x/.env")  # this file is the one checked and offered for saving
tdfs4ds.launch_app(ask=False)            # never ask: what is missing is listed in the notes

When something is missing, or the database refuses the settings, it asks:

tdfs4ds app - database connection
  Read from: the environment (no .env file found)
    TDFS4DS_TD_HOST         NOT SET
    TDFS4DS_TD_USERNAME     NOT SET
    TDFS4DS_TD_PASSWORD     NOT SET
    TDFS4DS_TD_ENCRYPT      NOT SET
The app cannot connect with these settings. What do you want to do?
    1. enter the missing settings now
    2. use /home/me/project/.env
    3. use another .env file (type its path)
    4. start without them (connect from the app's Connect page)

Typing them: host, username, password (hidden), whether to encrypt, and the logon mechanism (TD2 or LDAP) when it is not set. The logon is tested; when refused, it asks again for the username and password.
Saving: you are then offered to write them for the next launches, in one of the .env files found (the one the app reads is marked), in a new file (by default ~/.tdfs4ds/.env, which the app reads with no argument), or not at all. The file is edited line by line (comments and other variables are kept). The password is asked separately: by default yes for a file under ~/.tdfs4ds (created readable by you only), no for a file in a project folder. A file the app does not find by itself is named in the message, with the launch_app(env_file=...) call to use next time.
Only TDFS4DS_TD_ENCRYPT unset: one question (encrypt? default no), then the same offer to save.
Nobody to answer (a script, papermill) or ask=False: nothing is asked, the app starts and the missing settings are listed in the returned notes. Nothing is asked when an app already runs on the port.

The feature store (since 0.3.3.20). Without TDFS4DS_SCHEMA the app would open connected but on no store. When it is not set, launch_app lists the feature stores this user can see (read with the same logon) and asks which one to open, or to type a database name, or to start without one and choose it in Parameters; the choice can be saved like the connection settings. Not interactive: a note says so.

Query band and DBQL (since 0.3.3.20). With the same logon, launch_app checks that the session query band can be set and that DBQL can be read and logs this user's queries: what the Execution log needs to show CPU and I/O. Nothing is asked (the app works without it); a note says what to ask the DBA for, and the returned connection carries query_band. The same check is the Query band & DBQL row of check_configuration().

Typed values reach the app through its environment for this launch, whether or not they are saved. The returned connection says which variables were typed, where they were saved and whether the logon was accepted, never a value. - Waiting and reusing. launch_app waits until the app answers. The first start connects to the database and may build the vector index, which can take a minute or more. When an app already runs on that port, launch_app shows its link instead of starting a second one. - Checking the proxy. The function asks Jupyter's proxy whether it reaches the app. The proxy is the jupyter-server-proxy extension of the Jupyter server, which on SageMaker lives in its own environment, not in your kernel's. If the proxy does not reach the app, install jupyter-server-proxy in the server's environment and restart Jupyter. If the proxy reaches the app but the app answers 404, the prefix is wrong: the message says so. - The log goes to ~/.tdfs4ds/app_8000.log; the state is kept in ~/.tdfs4ds/app_8000.json.

SageMaker notebook instance: install and start at every start¶

tdfs4ds.sagemaker_lifecycle_script() returns the Start notebook script of a lifecycle configuration. It installs the latest tdfs4ds in the python3 conda environment, the one behind the conda_python3 kernel, so notebooks get it too. It then starts the app with /home/ec2-user/SageMaker/tdfs4ds.env when that file exists.

print(tdfs4ds.sagemaker_lifecycle_script())                    # latest version, app started
print(tdfs4ds.sagemaker_lifecycle_script(version="0.3.3.4"))   # a fixed version
print(tdfs4ds.sagemaker_lifecycle_script(start_app=False))     # install only; start with launch_app()

Everything runs in the background, because a lifecycle script must finish within 5 minutes. The output goes to /home/ec2-user/SageMaker/tdfs4ds_start.log.

The settings in notebooks and terminals (since 0.3.3.15; share_env=False turns it off). The script makes the env file's TDFS4DS_* settings visible outside the app too:

Notebooks: ~/.tdfs4ds/.env is linked to the env file, and import tdfs4ds reads it when the notebook's folder has no .env of its own. Nothing to export: a kernel does not read shell profiles, so variables exported in /etc/profile.d or ~/.bashrc would never reach it.
Terminals: ~/.tdfs4ds/env.sh holds the same settings as quoted export lines (written by tdfs4ds's own .env reader, so a password with spaces, # or quotes, or a JSON header, is kept whole), and ~/.bashrc sources it. echo $TDFS4DS_TD_HOST then works in a new terminal.
The env file and env.sh are made readable by ec2-user only. Do not put credentials in /etc/profile.d: files there are readable by every account on the instance.

How the start behaves: - Only /home/ec2-user/SageMaker survives a stop and start, so keep the .env there and restrict it with chmod 600, because it holds credentials. - Packages are installed again at each start, and ~/.tdfs4ds/env.sh is written again: an edited env file is taken into account at the next start (or right away in a notebook after restarting its kernel). - Right after a start the install may still be running: check tdfs4ds.__version__ in a notebook, and restart a kernel that was opened before the upgrade.

Address. The app's address is your notebook URL with /proxy/absolute/8000/, for example https://<instance>.notebook.<region>.sagemaker.aws/proxy/absolute/8000/; launch_app prints it.

MCP server at start. sagemaker_lifecycle_script(mcp=True) also installs teradata-mcp-server and starts it with the .env credentials before the app (see below).

Teradata AI Studio¶

Logon with a token (since 0.3.5.6). An AI Studio .env written by generate_ai_studio_env(auth="jwt") holds a token (TDFS4DS_TD_LOGMECH=JWT, TDFS4DS_TD_LOGDATA="token=…") and no password; the app logs on with it. The token expires about two hours after it was minted, and the app does not renew it: write a fresh one into the .env before launching, then relaunch after two hours.

%gen_token                                   # in the notebook: user_token
tdfs4ds.patch_dotenv(".env", {"TD_LOGDATA": "token=" + user_token,
                              "VECTOR_STORE_AUTH_AUTH_TOKEN": user_token,
                              "MCP_SERVER_HEADERS": '{"Authorization": "Bearer %s"}' % user_token},
                     allow_secrets=True)
tdfs4ds.stop_app()                           # an app already running keeps its old token
tdfs4ds.launch_app(env_file=".env", ask=False)

jupyter-server-proxy is not installed on the AI Studio image. /opt/conda is read-only, so it goes in your home folder (pip install --user jupyter-server-proxy), the only folder that survives a server restart. The Jupyter server does not import packages installed that way by itself: add the folder to its path in ~/.jupyter/jupyter_server_config.py, then restart the server (File → Hub Control Panel → Stop My Server, then Start My Server):

import site, sys
if site.getusersitepackages() not in sys.path:
    sys.path.append(site.getusersitepackages())
c.ServerApp.jpserver_extensions.update({"jupyter_server_proxy": True})

pip reads an in-cluster mirror (PIP_INDEX_URL) that can list a new release a while after PyPI. To install a version published minutes ago: %pip install --user --index-url https://pypi.org/simple tdfs4ds==<version>.

SageMaker Studio¶

In a Studio JupyterLab space the address is the space's own address with /jupyterlab/default/proxy/absolute/8000/: https://<your Studio domain>/jupyterlab/default/proxy/absolute/8000/. Click the link launch_app shows; it cannot print the full address, because the Studio domain cannot be worked out from inside the space.

A local MCP server for the agent¶

The agent's MCP skills (sample rows of a dataset, the model behind a feature) need a Teradata MCP server. start_mcp_server() runs the Community teradata-mcp-server next to the notebook, logged on with the same credentials as tdfs4ds (TDFS4DS_TD_HOST, TDFS4DS_TD_USERNAME, TDFS4DS_TD_PASSWORD), and points tdfs4ds.MCP_SERVER_URL at it.

%pip install teradata-mcp-server      # once; or pipx install teradata-mcp-server
tdfs4ds.start_mcp_server()            # 127.0.0.1:8001, profile 'eda'
tdfs4ds.mcp_server_status()
tdfs4ds.stop_mcp_server()             # also restores the previous MCP settings

tdfs4ds.launch_app("tdfs4ds.env", mcp=True)   # the app and its MCP server together

Credentials: the database defaults to TDFS4DS_TD_DATABASE, else tdfs4ds.SCHEMA. The notebook's own teradataml connection cannot be reused, because it does not give its password back.
Logon: a user name and a password, with TD2 or LDAP. A token logon (TDFS4DS_TD_LOGDATA) cannot drive this server.
Security:
The password reaches the server through its environment, never its command line.
The server listens on 127.0.0.1 only. It has no authentication of its own, so anyone on this machine who can reach the port queries the database as you.
A logon the database refuses starts nothing.
Not a tdfs4ds dependency. The server runs in its own process, so it can live in its own environment; start_mcp_server finds teradata-mcp-server on PATH. Its log is ~/.tdfs4ds/mcp_8001.log.
Tools: profile=None loads every tool instead of the eda set; the agent then keeps the most relevant ones (MCP_MAX_TOOLS).