tdfs4ds.agentops.session_keeper

tdfs4ds.agentops.session_keeper

Keep a deployed agent's Teradata session usable through long idle periods.

An AgentOps pod holds ONE teradataml context for the life of the process (turns are serialized by the agent's turn lock). Seen live on AI Studio (0.3.6.0): after about two idle hours, the first request with a new caller token never answered. The pod log stopped on "opening Teradata session for caller ...", i.e. inside the switch: closing the old context (tdml.remove_context(), whose garbage collector sends a DROP VIEW over the old connection) or opening the new one. Every later request then waited behind it on the turn lock. The most likely cause is an idle connection dropped silently by the network, on which the driver waits for an answer that never comes. Not proven from inside the pod, which is why every step here also reports where it is.

Three tools, all fail-open and never raising:

• :func:keepalive_loop - a light SELECT 1 every interval seconds under the agent's turn lock, so the connection is never idle long enough to be dropped. A ping that fails or times out reports the session dead.
• :func:close_context_bounded - remove_context() with a time limit. When it does not return, teradataml's connection globals are cleared so the next create_context() does not try to close the dead connection again; the stuck call is left to finish (or not) in its daemon thread.
• :func:watchdog - writes every thread's stack to the pod log when a step takes longer than expected, so a hang names its own location.

logger module-attribute

logger = getLogger('tdfs4ds.agentops')

KEEPALIVE_SECONDS module-attribute

KEEPALIVE_SECONDS = 600

PING_TIMEOUT module-attribute

PING_TIMEOUT = 60

CLOSE_TIMEOUT module-attribute

CLOSE_TIMEOUT = 30

WATCHDOG_SECONDS module-attribute

WATCHDOG_SECONDS = 60

_bounded

_bounded(fn, timeout, name)

Run fn in a daemon thread. Returns (finished, result_or_exception).

_abandon_context

_abandon_context()

Forget the current teradataml connection without sending anything over it.

close_context_bounded

close_context_bounded(timeout=CLOSE_TIMEOUT)

Close the teradataml context, giving up after timeout seconds.

Returns 'none' (nothing open), 'closed', 'failed' (the close raised; the context is cleared anyway) or 'abandoned' (it did not return in time: the context is cleared and the stuck call is left in its daemon thread).

_context_open

_context_open()

True when teradataml holds an open connection (either layout).

ping

ping(timeout=PING_TIMEOUT)

True when the current context answers SELECT 1 within timeout seconds.

keepalive_loop async

keepalive_loop(lock, on_dead, interval=KEEPALIVE_SECONDS, timeout=PING_TIMEOUT)

Ping the session every interval seconds while holding lock (the turn lock).

on_dead is called (inside the lock) when a ping fails, so the agent can close the dead session and open a new one on the next request. Runs until cancelled.

start_keepalive

start_keepalive(lock, on_dead, interval=KEEPALIVE_SECONDS)

Start :func:keepalive_loop on the running event loop. Returns the task.

watchdog

watchdog(step, seconds=WATCHDOG_SECONDS)

Dump every thread's stack to stderr (the pod log) if step exceeds seconds.