tdfs4ds.utils.time_management¶
module-attribute
¶
_FROM_QUALIFIED = compile('\\bFROM\\s+((?:"[^"]+"|[^\\s.,()"]+)\\s*\\.\\s*(?:"[^"]+"|[^\\s,()"]+))', IGNORECASE)
module-attribute
¶
_LITERAL_SELECT = compile("\\bSEL(?:ECT)?\\s+(?:/\\*.*?\\*/\\s*)?(?:CAST\\s*\\(\\s*TIMESTAMP\\s*'|DATE\\s*')", IGNORECASE | DOTALL)
Manage versioned business time steps behind a Teradata-backed public view.
TimeManager stores a sequence of time “snapshots” in a hidden physical
table (<view_name>_HIDDEN) with two columns:
- TIME_ID (1..N): the step index, assigned deterministically via
ROW_NUMBER() over the input time column.
- BUSINESS_DATE: the business date/time associated with each step. When
loading, the SQL type is inferred and upcast to TIMESTAMP WITH TIME ZONE
when needed to preserve offsets.
A companion public view (<view_name>) always exposes the current business
date by filtering the hidden table on a single TIME_ID. Changing the
current step only rewrites the view definition—no data is mutated.
Key capabilities
- Load/replace the hidden table from a pandas DataFrame (load_time_steps).
- Switch the active time step by TIME_ID (update).
- Inspect the current date/time (display, get_date_in_the_past).
- Generate a timeline view up to (or strictly before) the current step
(generate_timeline).
- Prune older steps and renumber so the earliest remaining step becomes 1
(prune_time).
- Clone from another TimeManager (soft link or hard copy) and optionally
take ownership of the hidden table (clone_timer, take_ownership).
- Introspect the active step by parsing the view DDL (get_current_timeid,
print_view_ddl).
Workflow overview
1) Instantiate TimeManager with a target view_name and schema_name.
2) Call load_time_steps(df, time_column) to (re)create the hidden table and
point the public view at TIME_ID = 1.
3) Use update(time_id) to switch the active business date.
4) (Optional) Create derivative timeline views, prune older steps, or clone.
Parameters table_name (str): Base public view name to manage (e.g., "MY_VIEW"). schema_name (str): Teradata schema/database that holds the artifacts.
Attributes
schema_name (str): Target schema for the view and hidden table.
table_name (str): Hidden table name (<view_name>_HIDDEN).
view_name (str): Public view name (<view_name>).
time_id (str): Name of the step identifier column (default: "time_id").
nb_time_steps (int | None): Number of steps detected after load/inspection.
data_type (str | None): SQL data type of BUSINESS_DATE (e.g., DATE,
TIMESTAMP WITH TIME ZONE), inferred during load/inspection.
Notes
- On initialization, if the hidden table already exists, metadata
(data_type, nb_time_steps) is auto-detected.
- load_time_steps will drop and recreate the hidden table to match the
inferred schema, then rebuild the public view.
- “Soft” cloning points this manager’s view at the source hidden table;
“hard” cloning copies the table into this schema and marks it owned.
- Ownership controls whether _drop() is allowed to remove the hidden
table (use take_ownership to promote ownership when appropriate).
property
writable
¶
Database of the backing table: schema_name unless set (a soft clone of a
manager in another database). A property so that objects built without
__init__ (tests, older pickles) keep working.
Initialize a TimeManager for an existing or future hidden table/view.
On initialization, if the hidden table already exists, the instance
inspects it to populate data_type and nb_time_steps.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
table_name
|
str
|
Base public view name to manage (e.g., |
required |
schema_name
|
str
|
Schema that contains/should contain the objects. |
required |
storage
|
str
|
Where to create the hidden backing table when this manager
(re)creates it. |
None
|
ofs_map
|
str
|
Map paired with |
'TD_MAP2'
|
Load/replace the hidden table and (re)point the public view to step 1.
Workflow
1) Build a DataFrame with sequential TIME_ID and BUSINESS_DATE.
2) Infer SQL types and upcast to TIMESTAMP WITH TIME ZONE when
needed (to preserve offsets).
3) Drop and recreate the hidden table with inferred schema.
4) Append the rows.
5) Replace the public view to expose TIME_ID = 1.
6) Store nb_time_steps.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Input pandas DataFrame with a time column. |
required |
time_column
|
str
|
Name of the time column in |
required |
Point the public view at a specific TIME_ID.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
time_id
|
int
|
The time step identifier to expose via the public view. |
required |
Point the view back at a TIME_ID of its table (0.3.6.0).
For a view left on a literal date by an interrupted run(..., at=): time_id
None = the cursor it had (cursor_of), else the cursor of the manager owning
its table, else 1. Works for a soft clone whose table is in another database.
Returns the TIME_ID set.
Whether the view holds a literal date (set by run(..., at=)) instead of a cursor.
Return the current public view (one row: current BUSINESS_DATE).
Returns:
| Type | Description |
|---|---|
DataFrame
|
A pandas DataFrame with the current |
Return the earliest BUSINESS_DATE from the public view as a string.
The format includes timezone offset when applicable:
- YYYY-MM-DD HH:MM:SS±HH:MM if timezone info is present;
- otherwise YYYY-MM-DD HH:MM:SS.
Returns:
| Type | Description |
|---|---|
str | None
|
The formatted earliest date/time string, or |
The BUSINESS_DATE of step time_id, formatted like :meth:get_date_in_the_past
(the valid time a run at that step stamps), without moving the cursor. None when
the step does not exist.
Return the full list of time steps from the hidden table.
Returns:
| Type | Description |
|---|---|
DataFrame
|
A Teradata DataFrame over |
DataFrame
|
|
generate_timeline(schema_name: str, view_name: str, current_included: bool = True) -> tdml.DataFrame
Create a timeline view filtered relative to the current business date.
The new view (schema_name.view_name) selects dates from the hidden
source (self.view_name + '_HIDDEN') up to the current business date
exposed by the public view (self.view_name).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
schema_name
|
str
|
Schema where the new timeline view will be created. |
required |
view_name
|
str
|
Name of the new timeline view. |
required |
current_included
|
bool
|
If True, include the current business date; otherwise, exclude it. |
True
|
Returns:
| Type | Description |
|---|---|
DataFrame
|
A Teradata DataFrame bound to the newly replaced timeline view. |
Return the TIME_ID corresponding to the current BUSINESS_DATE in the view.
Returns:
| Type | Description |
|---|---|
int | None
|
The current |
clone_timer(source_timemanager, time_id_to_apply: int = 1, take_ownership: bool = False, clone_mode: str = 'soft', if_exists: str = 'error')
Clone time-step definitions from another TimeManager.
Supports: - soft clone (default): point this manager's view to the source _HIDDEN table - hard clone: copy the source _HIDDEN table into this schema and own the copy
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source_timemanager
|
TimeManager
|
Source manager to clone from. |
required |
time_id_to_apply
|
int
|
TIME_ID to activate in the public view. Default: 1. |
1
|
take_ownership
|
bool
|
For soft clones only, whether this manager should consider itself the owner of the hidden table. (Hard clones always own their copy.) Default: False. |
False
|
clone_mode
|
str
|
"soft" or "hard". Default: "soft". |
'soft'
|
if_exists
|
str
|
What to do if the destination hidden table already exists - "error" (default): raise an exception - "replace": drop and recreate - "skip": reuse existing table |
'error'
|
Returns:
| Name | Type | Description |
|---|---|---|
TimeManager |
self |
Raises:
| Type | Description |
|---|---|
ValueError
|
On invalid clone_mode/if_exists or missing source table. |
RuntimeError
|
If destination exists and if_exists="error". |
Promote this manager to OWN the hidden table.
Two modes:
- create_copy=True (default): Hard-promote by copying the current source
hidden table into this manager's schema as
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
create_copy
|
bool
|
If True, copy data into this schema and repoint view. |
True
|
if_exists
|
str
|
Behavior when the destination |
'error'
|
Returns:
| Name | Type | Description |
|---|---|---|
TimeManager |
TimeManager
|
self |
Extract the currently active TIME_ID from the public view's DDL.
Returns:
| Name | Type | Description |
|---|---|---|
int |
int
|
TIME_ID parsed from the view definition. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the TIME_ID cannot be parsed from the DDL. |
Log the view definition (DDL) for troubleshooting/traceability.
Remove all time steps with TIME_ID lower than time_id and renumber remaining ones.
If time_id is omitted, the method uses the current TIME_ID from the view.
After pruning, TIME_ID values are normalized so the smallest remaining id becomes 1,
and the public view is repointed to TIME_ID=1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
time_id
|
int
|
Threshold id; rows with TIME_ID < time_id are deleted. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
TimeManager |
Self, to allow method chaining. |
Wrapper around the global logger that only emits logs when
tdfs4ds.DISPLAY_LOGS is True. level is a string like "info", "error", etc.
Build db.obj with each part quoted only as needed.
qualified('TDB', 'FS_X') == 'TDB.FS_X' (unchanged for ordinary names);
qualified('SVC@CORP', 'FS_X') == '"SVC@CORP".FS_X'. An empty/None
database yields the bare object name, so unqualified callers keep working.
force=True quotes both parts unconditionally — only for the sites that
have always emitted quotes and whose exact text is already persisted (dataset
view DDL, returned volatile-table names).
Split db.obj into (database, object), both de-quoted.
Quote-aware, so a dot inside a quoted part does not split the name:
split_qualified('"DEINF.CARLOSN"."FS_X"') == ('DEINF.CARLOSN', 'FS_X').
The naive rsplit('.', 1) this replaces returns ('"DEINF.CARLOSN"',
'"FS_X"') only by luck and ('"DEINF', 'CARLOSN"."FS_X"') in the
split('.') variant.
An unqualified name returns ('', name).
Return the CREATE TABLE options fragment for the requested storage.
The fragment is inserted immediately after the table name, e.g.::
CREATE TABLE db.tbl{storage_clause(...)} ( ... ) PRIMARY INDEX (...)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
storage
|
storage selector (see :func: |
required | |
ofs_map
|
the map paired with |
DEFAULT_OFS_MAP
|
Returns:
| Type | Description |
|---|---|
|
|
|
|
|
The backing table a manager view reads: the object after its FROM.
A manager view is ... SEL <cols> FROM <db>.<table> WHERE <id> = n. The
name is read with the quote-aware splitter, so "DB"."T_HIDDEN" (the form
a migrated view has) gives T_HIDDEN, not "T_HIDDEN". Shared by
TimeManager and FilterManager. Returns None when no qualified FROM is found.
(database, table) a manager view reads (the object after its FROM), or None.
A soft clone's view may read a table in another database than its own (0.3.6.0): the database is what tells where the dates live.
(table database, table) recorded in the manager registry for a view, or None.
(database, table) of the table a manager view reads.
The database is schema_name unless the view reads a table elsewhere (a soft
clone of a manager in another database, 0.3.6.0); compared without case or
quotes, so an ordinary manager keeps its schema_name spelling and its SQL is
byte-identical. Falls back to (schema_name, <view>_HIDDEN).
Point a TimeManager view at one TIME_ID of a given table (what update does).
For a soft clone whose table lives in another database than its view, which a
TimeManager(view, schema) rebuilt from the view name would not find. Used by
roll_out to move soft clones with the manager it drives (0.3.5.15).
The TIME_ID a TimeManager view points at, read from its DDL, or None (the view
cannot be read, or holds a literal date with no recorded cursor). For a view holding a
literal date (run(..., at=)), the cursor it goes back to. Never raises, never logs
an error.
REPLACE VIEW of a TimeManager view onto one literal business date (0.3.6.0).
literal is a TIMESTAMP literal body (dataset.temporal.format_timestamp_literal).
The value is cast to the manager's own BUSINESS_DATE type, so every view reading it
compares the same type as with the table; a DATE calendar gets the date part.
Only the selected value changes: the view still reads FROM <table> WHERE TIME_ID = n
(time_id, its cursor; one row). So a swapped view keeps its lineage edge to the table of
dates, and the table and the cursor are read from it exactly as from a cursor view
(backing_location_from_view_ddl, cursor_of): an interrupted run leaves a view the
next run can restore (seen live: a SELECT <literal> with no FROM lost its table).
LITERAL_MARK (a comment) and the literal in the select list tell it holds a date.
Whether a TimeManager view text selects a literal date instead of the table's
BUSINESS_DATE column (a view swapped by run(..., at=)).
Where a swapped view goes back to: its own cursor, else (a view left swapped by an interrupted run) the cursor of the manager owning the table, else 1.
Set each TimeManager view of managers to the literal date literal for the duration
of the block, then put each back on its TIME_ID (0.3.6.0).
Used by run([...]) / run(data_domain=...) / run(..., at=) to compute processes
at one time without moving any cursor. The restore runs in a finally; before each swap
the statement that restores the view is logged, and a view left with a literal by an
interrupted run is recognised (cursor_of is None) and restored by
:meth:TimeManager.restore_cursor. Yields the list of (manager, step) swapped.
A BUSINESS_DATE value as the valid-time string a run stamps:
YYYY-MM-DD HH:MM:SS±HH:MM with a time zone, else YYYY-MM-DD HH:MM:SS.
None for a type it cannot read.