TimeManager & `roll_out`¶

TimeManager advances a "current business date" view, and roll_out iterates registered processes along that sequence. Together they backfill features across an arbitrary date range.

A dataset that follows the manager¶

upload_features(..., time_manager=tm, dataset_view_name='db.ds', dataset_time_manager=True) builds the dataset with build_dataset(time_manager=tm) instead of at CURRENT VALIDTIME: after a roll_out, tm.update(i) returns the values of date i with no re-run. The setting is recorded for the process, so later run() / roll_out() keep it. A dataset already built current is left untouched with a warning naming rebuild_dataset(..., time_manager=..., replace=True).

How TimeManager works internally¶

TimeManager maintains two database objects:

Hidden table (<view_name>_HIDDEN) — stores every time step as a row with TIME_ID (1..N, deterministic ROW_NUMBER()) and BUSINESS_DATE. Populated once by load_time_steps() and never mutated during replay.
Public view (<view_name>) — always exposes exactly one row: WHERE TIME_ID = N. Advancing to the next step only rewrites this view definition — no data is touched.

Your feature engineering view reads the current date with a scalar subquery:

WHERE TX_DATE <= (SELECT BUSINESS_DATE FROM MY_DB.FS_BUSINESS_TIME)

Teradata resolves the subquery at plan time and uses it as a filter constant.

load_time_steps()      → writes hidden table + creates view at TIME_ID=1
update(time_id=N)      → REPLACE VIEW … WHERE TIME_ID = N
display()              → pandas DataFrame with current BUSINESS_DATE
get_date_in_the_past() → formatted string e.g. "2024-03-15 00:00:00"

Set up TimeManager¶

from tdfs4ds.utils.time_management import TimeManager

bt = TimeManager(table_name='FS_BUSINESS_TIME', schema_name=tdfs4ds.SCHEMA)

# First time only: populate from a date column in the source
bt.load_time_steps(
    df       = source_data.groupby('TX_DATE').count().to_pandas(),
    time_column = 'TX_DATE',
)

print('Steps loaded:', bt.nb_time_steps)
print('Current step:', bt.get_current_step())

# Sync FEATURE_STORE_TIME with the current step
tdfs4ds.FEATURE_STORE_TIME = bt.get_date_in_the_past()

After the first call, TimeManager reconnects to the existing hidden table on every subsequent instantiation — load_time_steps is a one-time setup.

TIMESTAMP WITH TIME ZONE upcasting¶

When the time column carries timezone information, load_time_steps automatically upcasts the BUSINESS_DATE column from TIMESTAMP(N) → TIMESTAMP(N) WITH TIME ZONE to preserve offsets. The data_type attribute on the instance reflects the stored SQL type.

Backing-table storage on Vantage Cloud Lake (`storage='OFS'`)¶

By default the hidden table is created on block storage — unchanged, and the right choice for full primary-cluster processes. On Vantage Cloud Lake you can create it on the Object File System (OFS) instead:

bt = TimeManager(
    table_name  = 'FS_BUSINESS_TIME',
    schema_name = tdfs4ds.SCHEMA,
    storage     = 'OFS',        # None (default) = block storage, unchanged
)
bt.load_time_steps(df=..., time_column='TX_DATE')

This emits MAP = TD_MAP2, STORAGE = TD_OFSSTORAGE. The choice is per manager, so block-storage and OFS managers coexist in one session; it only affects table creation (ofs_map= overrides the map, default TD_MAP2). Confirm with SHOW TABLE <view>_HIDDEN.

tdstone2 Apply on the ACC: OFS manager table and a JOIN (not a scalar subquery)

A tdstone2 model scored via the Apply backend runs on the Analytical Compute Cluster (ACC), which reads from OFS. For a scoring view that references the TimeManager, two conditions must both hold:

The manager table on OFS — storage='OFS' (above), so the ACC can read it. A block-storage manager table read by the ACC raises Error 6881.
Reference the date via a JOIN, not a correlated scalar subquery. The ACC rejects ... WHERE tx_date <= (SELECT BUSINESS_DATE FROM <tm_view>) with 6881, but accepts a JOIN to the manager view. Confirmed live on VCL: OFS + JOIN scores; OFS + scalar subquery still 6881s.

Authoring the filter as a JOIN is the user's responsibility — it is a property of the scoring view's SQL, e.g.:

bt.update(time_id=i)                       # advance the manager view
tdml.execute_sql(f"""
    REPLACE VIEW my_db.vw_scoring_input AS
    SELECT s.*
    FROM source s
    JOIN {tdfs4ds.SCHEMA}.{bt.view_name} tm   -- JOIN, not a scalar subquery
      ON s.tx_date <= tm.BUSINESS_DATE
""")
model.score()                              # lands predictions in the OFS scores repo
tdfs4ds.upload_features(...)               # register the scored rows

The JOIN keeps the date dynamic, so time-travel via update() still works (a hard-coded literal also passes on the ACC, but freezes the date). roll_out/run() cannot drive an Apply model themselves — their ingestion CTAS (CREATE VOLATILE TABLE AS SELECT * FROM <live scoring view>) also hits 6881; call score() first, then upload_features.

`roll_out` across a range¶

from tdfs4ds.process_store.process_query_administration import get_process_id

views = tdfs4ds.process_catalog()[['VIEW_NAME']].to_pandas().VIEW_NAME.values
v1 = [v for v in views if 'FEAT_ENG_TX' in v][0]
v2 = [v for v in views if 'FEAT_ENG_CUST' in v][0]

pid_1 = get_process_id(view_name=v1)
pid_2 = get_process_id(view_name=v2)

tdfs4ds.roll_out(
    process_list      = [pid_1, pid_2],
    time_manager      = bt,
    time_id_start     = 1,
    time_id_end       = 30,           # roll over 30 steps
    force_display_logs= False,
)

time_id_start and time_id_end are 1-based positions in the ordered time sequence, not timestamps. Step 1 is the earliest date in the loaded range.

roll_out does three things per step:

bt.update(time_id=i) — advance the TimeManager view.
Sync tdfs4ds.FEATURE_STORE_TIME to bt.get_date_in_the_past().
Call tdfs4ds.run(process_id=pid) for every process in process_list.

It also records, for each process, which TimeManager drives it, and restores tdfs4ds.FEATURE_STORE_TIME to its previous value when it finishes (from 0.3.1.227; it used to leave the last rolled-out date set for the session).

A history that already starts later¶

A run at business date t writes its values from t onward, but never before the first value already stored. A process that was first run in real time (upload_features runs at once unless run_now=False) or at a later date cannot be rolled out over earlier dates: each step would only rewrite the period from the first stored value onward.

Since 0.3.5.16 roll_out checks this before the first step and refuses with a ValueError naming the processes and where their values start; nothing runs. Two ways out:

# delete their stored values from the first date rolled out onward, then replay
tdfs4ds.roll_out([pid_1, pid_2], bt, rebuild_history=True)

# or roll out only from the first stored date
tdfs4ds.roll_out([pid_1, pid_2], bt, time_id_start=12)

The deletion is logged in FS_REMOVAL_LOG (kind VALUES). To avoid the case, register processes that a TimeManager drives with upload_features(..., time_manager=bt, run_now=False), then roll them out.

Entities that leave and come back¶

By default a date only writes the entities the view returns at that date: an entity that leaves the source keeps its last values. A process registered with upload_features(..., close_absent_entities=True) also ends, at each date, the validity of the entities it no longer produces — so a customer present in January, absent in February and back in March has two periods, with a gap in February. roll_out(..., close_absent_entities=True|False) overrides the recorded option for one roll-out. Replay such a history in date order, from its first date. See Entities that disappear.

When the dates are in the data¶

roll_out builds a history by running the process once per business date. If the source already holds one row per entity and valid time, a process registered with valid_time_column (see Upload features) writes the whole history in one run, and the TimeManager date does not set the values' valid time for it.

Attach the manager at registration¶

A TimeManager can be attached to a process when it is registered, the way a FilterManager is (0.3.1.240):

pid = tdfs4ds.upload_features(
    df=None, sql_query=sql, target_view_name='MY_DB.FEAT_MONTHLY',
    entity_id=['CustomerID'], feature_names=[...],
    time_manager=bt, run_now=False, return_process_id=True)

bt.update(time_id=1)
tdfs4ds.run(pid)            # no argument: the attached manager's current date

The pairing is recorded in FS_TIME_MANAGER, the table roll_out already writes. Without time_manager= the registration statements are unchanged.

A manager the view does not read¶

Before run(..., time_manager=), roll_out or upload_features(..., time_manager=) use a manager, tdfs4ds looks in the lineage graph for the manager the process view really reads, through any number of intermediate views. A manager it does not read would stamp the values with a date they were not computed at.

`tdfs4ds.TIME_MANAGER_CHECK`	Unrelated or different manager
`'warn'` (default)	runs as before, logs a warning naming the manager the view reads, records no pairing
`'error'`	`ValueError` before anything runs; `upload_features` registers nothing and returns `None`
`'off'`	no check

A process the lineage graph does not know (registered after the last refresh) is only warned about, with the rebuild_lineage_graph(objects=[...]) command to run. A real-time run(pid) of a process whose view reads a manager that is not declared logs a warning too.

tdfs4ds.time_manager_dependency(pid)            # declared vs read, from the lineage
tdfs4ds.link_time_managers(dry_run=True)        # what it would record or correct
tdfs4ds.link_time_managers(dry_run=False)       # record it (processes registered without it)

link_time_managers returns one row per process with ACTION = RECORD (reads one, none declared), CORRECT (declares another), OK, NONE, AMBIGUOUS (reads two; left alone) or UNKNOWN (not in the lineage graph).

Every current usage keeps working:

Usage	Behaviour
`FEATURE_STORE_TIME` set by hand, `run(pid)`	unchanged: the explicit time wins
`roll_out(pids, tm)` / `run(pid, time_manager=tm)`	unchanged, plus the check (a warning by default)
`run(pid)` after a `roll_out`	unchanged: the recorded manager's date
no manager at all	unchanged: real time (plus a warning if the view reads a manager)
`upload_features(...)` without `time_manager`	identical registration statements
a process registered by an older client	read through `FS_TIME_MANAGER` as before; `link_time_managers` adds the pairing

One clock per data domain (0.3.6.0)¶

An alternative to attaching a TimeManager to each process, for the common case where the processes of a data domain follow one business calendar. Attaching a manager per process (time_manager=, the sections above) stays fully supported and gives more control: a calendar of its own for a process, a hand-made clone, a manager shared on purpose. Declare the domain's calendar once:

clock = TimeManager('SALES_CLOCK', 'MY_DB')
clock.load_time_steps(dates_df, 'BUSINESS_DATE')     # once
tdfs4ds.set_domain_time_manager(clock)                # for tdfs4ds.DATA_DOMAIN

From then on:

upload_features needs no time_manager=. It walks the source view's lineage (through intermediate views) and finds the manager the view reads. When that is the domain clock, the process gets its own soft clone of it (<process view>_TM, next to the view: same dates, own cursor) and its view is rewritten to read the clone, with a log line. Write the process SQL against the clock itself: WHERE A.EVENT_DATE <= (SELECT BUSINESS_DATE FROM MY_DB.SALES_CLOCK). A time_manager= passed explicitly is paired as before 0.3.6.0, even when it is the clock: no clone, so tm.update(i); run(pid) keeps following it. soft_clone=True asks for the clone anyway; soft_clone=False refuses it for a detected clock.
roll_out(data_domain='SALES') (or roll_out() for tdfs4ds.DATA_DOMAIN) builds the history of every process of the domain. The clock drives, and at each date every clone is moved to the same date. The processes of a date run in dependency order.
run(data_domain='SALES') computes every process of the domain at one time. That time is the database clock, read once, unless you pass at='2024-03-31'. Each clone is swapped for that literal date for the duration of the computation, then put back on its cursor. Nothing moves, so after a roll-out you can refresh "now" and the next roll-out step is unchanged. run([pid_1, pid_2], at=...) does the same for a list.

tdfs4ds.roll_out(data_domain='SALES')                  # history, date by date
tdfs4ds.run(data_domain='SALES')                       # everything at one "now"
tdfs4ds.run([pid_7], at='2024-03-31', force_compute=True)   # one process, a past date

run over a list returns {process_id: 'COMPLETED' | 'FAILED: ...' | 'SKIPPED: ...'}:

Order: the processes run one after the other, in dependency order from the lineage graph. order='given' keeps yours.
Failures: a failure skips the processes downstream of it. on_failure='raise' raises FeatureIngestionError after the list, 'stop' stops at the first failure.
Processes reading no TimeManager are computed on the current source data and stored at the same time, with a warning.
A manager read outside the run: when a manager is also read by a process outside the list, that process is named. While the run lasts, that process would read the literal date.

Function	What it does
`set_domain_time_manager(tm, data_domain=None)`	declares the clock (stored in the optional table `FS_DATA_DOMAIN_SETTINGS`)
`get_domain_time_manager(data_domain=None)`	the clock, or `None`
`detach_domain_time_manager(data_domain=None)`	removes it; processes keep their clones
`align_domain_time_managers(data_domain=None, dry_run=True)`	gives processes registered before the clock their own clone
`clone_shared_view_time_manager(view, dry_run=True)`	one clone for the processes reading the clock through a shared view (0.3.6.8)
`domain_processes(data_domain=None)`	the processes of a domain (primary store)
`get_process_info(pid)['DOMAIN_TIME_MANAGER']`	the clock of the process's domain

Processes that read the clock through a shared view. When a view of yours filters a source with the clock (WHERE d <= (SELECT BUSINESS_DATE FROM <clock>)) and several processes read that view, none of them can get its own clone: rewriting the view for one process would rewrite it for all. They keep the clock and share its cursor. Their values are right, but a run of some of them at another date is seen by the others while it lasts. upload_features says so, naming the view and its other readers, and the Report and the workflow assessment (processes_share_time_filtered_view) list them. Three ways out:

A cursor per process (your SQL): remove the date filter from the view, expose the date column and filter in each process view; each process then gets its own clone at registration. Or give each process its own copy of the view.
One cursor for the view's readers: tdfs4ds.clone_shared_view_time_manager('<db>.<view>', dry_run=True) shows the plan; dry_run=False makes one soft clone <view>_TM, rewrites only the clock reference inside the view (it is never copied), and records the clone for every process reading it. They stay together, apart from the clock and from the other processes, and roll_out(data_domain=...) keeps the clone in step. A process registered later on the view is recorded on that clone.

An object that already has the clone's name is reused only when it is a soft clone over the same dates; a table, another view, or a clone of another calendar is refused and left unchanged (pass clone_name=). The same check guards the clone upload_features makes for each process.

What can interrupt the swap. A literal date is a REPLACE VIEW of the clone that changes only the selected value: the view still reads FROM <table> WHERE TIME_ID = n. Lineage to the table of dates holds during the run, and if the run is killed the next run / roll_out (or TimeManager('<clone>', '<db>').restore_cursor()) puts it back.

run(pid, time_manager=clock) for a process reading a clone of that clock sets the clone to the clock's date for the run (it used to warn and stamp a date the view did not compute at), like roll_out moves clones with the manager it drives.

Documentation. A clone means what its clock means: the documentation pipeline copies the clock's documentation onto it (PROPAGATED), with no LLM call per clone. Document the clock once (set_object_documentation); a clock without documentation leaves its clones to the usual path.

In the admin app. The feature catalog's process card shows the clock of the process's domain and how the process stands with it (reads the clock, its own clone, no TimeManager, or another calendar); the dataset builder lists the domain clocks first among the TimeManagers a dataset can follow; the Report's Time drivers lists each clock with the processes on and off it.

Migration. plan_migration / migrate_objects carry the domain clock: the clones and their table of dates come with the processes, the clock view is recreated over that table at its cursor and declared for the target domain.

When the clone cannot be made. A process view that reads the clock only through an intermediate view keeps the clock (that view may be shared by other processes). It is still rolled out and run with the domain, but its cursor is the clock's own.

Nothing changes for a domain without a clock:

Usage	Behaviour
`run(pid)`, `roll_out(pids, tm)`	unchanged (the calls they make are compared with 0.3.5.19 in the tests)
`upload_features(...)` without `time_manager=` and no domain clock	the manager the view reads is recorded (new; `tdfs4ds.TIME_MANAGER_DETECT = False` turns it off)
`roll_out` with several processes	sorted by dependency (stable: a list in a valid order is unchanged; `order='given'` keeps it)
soft clone in another database than its table	now found and moved correctly (it used to look for the table in the view's database)

Manual single-step¶

bt.update(time_id=10)
tdfs4ds.run(process_id=pid_1)                     # uses the TimeManager recorded for pid_1
tdfs4ds.run(process_id=pid_2, time_manager=bt)    # or name it; the pairing is then recorded

From 0.3.1.227 on, run() maps the TimeManager to FEATURE_STORE_TIME itself: the run's values are stamped with the manager's current business date, the follow-up row records that date, and FEATURE_STORE_TIME is restored after the run. Setting tdfs4ds.FEATURE_STORE_TIME = bt.get_date_in_the_past() by hand still works and always takes precedence.

Use manual stepping when you need to inspect intermediate output between steps — roll_out runs to completion (or until a process raises).

Which valid time a run uses¶

Every value a run writes gets one valid start, decided once per run:

Order	Source	When
1	`tdfs4ds.FEATURE_STORE_TIME`	it is set (each `roll_out` step sets it)
2	`run(..., time_manager=tm)`	the manager's current business date
3	the TimeManager recorded for the process	recorded by `upload_features(time_manager=)`, `roll_out` or a previous `time_manager=`
4	real time	none of the above: the database time, read once for the whole run

Drive processes with a TimeManager, or set FEATURE_STORE_TIME. It is the only way several processes share the same valid time, and the only way values computed for a business date are stamped with that date. Real time is a fallback. Since 0.3.1.227, every feature and every FilterManager partition of a real-time run shares one valid start; earlier versions stamped each feature separately, about a second apart. A real-time run's follow-up row keeps VALIDTIME_DATE NULL.

A dataset that follows the TimeManager¶

A dataset built with time_manager= reads every feature as of the manager's current BUSINESS_DATE. Moving the manager moves the dataset, with no rebuild:

ds = tdfs4ds.build_dataset(
    entity_id         = ['CUSTOMER_ID'],
    selected_features = selected_features,
    view_name         = 'DS_CUSTOMER_BACKTEST',
    time_manager      = bt,
)

bt.update(time_id=10)    # ds now returns the values valid at step 10's date

Each feature sub-query is NONSEQUENCED VALIDTIME and filters the valid period with the manager's business date as a scalar:

WHERE B1.ValidStart <= (SELECT CAST(BUSINESS_DATE AS TIMESTAMP(0) WITH TIME ZONE) FROM <bt view>)
  AND B1.ValidEnd   >  (SELECT CAST(BUSINESS_DATE AS TIMESTAMP(0) WITH TIME ZONE) FROM <bt view>)
  AND FEATURE_ID = … AND FEATURE_VERSION = '…'

This shape was chosen by reading the EXPLAIN plans on Vantage 20:

The manager is read once per query, through its primary index, and its value is substituted as a parameter before the feature tables are read. That holds however many features the dataset has.
The storage tables keep their partition elimination ("from 3 partitions of…"), exactly as with a literal date, in both the multi-join and the grouped strategy.
VALIDTIME AS OF (SELECT BUSINESS_DATE FROM <bt view>) was rejected. It selects the same rows, but in the grouped strategy it turned every storage-table read into an all-rows scan.
Joins to the manager view were rejected too. They doubled the plan steps and added a product join per feature.

The predicate is the one Teradata itself generates when it rewrites an AS OF, so the selected instant is identical. The business date is never aggregated (no MAX): the manager's view returns one row, and if it ever returned two, the query fails with Error 3669 instead of silently picking a date.

The view must exist when the dataset is built

build_dataset creates a view that references the manager's public view, so load the time steps (load_time_steps) first.

See Datasets.

Inspection methods¶

Method	Returns	Description
`display()`	`pd.DataFrame`	Current `BUSINESS_DATE` from the public view
`get_date_in_the_past()`	`str`	Formatted date string (with tz offset when applicable)
`get_list_date()`	`teradataml.DataFrame`	All rows in the hidden table
`get_current_step()`	`int`	`TIME_ID` that matches the current business date
`get_current_timeid()`	`int`	Active `TIME_ID` parsed from view DDL
`generate_timeline()`	`teradataml.DataFrame`	Timeline view up to (or strictly before) current date
`print_view_ddl()`	—	Logs the view DDL for troubleshooting

# Inspect all available dates
bt.get_list_date().to_pandas()

# Create a cumulative timeline view for a dataset join
bt.generate_timeline(
    schema_name      = tdfs4ds.SCHEMA,
    view_name        = 'FS_TIMELINE',
    current_included = True,    # include today's date in the window
)

Pruning¶

prune_time() removes all steps with TIME_ID < N and renumbers the remaining ones so the smallest id becomes 1. Use it to free space in the hidden table after a successful backfill.

# Compact to the current position (discard everything before today)
bt.prune_time()                   # uses get_current_timeid() automatically

# Or keep only from step 20 onwards
bt.prune_time(time_id=20)

After pruning, the public view is repointed to TIME_ID=1.

Cloning¶

Cloning lets one TimeManager re-use the step sequence owned by another. The API mirrors FilterManager's clone_filter and supports the same two modes.

Default: share one manager. Processes whose views read the same manager view share one cursor. They always compute the same business date, their values carry that date, and a dataset joining their features at an instant combines values of the same date. A generated parallel script keeps this true: processes on one manager run in lockstep, one date at a time (execution plan).

The problem a clone solves. A shared cursor moves every process at once. Give a process its own cursor when it must stand at another date than the others:

catching up a process that fell behind, while the others stay where they are;
back-filling a new process over past dates while the existing ones keep running at the current date;
running a process at another cadence over the same calendar;
letting independent timelines run in parallel instead of in lockstep.

	Share one manager	Soft clone	Hard clone
Same dates as the original	yes	yes, always (it reads the original's table)	at the moment of the copy only
Own position (cursor)	no	yes	yes
Dates maintained in one place	yes	yes	no, two copies
Typical use	processes that must stay synchronised	own pace, same calendar	a calendar that must be allowed to differ (another holiday set, a test, an audit snapshot)

Cautions.

A soft clone reads the original's table of dates. load_time_steps on the original changes the clone's dates too: that is why you choose soft. Dropping the original's table breaks the clone (it does not own it unless take_ownership).
A clone's cursor is its own. roll_out on the original does not move it, and a roll_out on the clone does not move the original.
A hard clone drifts silently. A date added to the original is not in the copy. The Report shows hard clones as copies, not as synchronised.
Once cursors differ, processes compute at different business dates. A dataset that joins their features at one instant then mixes values valid at different business dates. Read it as_of a date both have computed (build_dataset(..., as_of=...)), or bring the processes back onto one manager.
Freshness is per cursor. "Up to date" means up to that process's own manager date.
Check where each process stands. The Report's Time drivers section lists the groups (synchronised, soft clones with whether they point at the same date, hard clones) and the date of each cursor.
Bringing them back. unify_time_managers repoints processes onto one manager. A history is built forward only, so re-run the late ones from the first date where they diverged (roll_out(..., time_id_start=..., force_compute=True)).

Soft clone (default)¶

The new manager's view points directly to the source hidden table. No data is copied. Both managers see the same steps; their active TIME_ID is independent.

# Source manager already loaded and in use for process 1
bt_source = TimeManager(table_name='FS_BUSINESS_TIME', schema_name=tdfs4ds.SCHEMA)

# New manager for process 2 — shares the same hidden table
bt_new = TimeManager(table_name='FS_BUSINESS_TIME_P2', schema_name=tdfs4ds.SCHEMA)
bt_new.clone_timer(
    source_timemanager = bt_source,
    time_id_to_apply   = 1,       # start at the first step
    clone_mode         = 'soft',  # default
)

The FS_BUSINESS_TIME_P2 view is created (or replaced) in the same schema. The underlying hidden table remains FS_BUSINESS_TIME_HIDDEN in the source schema. roll_out can now advance bt_new independently of bt_source.

Ownership note: by default take_ownership=False so this manager will not drop the shared hidden table. Pass take_ownership=True only when this manager should be responsible for the shared table's lifecycle.

Hard clone¶

The new manager gets its own physical copy of the hidden table. Changes to the source afterwards do not affect the clone.

bt_copy = TimeManager(table_name='FS_BUSINESS_TIME_ARCHIVE', schema_name=tdfs4ds.SCHEMA)
bt_copy.clone_timer(
    source_timemanager = bt_source,
    time_id_to_apply   = 1,
    clone_mode         = 'hard',
)
# bt_copy._owns_hidden is now True

A CREATE TABLE … AS (SELECT * FROM …) WITH DATA is issued into bt_copy.schema_name. The copy is fully independent.

Handling existing tables with `if_exists`¶

Value	Behaviour
`"error"` (default)	Raises `RuntimeError` if destination already exists
`"replace"`	Drops and recreates the destination table
`"skip"`	Reuses the existing table without re-copying

bt_copy.clone_timer(
    source_timemanager = bt_source,
    clone_mode         = 'hard',
    if_exists          = 'replace',    # safe for repeated pipeline runs
)

Taking ownership after a soft clone¶

take_ownership() on a TimeManager instance can either just flag ownership or physically copy the data into this schema, depending on the create_copy parameter:

# Option A — copy data into this schema and repoint the view
bt_new.take_ownership(create_copy=True, if_exists='replace')

# Option B — just mark as owned (only works when the hidden table is already
#             in this schema with the expected name)
bt_new.take_ownership(create_copy=False)

After create_copy=True, the manager's hidden table is the new local copy; the view is repointed to the active TIME_ID before the call.

Giving processes their own cursor¶

Processes on one manager view share one cursor, so they always compute the same date. To let some of them move on their own (a backfill, a faster cadence), split_time_manager makes a clone and rewrites their views onto it:

tdfs4ds.split_time_manager([pid_cat], 'BD_CATEGORY', clone_mode='soft', dry_run=True)   # the statements
tdfs4ds.split_time_manager([pid_cat], 'BD_CATEGORY', dry_run=False)
tdfs4ds.unify_time_managers([pid_cust, pid_cat], dry_run=False)                         # back to one cursor

'soft' keeps the same dates (the clone reads the source's table); 'hard' copies them. The pairing in FS_TIME_MANAGER and the lineage of the rewritten views are refreshed. A process that reads the manager only through an intermediate view is reported not_direct and left alone: rewrite the intermediate view instead.

The manager registry¶

FS_MANAGER_REGISTRY / FS_V_MANAGER_REGISTRY (optional, 0.3.1.240) record one row per TimeManager and FilterManager object: its view, the table of dates it reads, what it was cloned from and how. load_time_steps, clone_timer, take_ownership, load_filter and clone_filter write it, best effort; a dropped manager's row is ended. It is created by setup() / connect(); when it cannot be created the store works as before and the Report infers clones from the lineage. tdfs4ds.register_managers(dry_run=True) backfills the managers the store already uses (soft clones are recognised from the lineage; hard clones are compared by content in the Report).

Choosing between soft and hard¶

	Soft clone	Hard clone
Data footprint	No extra storage	Doubles the hidden table
Sync with source	Automatic	Independent snapshot
Can drop hidden table	Only with `take_ownership=True`	Always (owns the copy)
Typical use	Multiple processes sharing one calendar	Archived snapshot for audit / replay

Combining with FilterManager¶

roll_out and FilterManager compose naturally:

Setup	Effect
`roll_out` only	One ingestion call per business date
`roll_out` + standard `FilterManager` (on a different column)	One ingestion call per (business date × filter value)
Hybrid `FilterManager` only	Date × partition pre-computed; no `TimeManager` needed

Pick the hybrid FilterManager when the date × partition combinations are sparse and known in advance; pick roll_out when you need a dense daily roll over a continuous range.

Pattern: cloned timer + filtered roll-out

When multiple processes share the same date calendar but each has its own partition filter, clone the timer and attach the relevant FilterManager to each process:

# Both processes use the same calendar
bt_p1 = TimeManager(table_name='FS_BT_PROCESS1', schema_name=tdfs4ds.SCHEMA)
bt_p1.clone_timer(bt_source, clone_mode='soft')

bt_p2 = TimeManager(table_name='FS_BT_PROCESS2', schema_name=tdfs4ds.SCHEMA)
bt_p2.clone_timer(bt_source, clone_mode='soft')

# Process 1 is also segmented by region
fm = FilterManager(table_name='FS_FILTER_REGIONS', schema_name=tdfs4ds.SCHEMA)

# roll_out automatically calls fm.update() on each time step when filtermanager is set
tdfs4ds.roll_out(process_list=[pid_1], time_manager=bt_p1, filtermanager=fm,
                  time_id_start=1, time_id_end=bt_source.nb_time_steps)

# Process 2 rolls without a segment filter
tdfs4ds.roll_out(process_list=[pid_2], time_manager=bt_p2,
                  time_id_start=1, time_id_end=bt_source.nb_time_steps)

Synchronisation in the Report¶

The admin app's Report has a Time drivers section. Per process: the declared manager, the one its view reads, and a status:

Status	Meaning
`OK`	declared and read
`UNDECLARED`	read, not declared: `run(pid)` stamps the real time
`MISMATCH`	declares one manager, reads another
`DECLARED_UNUSED`	declared, not read
`NONE`	no TimeManager
`UNKNOWN`	the lineage graph does not know the view

and the groups: synchronised (one manager view, one cursor), soft clones (one table of dates, own cursors, with whether they point at the same date), hard clones (copies of the dates), independent. Each finding names its fix.

Monitoring¶

tdfs4ds.process_store.process_followup.follow_up_report().sort('START_DATETIME', ascending=False)

The follow-up report lists every run with STATUS, START_DATETIME, END_DATETIME, and the failure message when applicable. See Inspect & monitor.

TimeManager & roll_out¶