tdfs4ds.data_domain_settings¶
One clock per data domain: FS_DATA_DOMAIN_SETTINGS (0.3.6.0).
A data domain may declare a domain clock: a TimeManager whose dates every process
of the domain follows. Each process then reads its own soft clone of it (same table
of dates, own cursor), made by upload_features:
roll_out(data_domain=...)moves the clock and every clone date by date;run([...])/run(data_domain=...)computes every process at one fixed time, swapping each clone for that date during the computation (seeutils.time_management.literal_time_managers);- a partial run never moves another process's cursor.
Storage: a key/value valid-time table, PRIMARY INDEX (DATA_DOMAIN), the
FS_PROCESS_SETTINGS pattern (whose key is a PROCESS_ID, too short for a domain name).
It is optional: created at setup() and by connect() when missing, a failed CREATE
is one warning, remembered and not retried. Without the table (or without a row) a
domain has no clock and everything behaves as before 0.3.6.0. Every INSERT names its
columns, so a later setting is a new SETTING_NAME, never a new column.
Escape value for use inside a single-quoted SQL string literal.
For the DBC probes and catalog predicates that compare a name as data
(WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.
Forget what was learnt about the table (tests, setup(), connect()).
Create FS_DATA_DOMAIN_SETTINGS when missing. Never raises.
existing is the lower-cased object set connect() read from DBC: a table found
there costs no CREATE. Returns whether the table is usable.
Whether FS_DATA_DOMAIN_SETTINGS exists in the primary store (checked once per store).
Record (or change) one setting. Every column is named.
The JSON recorded for a TimeManager: where its view and its table of dates are.
The dict recorded by :func:clock_value, or None for a missing / unreadable value.
{SETTING_NAME: SETTING_VALUE} of data_domain. Fail-open: no table, no row or an
unreadable table is an empty dict.
{database, view, table_database, table} of the domain clock, or None. Never raises.
The TimeManager of the domain clock (rebuilt read-only), or None when the domain
has none. data_domain=None = tdfs4ds.DATA_DOMAIN.
Declare time_manager the clock of a data domain.
From then on upload_features gives each process of the domain that reads this
manager its own soft clone of it (same dates, own cursor), roll_out(data_domain=...)
moves the clock and every clone date by date, and run([...]) /
run(data_domain=...) computes every process at one time.
The clock must own its table of dates (a manager, not a soft clone of another one).
Processes registered before keep their manager; align_domain_time_managers shows
what an adoption would change. data_domain=None = tdfs4ds.DATA_DOMAIN;
dry_run=True returns the statement without writing it. Returns
{data_domain, clock, sql, written}.
Remove the clock of a data domain (data_domain=None = tdfs4ds.DATA_DOMAIN). The
processes keep their soft clones (each one is an ordinary TimeManager); only the automatic
cloning and the domain-wide roll-out stop. dry_run=True returns the statement without
executing it.
Whether time_manager is the clock itself (its view), compared without case or quotes.
Whether time_manager reads the clock's table of dates (the clock or one of its clones).
Process ids registered in data_domain in the primary store (read-only stores are
never run). data_domain=None or True = tdfs4ds.DATA_DOMAIN. proc_df: an
already read process catalog (PROCESS_ID, DATA_DOMAIN), else it is read.
Name of the soft clone a process gets: <process view>_TM (cut to 128 characters).
align_domain_time_managers(data_domain=None, process_ids=None, *, dry_run: bool = True, graph=None, proc_df=None)
Give the processes registered before the domain clock their own soft clone of it.
For each process of the domain (or of process_ids) whose view reads the clock
itself, a soft clone <view>_TM is made and the process view is rewritten to read
it (split_time_manager). Processes reading a clone already, or no manager, or a
manager on other dates, are reported and left alone. Returns a pandas DataFrame
(PROCESS_ID, VIEW_NAME, READS, ACTION, DETAIL); dry_run=False
applies the CLONE rows. data_domain=None = tdfs4ds.DATA_DOMAIN; graph /
proc_df: an already loaded lineage graph / process catalog (read when omitted).
The views between view_name and the TimeManager manager ({database, view})
that name the manager, each with the processes that read it (0.3.6.8): the reason a
process reading the manager through them cannot get its own clone. The views come from
the process view's own lineage (SHOW VIEW), the readers from FS_LINEAGE_GRAPH.
Fail-open: [] when a lineage cannot be read.
shared_view_message(process_id: str, view_name: str, manager: Dict[str, str], shared: List[Dict]) -> str
The warning for a process that reads a TimeManager through a view other processes
share (0.3.6.8): what it means and the three ways out, none of them done automatically.
One text, used at registration and by align_domain_time_managers.
clone_for_process(process_id: str, view_name: str, clock: Dict[str, str], *, dry_run: bool = False) -> Dict
Make the soft clone of the clock for one registered process and repoint its view.
The clone goes next to the process view (its database), named <view>_TM, at the
clock's current step (clone_timer(clone_mode='soft', if_exists='skip'): an existing
clone is reused). The process view is then rewritten to read the clone, its
FS_TIME_MANAGER row records the clone and its lineage is refreshed
(scheduling.unify._repoint_process_time_manager, what split_time_manager uses).
A process view that does not name the clock itself (it reads it through an
intermediate view, which other processes may share) is not rewritten: it keeps the
clock (not_direct; shared names the views and their readers, 0.3.6.8). An
existing object named like the clone that is not a soft clone of the clock's dates is
neither replaced nor used (refused). Never raises:
{ok, clone, detail, not_direct, shared[, refused]}.