tdfs4ds.process_store.process_options¶

tdfs4ds.process_store.process_options ¶

Per-process execution options: FS_PROCESS_OPTIONS (0.3.3.22).

One valid-time row per process, PRIMARY INDEX (PROCESS_ID), holding options that change how run() executes the process. The first one is CLOSE_ABSENT_ENTITIES: when 'Y', every run ends the validity of the entities the process no longer produces (see feature_data_processing.close_absent_entities).

Why a table of its own:

METADATA in the process catalog is descriptive; an option that changes what a run writes is not.
No ALTER of the process catalog, and FS_V_PROCESS_CATALOG does not join this table: its text stays byte-identical, and a store where the table is missing (read-only user, refused CREATE) keeps a working catalog view.
A later option is a new column here.

The table is optional, like the manager registry: created at setup() and by connect() when missing; a failed CREATE is one warning, remembered and not retried in this process. A missing table or row reads as "every option off".

OPTION_COLUMNS `module-attribute` ¶

OPTION_COLUMNS = ('CLOSE_ABSENT_ENTITIES',)

_STATE `module-attribute` ¶

_STATE: dict = {}

_WARNED `module-attribute` ¶

_WARNED: set = set()

sql_literal ¶

sql_literal(value)

Escape value for use inside a single-quoted SQL string literal.

For the DBC probes and catalog predicates that compare a name as data (WHERE DatabaseName = '…'). The caller supplies the surrounding quotes.

_table_name ¶

_table_name() -> str

_qualified_table ¶

_qualified_table() -> str

_key ¶

_key() -> Optional[str]

reset_state ¶

reset_state() -> None

Forget what was learnt about the table (tests, setup(), connect()).

_warn_once ¶

_warn_once(key: str, message: str, *args) -> None

_execute ¶

_execute(sql: str)

process_options_ddl ¶

process_options_ddl() -> str

process_options_creation ¶

process_options_creation(existing: Optional[Iterable[str]] = None) -> bool

Create FS_PROCESS_OPTIONS when missing. Never raises.

existing is the lower-cased object set connect() read from DBC: a table found there costs no CREATE. Returns whether the table is usable.

available ¶

available() -> bool

Whether FS_PROCESS_OPTIONS exists in the primary store (checked once per store).

_flag ¶

_flag(value) -> str

read_options_sql ¶

read_options_sql(process_id: str) -> str

upsert_options_sql ¶

upsert_options_sql(process_id: str, close_absent_entities: bool) -> str

switch_off_sql ¶

switch_off_sql(process_id: str) -> str

Turn an existing option off; a no-op when the process has no row.

delete_options_sql ¶

delete_options_sql(process_id: str, purge: bool = True) -> str

read_process_options ¶

read_process_options(process_id: str) -> dict

The options of process_id: {"close_absent_entities": bool, "track_entity_refresh": bool}.

close_absent_entities is read from FS_PROCESS_OPTIONS (unchanged), the newer options from FS_PROCESS_SETTINGS (key/value, see process_settings). Each table is optional and read on its own: fail-open, no table, no row or an unreadable table reads as that option off, and a missing table never hides the other one.

record_process_options ¶

record_process_options(process_id: str, close_absent_entities: bool) -> bool

Record the options of process_id. Returns True when written (or nothing to write).

True writes the row (UPDATE ... ELSE INSERT). False only switches off an existing row and is skipped when the table does not exist, so a store that never uses the option gets no row. Raises on a database error when writing True.

remove_process_options ¶

remove_process_options(process_id: str, purge: bool = True) -> str

filter_scope ¶

filter_scope(filtermanager, entity_columns) -> tuple

(ok, filter_columns, not_entity) for closing absent entities segment by segment.

A segment of a FilterManager can only tell which stored entities belong to it when every filter column (BUSINESS_DATE of a time-filtering manager aside) is an entity column: the close is then restricted to the stored entities that match the active filter row. Pure.

scope_error ¶

scope_error(not_entity, entity_columns) -> str

set_process_options ¶

set_process_options(process_id: str, close_absent_entities: Optional[bool] = None, track_entity_refresh: Optional[bool] = None, dataset_time_manager: Optional[bool] = None, valid_time_column=None, force_varchar_length=None) -> dict

Change the execution options of an already-registered process.

close_absent_entities=True: every later run() / roll_out() of the process ends the validity of the entities it no longer produces, at the run's valid time. False switches it off. None leaves it as it is.

track_entity_refresh=True: every later run records, in an admin table (FS_R_*, see tdfs4ds.entity_refresh), when each entity was last confirmed by the process. False switches it off (the recorded history is kept).

dataset_time_manager=True: the dataset view a later run() builds follows the process's TimeManager (build_dataset(time_manager=...)); False switches it off.

valid_time_column='<COLUMN>' (0.3.5.0): every later run writes each row of the process view at its own valid time, read from that DATE / TIMESTAMP column (see upload_features). False or '' switches it off. The next run replaces each entity's history from its earliest valid time in the load onward.

force_varchar_length=N (0.3.5.1): the VARCHAR rounding later runs use for features they register ('none' = exact lengths). A registered feature keeps its stored type: a run whose source no longer fits it logs a warning naming tdfs4ds.plan_type_change.

Returns {"process_id", "close_absent_entities", "track_entity_refresh"} (the options now in force). Raises ValueError for an unknown process or a FilterManager whose columns are not entity columns, and RuntimeError when an options table cannot be used.

tdfs4ds.process_store.process_options¶