tdfs4ds.agent.eval_facts¶
Ground truth for the agent evaluation, read from the store (0.3.1.240).
Every expected fact of an evaluation turn is computed here, from the store,
through the same authorities the Report and the agent use: the catalog snapshot
at a valid time (store_consistency.collect_catalog_snapshot), the one count
definition (store_consistency.catalog_counts), the follow-up in business dates
(followup_business_dates), the persisted lineage graph at a valid time, the
business dictionary, and the process view DDL (sql_context). A disagreement
between the agent and these facts is then the agent's fault, not a second
definition's.
The facts are not the answer. They are what the judge checks the answer
against: what must appear (names, figures, dates), what must not appear
(absent_names: objects that did not exist at the turn's valid time), and the
valid time the turn is about.
Layers:
- :func:
collect_eval_facts- the only I/O: the store at a valid time. - :func:
build_facts- pure:(fact_spec, store) -> expected_facts. A turn keeps itsfact_spec, so the facts can be rebuilt at run time and a stale set detected (run_evaluation(refresh_facts=...)). - :func:
catalog_change_points, :func:pick_time_point- pure helpers choosing dates where the store at that date really differs from now.
module-attribute
¶
_STOP = {'the', 'and', 'for', 'with', 'that', 'this', 'from', 'into', 'over', 'each', 'per', 'are', 'was', 'were', 'has', 'have', 'its', 'their', 'which', 'when', 'where', 'what', 'how', 'number', 'value', 'values', 'feature', 'features', 'computed', 'calculated', 'based', 'le', 'la', 'les', 'des', 'une', 'pour', 'par', 'dans', 'sur', 'avec', 'est', 'sont', 'total', 'average', 'mean', 'count', 'sum', 'data', 'given', 'used', 'using', 'between', 'column', 'represents', 'indicates', 'counts', 'measures', 'reflects', 'provides', 'within', 'across', 'specific', 'overall', 'calculates', 'calculated', 'number', 'amount', 'amounts', 'colonne', 'représente', 'indique', 'nombre', 'montant', 'chaque'}
module-attribute
¶
_BOILERPLATE = compile('^\\s*(this|the)\\s+(column|feature|value|metric|field|variable)\\s+(represents|captures|shows|indicates|measures|reflects|provides|gives|is|contains|counts|records)\\s+(the\\s+|a\\s+|an\\s+)?', I)
module-attribute
¶
_BUILDERS = {'definition': _b_definition, 'explain': _b_explain, 'freshness': _b_freshness, 'lineage': _b_lineage, 'usage': _b_usage, 'dataset': _b_dataset, 'summary': _b_summary, 'data_domains': _b_data_domains, 'search': _b_search, 'feasibility': _b_feasibility, 'before_store': _b_before_store, 'not_yet': _b_not_yet, 'dataset_time': _b_dataset_time, 'static': _b_static, 'object': _b_object}
The store at a valid time (None = now), for :func:build_facts. Fail-open:
a part that cannot be read is empty and its facts are skipped.
The manager's current date and how many of its dates lie after the last one a process completed (independent of the agent's own computation).
One record per feature triplet at the store's valid time.
Datasets reading this feature (triplet, through FEATURE_ID), with entity keys.
Feature names and process view names (upper) present in the store.
Names that exist now but did not at store_at's valid time: an answer about that time naming them describes a later state.
Registration instants of every feature and process (UTC, sorted).
Instants processes were registered or ran (UTC): real time, unlike feature catalog rows, which a TimeManager-driven run stamps with its business date.
A valid time between two registration instants, so the store at that time
differs from now: the middle of the widest gap, as YYYY-MM-DD when the gap
spans a midnight (the midnight after the earlier instant), else to the second
(UTC). None when every object was registered at once.
A date before anything was registered (the day before the first instant).
Forms a date may take in an answer: ISO, '30 June 2023', 'June 30, 2023', '30 juin 2023', '30/06/2023'.
content_words(text: str, exclude: Iterable[str] = (), n: int = 4, frequency: Optional[Dict[str, int]] = None) -> List[str]
Words of a description that say what it is about. With frequency (how many descriptions of the store use each word), the rarest come first: they single the description out, where 'customer' or 'transaction' describe half the store.
What a description is about, as a user would ask for it: its first sentence without the 'This column represents the' opening, cut at n_words.
In how many of texts each word appears.
Features carrying the same name (other entities, other domains).
Identity facts. spec['scope'] == 'domain': the question names the domain;
spec['entity_known']: the conversation already fixed the entity (a pronoun
follow-up). Otherwise a name shared by several triplets expects the agent to
ask which one (or to answer for each).
Expected facts of one turn. store is the store at the spec's valid time;
store_now the current store (for absent_names). Raises nothing: an
object no longer found yields {"missing_object": ...} (the turn is stale).
Stable digest of a facts dict (keys sorted, valid time included).