tdfs4ds.agent.feature_groups

tdfs4ds.agent.feature_groups

Semantic groups of features, generated once and reused.

The consumer agent lists a large scope (a data domain, an entity) as business groups, each with a label, one sentence and the technical names. The grouping is proposed by the instruct model once per feature set, checked against the catalog (no invented or missing name), and stored in the business dictionary as the section FEATURE_GROUPS of a reserved object (__SCOPE__<domain>[__<entity>]), keyed by a hash of the feature set. Every later answer reuses it, so the same question gets the same answer; a change of the feature set regenerates it.

The section is an internal artefact, not documentation: the vector indexes and the documentation coverage skip it (SECTION_NAME = FEATURE_GROUPS_SECTION).

logger module-attribute

logger = getLogger(__name__)

FEATURE_GROUPS_SECTION module-attribute

FEATURE_GROUPS_SECTION = 'FEATURE_GROUPS'

SCOPE_PREFIX module-attribute

SCOPE_PREFIX = '__SCOPE__'

OTHER_LABEL module-attribute

OTHER_LABEL = {'en': 'Other', 'fr': 'Autres'}

MAX_GROUPS module-attribute

MAX_GROUPS = 10

_MEMO module-attribute

_MEMO: dict = {}

scope_object

scope_object(domain, entity=None) -> str

Reserved OBJECT_NAME of a scope's grouping.

feature_set_hash

feature_set_hash(names) -> str

Order-insensitive hash of a feature set (names upper-cased).

render_text

render_text(hash_, groups) -> str

Stored form: readable, one group per block.

hash=<h> then, per group, ## <label> | <description> and a line of names.

parse_text

parse_text(text)

Inverse of :func:render_text: (hash, groups) or (None, []).

validate

validate(groups, names, other_label='Other')

Keep only real names, each in exactly one group, every name somewhere.

Names are matched case-insensitively and returned as the catalog writes them. A name the model invented is dropped; a name it forgot goes to other_label. Empty groups disappear. Returns (groups, dropped).

_load

_load(obj)

_save

_save(obj, text) -> bool

_generate

_generate(rows, llm_json, fr=False)

Ask the model for groups. rows: [(name, description)].

groups_for

groups_for(domain, entity, frame, llm_json=None, *, fr=False, allow_generate=True, store=True)

Semantic groups of frame's features for a scope, or None.

Reuses the stored grouping when its hash matches the feature set; otherwise asks llm_json (the agent's _llm_call_json), validates and stores it. None when nothing is stored and no model can be asked: the caller then groups by producing process, as before.

clear_memo

clear_memo() -> None

document_feature_groups

document_feature_groups(data_domains=None, force: bool = False) -> dict

(Re)generate and store the semantic groups of every data domain.

The consumer agent does this lazily the first time a large domain is listed; calling it ahead (rebuild_documentation does) makes the first answer as fast as the next ones. force=True regenerates even when the stored grouping still matches the feature set. Needs an instruct model. Returns {domain: number_of_groups or None}; never raises.