2. Documentation¶
A registered feature is useful to its author for about a week. After that — or for anyone else on the team — the SQL needs a business description. tdfs4ds generates those descriptions with an LLM and stores them in a three-level business dictionary that the consumer agent later searches.
Two ways to document¶
Inline at upload time. Pass document=True to upload_features (the
default) and the registered process is documented immediately after it runs.
If you also pass a dataset_view_name, the dataset is documented bottom-up
from its constituent processes in the same call.
Out-of-band, on demand. Call document_process_incremental(process_id)
later — useful when the instruct model wasn't configured yet at upload time,
or after editing the process view.
from tdfs4ds.genai import (
document_process_incremental,
document_dataset_incremental,
)
document_process_incremental(process_id)
document_dataset_incremental(dataset_id)
What gets written¶
| Level | Table | Content |
|---|---|---|
| 0 | FS_BUSINESS_DICTIONARY_OBJECTS |
One business description per registered object |
| 1 | FS_BUSINESS_DICTIONARY_SECTIONS |
Five structured sections per dataset (OVERVIEW, ENTITY, FEATURE_THEMES, BUSINESS_QUESTIONS, INTENDED_AUDIENCE) |
| 2 | FS_BUSINESS_DICTIONARY_COLUMNS |
One description per feature column on the process view |
The hierarchy is designed for chunking-free hierarchical RAG: the consumer agent's SEARCH and DEFINITION skills query these tables directly.
When to choose which path¶
- Always documenting at upload? Keep
document=True. - Iterating on the SQL before you trust the description? Use the out-of-band call — it's idempotent and won't re-document unchanged objects.
Learn more¶
- Process documentation
- Dataset documentation
- Business dictionary
- Recipes › Upload & document
- LLM & embedding models — required config before the LLM path runs.