tdfs4ds.vector_store_selftest¶
End-to-end functional self-test for the Teradata Enterprise Vector Store.
check_configuration()'s Vector Store check proves the service is up, that
the credential is accepted, and that the reported version is supported. None of
that exercises the thing tdfs4ds actually needs: embedding rows and getting
them back. Every failure this package has hit in the field — the ephemeral
_inputdata view (0.3.1.140), update_failed collections written by a
call that raised nothing (0.3.1.139), add_datasets refusing an attached
source (0.3.1.167) — is invisible to a read-only probe and only appears once
something writes.
:func:selftest_vector_store writes. On a throwaway collection over a
throwaway table, it walks the whole lifecycle, records the outcome of each step
independently, and cleans up after itself:
service_version— the versionhealth()reports, vs the supported minimumcreate_source— a small permanent table with anIDand a text columncreate_collection—from_datasets+ confirm the collection reachesreadysearch—similarity_search: are the embedded sentences retrievable?update_source— change one sentence in the source tablerefresh_index— make the collection reflect that change (the contested step)search_after_update— is the NEW text retrievable, and the old text gone?cleanup_collection—destroy(admin-only)cleanup_source—DROP TABLE
Step 6 is the point of the exercise. Resyncing an already-attached source is
the one operation with no reliably working mechanism (see
vector_store_teradata.build_teradata_index's KNOWN LIMITATION):
add_datasets answers 400 "You can only add object names that do not exist in
the collection", and Collection.update(data_refresh=True) — the documented
way to "refresh/regenerate vectors using existing backing/source tables" —
worked once in testing and then failed consistently. Whether that is a bug in a
particular service version is precisely what this test settles, on the
system in front of you, in about a minute.
Contracts (each has a test):
- Never raises. Every step is individually wrapped; a failure is recorded and the walk continues to cleanup.
- Never leaks a secret. No token, password or API key is read into the report or the reproducer — only variable names and non-secret config.
- Cleans up, or says exactly what it left behind.
destroy()is admin-only, so cleanup can legitimately fail; when it does, the report names the leftover objects and the SQL to drop them rather than leaving the user to discover them later. - Unique names per run (
tdfs4ds_selftest_<8 hex>), so a leftover from an earlier failed run can never collide with, or be mistaken for, this one.
On any failure it emits a standalone reproducer (report["repro"]): a
single .py using only teradataml/teradatagenai/langchain_teradata
— never tdfs4ds — containing the steps up to and including the one that broke,
with credentials read from environment variables. That is the artefact to attach
to a support ticket.
module-attribute
¶
module-attribute
¶
_SENTENCES = [('s1', 'SELFTEST_RAINFALL', 'Annual rainfall in the Amazon basin exceeds two thousand millimetres.'), ('s2', 'SELFTEST_ESPRESSO', 'The espresso machine needs descaling after about two hundred cups.'), ('s3', 'SELFTEST_BIRDS', "Migrating birds navigate using the earth's magnetic field."), ('s4', 'SELFTEST_REVENUE', 'Quarterly revenue grew twelve percent in the northern region.')]
module-attribute
¶
module-attribute
¶
_REPRO_STEP_ORDER = ['create_source', 'create_collection', 'collection_details', 'search', 'parse_hits', 'update_source', 'refresh_index', 'search_after_update']
module-attribute
¶
Everything the steps and the reproducer need to know about this run.
Run one step, timing it and converting any exception into a FAILED record.
Once something has failed, later functional steps are skipped — chaining a
search onto a collection that was never created produces a second, derived
error that buries the first. Cleanup steps pass skip_if_failed=False
because they must run precisely when things went wrong.
get_details() — the read-only readiness probe every write waits on.
_wait_until_collection_ready polls it before each write in
build_teradata_index; if it never succeeds, indexing hangs for its full
30-minute timeout rather than failing. Worth its own step.
Return the row ids similarity_search gives back, best first.
Hit shape varies by client build, so ids are recovered leniently: any value
that looks like one of our known ids, from whichever key carries it. The
strict, tdfs4ds-specific reading is :func:_do_parse_hits's job — this one
answers "did the service return anything at all", which is a different
question and must not be conflated with it.
Can tdfs4ds read those hits, or only the raw client?
These are two different failures and only this step separates them. In
0.3.1.140, similarity_search returned eight real, relevant hits while
search_vector_index — the actual public entry point — returned zero
rows, silently: _parse_one looks the text up under a fixed list of
column names and the live hit shape used none of them, so every hit was
dropped by its own empty-text guard. A step that stops at "the service
answered" is blind to that entire class of bug.
The contested step: make an existing collection reflect a changed source.
Both known mechanisms are tried, in the order that does least damage, and the exact outcome of each is reported — this is the evidence the whole self-test exists to produce.
Exercise the Vector Store end to end on throwaway objects.
This writes to the database, unlike every other tdfs4ds diagnostic: it creates a small table and a vector-store collection, then removes both. Names are unique per run, and nothing pre-existing is touched.
Parameters¶
cleanup : bool, default True
Destroy the collection and drop the table at the end. False keeps
both for inspection — the report then names them, and it is on you to
remove them.
display : bool, default True
Print the per-step report.
repro : bool, default True
On failure, build the standalone reproducer into report["repro"].
Returns¶
dict
{"ok", "version", "collection", "table", "steps": [...], "repro",
"leftovers": [...]}. ok is True when no step failed. Each step is
{"name", "label", "status", "detail", "error", "elapsed_s"} with
status ok/failed/skipped.
Client package versions for the reproducer header.
teradatagenai and langchain_teradata expose no __version__
(confirmed live — both rendered as ?), so fall back to the installed
distribution metadata. The version is the first thing support asks for;
"?" makes the header worth less than the space it takes.
Code for each step, in order. Only blocks up to the failure are emitted — a reproducer that continues past the break is not minimal.
A standalone script reproducing the failure, for a support ticket.
Uses only teradataml/teradatagenai/langchain_teradata — never
tdfs4ds, which support will not have — and reads every credential from the
environment, so no secret is ever written into it. It stops at the step
that failed: everything after would be noise.
Force the reproducer to plain ASCII.
It is a file the user writes out (open(path, 'w').write(...)) and hands
to someone else, on a machine whose default encoding we do not control — a
Windows console codepage cannot encode —/→, so a single typographic
character in an echoed server error would make writing the file raise
UnicodeEncodeError. Same reasoning as diagnostics._print_report and
the ASCII-only violation strings in sql_invariants: the artefact that
exists to report a problem must not be able to create one.
Describe the embedding configuration WITHOUT any credential.
Render report as plain text (also used by the admin UI).
Print the per-step report, degrading rather than raising on any console.
Same contract and mechanism as diagnostics._print_report: displaying a
report must never be what breaks a diagnostic run, so a console that cannot
encode the emoji/typography gets ASCII markers and transliterated
punctuation, and anything still un-encodable is replaced, not raised.