Skip to content

← Training overview

13. Migration manifest

A feature store built in DEV eventually needs to land in STAGING and PROD. The migration manifest turns the lineage graph from Module 12 into a JSON document you can replay in a target environment — a portable, declarative bill of materials.

What the manifest contains

from tdfs4ds.lineage import (
    build_teradata_dependency_graph,
    graph_to_migration_manifest,
)

graph = build_teradata_dependency_graph(
    sql_query="SELECT * FROM DEV_DB.V_CUSTOMER_360",
)
manifest = graph_to_migration_manifest(graph, filter_database='DEV_DB')

The manifest is a dict with three lists:

Section Content
views Every view's name, schema, DDL, and dependency list
tables Source tables with PI and partition columns
edges The directed dependency edges, filtered to the target database

filter_database='DEV_DB' restricts the export to objects in that schema — edges that cross the boundary are excluded so the manifest is a self-contained slice you can apply elsewhere.

Replay it

A typical end-to-end migration:

  1. Source side. 
    import json
with open('manifest.json', 'w') as f:
    json.dump(manifest, f, indent=2, default=str)
  2. Target side. 
    tdfs4ds.connect(host='prod.example.com', user='prod_user', password='...')
tdfs4ds.create_data_domain('CUSTOMER_360')
tdfs4ds.setup()
  3. Re-create the views in dependency order (the manifest already lists them sorted) — for each entry, run its DDL against the target.
  4. Re-run each process — tdfs4ds.run(process_id) re-ingests features under the target schema using the same SQL.
  5. Rebuild datasets — build_dataset(...) from the manifest entries.

Compare two environments

Manifests are JSON, so a regular diff highlights drift between environments:

diff dev-manifest.json prod-manifest.json

A missing view, a stale partition clause, an extra dependency — all visible in plain text.

Composing with the rest of the training

  • Module 8 › Partitioning — the manifest preserves primary_index and partition_columns so PROD ends up partitioned the same way as DEV.
  • Module 10 › Catalogs and datasets — after replay, the target catalogs hold the same PROCESS_ID and DATASET_ID rows, which keeps lineage and documentation consistent across environments.
  • Module 11 › Logs — the follow-up table is per-environment, so each replay creates its own RUN_ID rows in PROD.

Learn more

That's the end of the training journey. From here, the Concepts section explains the underlying architecture, Configuration covers every knob in detail, and the API reference documents every public function.