00Research

Fix Tags$ default filters and add additive +Column syntax

Issue: sue5t · Generated 2026-06-01

01Strategy

Read this first — the rest is detail.

Strategy summary

The issue has two parts; both tractable with existing machinery.

Part 1 (broken Tags$ defaults). After auditing all 1,961 $-suffix entries in default_filters.yaml against live ShapeCache introspection plus a simulated transform_tags_structure + flatten_dict_keys pass, only 5 entries are truly broken in the Tags class: directconnect tags$, ec2 describe_vpcs Tags$, redshift describe_cluster_parameter_groups Tags$, redshift describe_cluster_security_groups Tags$, and ecr describe_images imageTags$. Two extra leftovers (redshift describe_cluster_parameter_groups Key$/Value$) should be deleted. directconnect requires a companion fix to _is_aws_tags_structure because its shape uses lowercase key/value.

Part 2 (additive +Column). Live verification shows +Foo has NO existing grammar meaning, argparse passes it through cleanly, and parse_multi_level_filters_for_mode routes it to the column-filter group untouched. The cleanest integration is at the top of cli.py::determine_column_filters.

  • Strip + in determine_column_filters.

    Single funnel between user input and validator + default merge. Doing it earlier (in parse_filter_pattern or parse_multi_level_filters_for_mode) leaks CLI semantics into the grammar layer — D7 forbids it.

  • Order-preserving dedup with dict.fromkeys.

    Python 3.7+ language spec; project requires 3.10+. set() would silently scramble order.

  • Patch _is_aws_tags_structure for lowercase keys.

    Without this, replacing tags$ with tags.Name for directconnect will still show empty Name columns. The transform never fires for case-variant member names.

  • Re-runnable audit script, not a YAML snapshot.

    Boto3 ships new shapes every release; a snapshot drifts. The audit is a candidate-broken detector with documented false-positives — manual review of the ~33 flagged entries is required.

1,847
Audit: correct
5–7
Truly broken
1
Companion bug (directconnect)

Primary recommendation

Implement Part 2 (additive +Column) first — tightly scoped, clear tests. Then Part 1 (audit + fixes + lowercase-key patch). Use @agent-test-writer, @agent-python-infra-automator, and @agent-code-reviewer per CLAUDE.md mandate.

02Flow

Task flow & dependencies

Phase A.1 Extend apply_default_filters Add additive=False kwarg; merge defaults + user cols when truthy.
Phase A.2 Strip + in determine_column_filters Partition into (additive_present, stripped_user_cols); route to additive branch.
Phase A.3 Tests + stderr echo test_default_column_filters.py additions; update _format_columns_copyable for additive view.
Phase B.1 Add scripts/audit_default_filters.py Heuristic auditor reusing ShapeCache; baseline report.
Phase B.2 Edit YAML + patch _is_aws_tags_structure 5 targeted YAML fixes + 2 leftover deletions + case-variant transform fix.
Phase B.3 Tag transform tests Add lowercase-key fixture; re-run audit; assert broken count drops.
Phase C README + code review Document +Column; @agent-code-reviewer sweep.

03Decisions

Key decisions & options

Where +-stripping lives

Chosen

At the top of cli.py::determine_column_filters — single funnel between user input and validator + default merge.

Inside parse_filter_pattern — would leak CLI semantics into the filter grammar (D7 forbids).

Inside parse_multi_level_filters_for_mode — affects both single/multi modes uniformly; + applies only to column-filter group (D3).

Dedup mechanism

Chosen

list(dict.fromkeys(defaults + stripped_user_cols)) — order-preserving, language-spec-stable on Python 3.7+.

set() — no order guarantee; project is 3.10+ but the spec is what matters.

directconnect lowercase tags fix

Chosen

Patch _is_aws_tags_structure and _transform_aws_tags_list in formatters.py to accept case-variant Key/Value. Add a unit-test fixture with lowercase-key tags.

Skip the directconnect substitution and document the limitation — user-visible inconsistency.

Substring filter tags.key — surfaces all key/value pairs; very noisy.

Audit as code, not snapshot

Chosen

Re-runnable scripts/audit_default_filters.py. Heuristic with documented false-positives; humans review the flagged candidates.

Snapshot of corrected default_filters.yaml — drifts when boto3 ships new shapes.

Gate CI on broken-count == 0 — false positives (e.g. value$ for top-level list-of-primitive data fields) would block legitimate config.

Graphify usefulness verdict

Affirmed

Worth integrating into research by default — but USE THE JSON, not the CLI query. Direct links/nodes traversal in Python answers all four D6 questions in seconds with HIGH accuracy. graphify query is keyword-BFS and noisy; graphify explain resolves to tests first.

ssm.get_parameters Parameters$ — out of scope

Deferred

Audit flagged it as broken (data-field extraction strips Parameters before flatten). Unrelated to Tags or additive — defer to a separate follow-up issue. Do not scope-creep.

04Detail

Full detail

Show full artifact source click to expand

05Provenance

Provenance

Generated from the canonical .md. This HTML is never hand-edited — if the .md changes, regenerate with /issue:view.