2026-07-17 Performance Optimization Requirements

Status

These requirements are the durable release-facing record for the parser and PCB projection optimization sweep released as 2026.7.17 and associated with GitHub issues #16 and #17. Public PRs #18 and #19 are research inputs only; their code is not accepted, merged, cherry-picked, copied, or ported into this implementation.

Requirements

ID	Requirement	Acceptance Evidence
PERF-REQ-001	Use public issue and PR reports as research signals only.	ADR-005, this requirement record, tests, and the changelog state that implementation is independently rewritten on the feature branch.
PERF-REQ-002	Reduce nested projection source-span rebasing cost without changing source metadata semantics.	`KiCadPcbProjection` uses an internal line-column index, and L0 tests assert exact offsets, line/column values, source text, and reparsed S-expression content.
PERF-REQ-003	Improve S-expression lexer and projection scanner throughput in pure Python.	`kicad_sexpr` keeps the public token and span contracts while using a compiled regex tokenizer for whitespace, parentheses, quoted strings, grouped numeric tokens, and atoms; quoted strings unescape lazily and no new native dependency is introduced.
PERF-REQ-004	Reuse direct child-span scans during PCB projection hydration.	Projection caches direct child spans per parent and filters by requested head while preserving object-to-source span attachment tests.
PERF-REQ-005	Reuse PCB net lookup tables while preserving KiCad v8/v9 and v10 fallback behavior.	Full parse and projection paths reuse net lookup maps, and L0 tests cover KiCad v10 name-only net references with no top-level net table.
PERF-REQ-006	Record before/after performance impact with public or synthetic evidence.	This requirement record preserves the citable synthetic, WREN, and Jumperless three-round timing summary; local internal corpus timings are treated as non-citable research evidence.
PERF-REQ-007	Keep performance benchmarks advisory rather than adding slow release gates.	Focused L0, L1 corpus, L99, and dev-std audit checks gate behavior; the multi-minute benchmark remains a review artifact.
PERF-REQ-008	Require external review and explicit release authorization before publishing.	External review approved the implementation after the regex tokenizer rewrite, and no tag, release, PyPI upload, or publish workflow is run from this branch without explicit authorization.

Validation

uv run --extra test python -m pytest tests/L0_foundation/test_L0_033_kicad_sexpr_parser.py tests/L0_foundation/test_L0_041_sexpr_projection.py tests/L0_foundation/test_L0_042_pcb_projection.py
uv run --extra test python tests/rack.py run L0_foundation
uv run --extra test python tests/rack.py run L99_signoff
uv run --extra test python -m pytest tests/L1_parsing/test_L1_020_pcb_projection_corpus.py
uv run dev-std audit . --format json

Measured Impact

Post-regex three-round signoff shows the largest public-corpus gains on WREN nested projection: 126.520s -> 20.859s (6.07x) and route projection 36.382s -> 19.979s (1.82x). Jumperless nested projection improves 33.024s -> 8.974s (3.68x). Synthetic nested projection improves 1.880s -> 0.549s (3.42x). Full-parse results remain mixed after the regex rewrite because token object and generic tree materialization dominate once raw scanning is faster; the earlier 3-8x full-parse expectation is not claimed. Internal 4-ch results are retained as local research evidence only.

External Review Evidence

External review rejected the first optimization draft until the regex-tokenizer rewrite and line-column index follow-ups were complete. The post-regex review then approved the behavior-preserving implementation. A follow-on fresh decomposition on the public Jumperless board measured 8,350,129 raw regex matches, 5,236,594 tokens, best raw scan 1.119s, best token production 9.541s, and best full parse_sexp 11.654s. An independent reviewer reproduced the same match and token counts and confirmed that token object plus generic tree materialization, not raw scanning alone, is the remaining full-parse floor. These review findings support PERF-REQ-008 and explain why pull-parser or streaming typed-parser work is deferred to a separate architecture decision rather than claimed in this optimization release.

Deferred Work

Native acceleration, pull-token parser architecture, direct streaming typed parse paths, file-level IR/SVG APIs that avoid a full KiCadPcb materialization step, and acceptance of third-party PR implementation code are outside this optimization release. User-facing project API guidance is canonical in docs/guides/project-workflows.html. Native acceleration is internal roadmap work: a follow-on plan should evaluate pure-Python pull parsing before C, C++, or Cython tokenizer/parser acceleration, and any native implementation must remain a validated accelerator behind the Python reference contracts.