Library Megamaid Extraction Design

Purpose

Library megamaid extraction turns a KiCad project into inspectable local libraries, extracted 3D models, and JSON metadata for downstream library import.

This design exists because project-local KiCad assets and HTTP library placement metadata are useful during design, but canonical library promotion needs a cleaner split: bare reusable symbols and footprints, metadata preserved in JSON, model payloads that do not depend on external KiCad install paths, and stable linkage between placed instances, symbols, footprints, and part-record candidate metadata.

Boundary

kicad_monkey is responsible for low-level KiCad file operations: scanning project assets, classifying symbol and footprint sources, classifying model references, extracting symbols and footprints, stripping metadata by policy, repairing standalone footprint embedded model payloads, writing folder-based symbol libraries and .pretty libraries, and emitting scan and extraction diagnostics.

kicad_cruncher is responsible for workflow commands that assemble those operations into cleaned megamaid ingestion bundles and metadata-preserving project-lib bundles. Downstream library tooling is responsible for part-record creation and update behavior.

Public API Candidates

The first kicad_monkey implementation should promote small, testable helpers before exposing a large workflow object:

• scan_project_assets(project_path)
• scan_3d_models(project_path)
• extract_symbols(project_path, mode)
• strip_symbol_metadata(symbol, policy)
• extract_footprints(project_path, mode, embed_models=True)
• strip_footprint_metadata(footprint, policy)
• rehydrate_embedded_model_payloads(board, footprint)
• extract_3d_models(project_path)
• extract_3d_models_from_footprint_records(records, output_dir)
• write_symbol_folder_library(...)
• write_pretty_library(...)

These APIs should return structured diagnostics rather than only writing files.

Extraction Modes

Internal Mode

• Dedupe symbols by KiCad library symbol name.
• Strip symbol metadata to KiCad-required/default fields.
• Strip footprint metadata and placement/net-specific board data.
• Preserve geometry, pins, pads, models, and embedded model payloads.
• Emit metadata associations in JSON rather than symbol or footprint fields.

Project-Local Mode

• Preserve metadata.
• Create inspectable project-local symbols and footprints.
• Support per-part assets even when multiple canonical parts share one generic symbol or footprint.

Model Handling

The model scanner must classify:

• kicad-embed://... references;
• project-relative references;
• absolute references;
• ${KICAD*_3DMODEL_DIR} references;
• unresolved or missing references.

When an extracted standalone footprint references kicad-embed://name.step, the matching embedded payload must be present inside the .kicad_mod. If the payload only exists at board level, extraction must rehydrate the footprint from the board embedded-file map.

Extracted STEP/STP files should be deduped by content hash.

Project health scans use the active project board/schematic pair plus local symbol and footprint libraries listed in the project library tables. They intentionally ignore unrelated recursive KiCad files under the project directory that are not part of those local library tables. Standalone footprint-library health checks use a model-only S-expression projection path instead of hydrating full footprint objects.

JSON Output

The workflow JSON should preserve enough data for downstream library import:

• reference designator, value, sheet path, and sheet UUID path;
• source symbol and footprint identities;
• raw fields and canonical fields;
• MPN, manufacturer, and cad-reference candidates;
• selected MPN and selected cad-reference when obvious;
• DNP, excluded-from-board, and on-board flags;
• model references, extracted payload records, and diagnostics.

kicad_cruncher should not create part records directly.

Fixture

The primary real-world fixture is:

tests/corpus/.unpacked/kicad/projects/4-ch-backplane

The preferred project file is:

input/4-ch-backplane.kicad_pro

The fixture should cover:

• local Libs/speedy.pretty;
• local and vendor symbol libraries;
• vendor footprint libraries;
• kicad-embed://... model references;
• external ${KICAD10_3DMODEL_DIR} and ${KICAD9_3DMODEL_DIR} references;
• board-level embedded files;
• footprint-level embedded files;
• missing or non-rehydrated embedded payload diagnostics;
• hierarchy and sheet on_board behavior already covered by the netlist corpus.

Test Requirements

Parser-Level Tests

• Verify model-reference classification.
• Verify symbol and footprint extraction counts for the fixture.
• Verify metadata stripping rules.
• Verify project-local metadata preservation.
• Verify every extracted kicad-embed://... footprint model has a matching embedded payload.
• Verify generated JSON contains raw and canonical metadata fields.

KiCad CLI Validation Tests

• Run only when kicad-cli is available.
• Verify generated symbol and footprint libraries are loadable by KiCad.
• Skip with a clear reason when KiCad CLI is unavailable.
• Do not replace parser-level assertions.

Rack Ownership

• Fast lane: focused parser/unit tests with small synthetic fixtures.
• Full lane: 4-ch backplane scan and extraction tests.
• Strict lane: KiCad CLI validation and complete megamaid/project-lib bundle checks.

Future Work

• sym-lib-table and fp-lib-table management;
• schematic symbol relinking;
• PCB footprint relinking;
• parameter coalescing and cleanup;
• native KiCad netlist/export comparison;
• downstream import tests for part-record creation.