# The shiboken6 bindings

How `_pyside6_scintilla` is built, what shiboken generates, problems
encountered while wrapping `ScintillaEditBase` (Phase 1), and how to update
everything when a new Scintilla release is vendored.

## How it fits together

- `src/scintilla/` -- vendored Scintilla release tarball, unmodified (see
  `.gitattributes`; see [docs/auditing.md](auditing.md) for how to verify
  this).
- `src/scintilla_qt/CMakeLists.txt` -- builds the vendored Scintilla core plus
  its Qt widget glue (`PlatQt`, `ScintillaQt`, `ScintillaEditBase`) as a shared
  library, `scintilla_qt`.
- `src/pyside6_scintilla/bindings/`
  - `bindings.h` -- header shiboken's ApiExtractor parses to discover the C++
    declarations referenced by `bindings.xml`.
  - `bindings.xml` -- the shiboken typesystem: declares which C++
    types/enums/classes get Python wrappers.
  - `CMakeLists.txt` -- runs shiboken to generate the wrapper sources below,
    then compiles and links them (with `scintilla_qt` and Qt) into
    `_pyside6_scintilla.{pyd,so,dylib}`.
- `src/pyside6_scintilla/__init__.py` -- re-exports the compiled extension's
  public API as the `pyside6_scintilla` package.

## Generated files

shiboken6 generates these files (under
`build/<tag>/src/pyside6_scintilla/bindings/_pyside6_scintilla/`) from
`bindings.h` + `bindings.xml`. They are build artifacts, not checked in, and
are not hand-edited or hand-commented.

| File | Purpose | Generated by |
| --- | --- | --- |
| `_pyside6_scintilla_module_wrapper.cpp` | Module init (`PyInit__pyside6_scintilla`); registers all wrapped types with the extension module | shiboken6, from the `<typesystem>` root |
| `scintilla_wrapper.cpp` / `.h` | Wrapper for the `Scintilla` namespace and its enums (`Update`, `Message`, `KeyMod`, ...) | shiboken6, from `<namespace-type name="Scintilla">` |
| `scintilla_notificationdata_wrapper.cpp` / `.h` | Wrapper for the `Scintilla::NotificationData` value type | shiboken6, from `<value-type name="NotificationData">` |
| `scintillaeditbase_wrapper.cpp` / `.h` | Wrapper for the `ScintillaEditBase` widget's public methods/signals/slots | shiboken6, from `<object-type name="ScintillaEditBase"/>` |
| `_pyside6_scintilla_python.h` | Master header included by every wrapper `.cpp`; includes `bindings.h` plus the PySide6/shiboken headers it needs | shiboken6 |

The `GENERATED_SOURCES` list in `bindings/CMakeLists.txt` only needs the
`.cpp` files -- the `.h` files are pulled in by those `.cpp` files.

`_pyside6_scintilla_python.h` is the key one to remember: it has its own
include list, separate from `bindings.h`. Anything done in `bindings.h` (e.g.
`#define`/`#undef` tricks) only affects what shiboken's ApiExtractor *sees*
while generating code -- it has no effect on what the generated `.cpp` files
*compile* against. Fixes that need to apply at compile time belong in
`bindings/CMakeLists.txt` (e.g. as `target_compile_definitions`).

## Issues encountered (Phase 1: `ScintillaEditBase`)

### 1. Duplicate `Scintilla::Message` enum codegen

`ScintillaStructures.h` forward-declares `enum class Message;` ("in case
ScintillaMessages.h isn't included") in addition to the full definition in
`ScintillaMessages.h`. Both are in `namespace Scintilla`, and
`<enum-type name="Message"/>` matches both. shiboken's clang-based parser
treats them as two separate `EnumDecl`s and emits duplicate
converters/type-index entries for `Scintilla::Message`
(`SBK_Scintilla_Message_IDX` defined twice, etc.) -- this fails to compile.
Reordering includes does not help.

**Fix**, in `bindings.h`: include `ScintillaMessages.h` first (the real
`Message` definition), then wrap `#include <ScintillaStructures.h>` in
`#define Message ScintillaMessageFwdDeclUnused` / `#undef Message` so its
forward declaration becomes an unrelated, unused enum instead of a second
`Scintilla::Message`. As a side effect, `NotificationData::message` (whose
type is now that renamed dummy enum) has to be dropped via
`<modify-field name="message" remove="true"/>` on `NotificationData` in
`bindings.xml`.

A leftover, mostly cosmetic warning remains: "Scoped enum
`Scintilla::ScintillaMessageFwdDeclUnused` does not have a type entry" --
consistent with several other pre-existing warnings for unbound scoped enums
and safe to ignore.

### 2. `FindText`/`FindTextW` macro collision (Windows)

`<windows.h>` is pulled in transitively via `Python.h` *before* any Qt header
gets a chance to define `WIN32_LEAN_AND_MEAN`. That drags in `<commdlg.h>`,
which `#define`s `FindText` to `FindTextA`/`FindTextW`. This mangles
`Scintilla::Message::FindText` in the *generated* wrapper `.cpp` files -- and
since those files don't include `bindings.h` (see "Generated files" above), an
`#undef FindText` there has no effect on the compile step.

**Fix**, in `bindings/CMakeLists.txt` (Windows only):
`target_compile_definitions(_pyside6_scintilla PRIVATE WIN32_LEAN_AND_MEAN)`.
This stops `<commdlg.h>` from ever being pulled in, so `FindText` is never
redefined.

### 3. `Qt6Core5Compat.dll` not bundled by the PySide6 wheel (Windows)

`scintilla_qt` links `Qt6::Core5Compat` (vendored Scintilla's `PlatQt.cpp`
uses `QTextCodec`, removed from `QtCore` in Qt6). `Qt6Core5Compat.dll` is the
one `Qt6*.dll` the PySide6 wheel does *not* bundle, so
`_pyside6_scintilla.pyd` -> `scintilla_qt.dll` -> `Qt6Core5Compat.dll` fails to
load at import time.

**Fix**, in `bindings/CMakeLists.txt` (Windows only):
`install(FILES $<TARGET_FILE:Qt6::Core5Compat> DESTINATION pyside6_scintilla)`
ships `Qt6Core5Compat.dll` alongside `scintilla_qt.dll`/`_pyside6_scintilla.pyd`,
where Windows' default same-directory DLL search finds it. Also requires
`scintilla_qt` itself to be installed
(`RUNTIME`/`LIBRARY DESTINATION pyside6_scintilla`), and
`src/pyside6_scintilla/__init__.py` to `import PySide6.QtWidgets` *before*
importing the compiled extension, so PySide6 registers its own DLL search
directory for `Qt6Widgets.dll`/`Qt6Core.dll`/`pyside6.abi3.dll`/
`shiboken6.abi3.dll` first.

**Not yet addressed for Linux/macOS wheels**: `Qt6Core5Compat` bundling there
will need `delvewheel`/`auditwheel`/`delocate` in CI, or an
`install(FILES ...)` step analogous to the Windows one.
`_pyside6_scintilla`'s `INSTALL_RPATH` is already set to `$ORIGIN`
(Linux)/`@loader_path` (macOS) in anticipation of this.

### 4. `ScintillaEditBase`/`ScintillaEdit` signals carrying `Scintilla::Position`/enum-typed parameters never reach a Python slot

shiboken marshals `Scintilla::Position`/`ModificationFlags`/`FoldLevel`/etc.
fine for *ordinary* method arguments (they're registered as
`<primitive-type>`/`<enum-type>` in `bindings.xml`), but Qt's generic
meta-call path -- used to invoke a Python slot from a C++ signal emission --
can't convert them. Connecting a Python slot to e.g.
`ScintillaEditBase.modified` looked like it worked (`connect()` raised
nothing), but the slot was never actually called: Qt's event loop swallows
the resulting `TypeError`. Registering these as Qt meta-types was considered
and rejected in favor of re-emission (below), to avoid baking
Scintilla-specific `Q_DECLARE_METATYPE` registration into a generic binding
layer.

**Fix**: `bindings/scintilla_signal_fixes.h` (a hand-written, non-vendored
header, included from `bindings.h`) declares two subclasses,
`ScintillaEditBaseFixed`/`ScintillaEditFixed`, that connect each affected
signal in C++ to a lambda re-emitting the same data as a plain-typed
(`int`/`QByteArray`/`QString`) signal of the *same name* -- the same pattern
already used by `ScintillaDocument::modified` (see
`src/scintilla/qt/ScintillaEdit/ScintillaDocument.h`). PySide creates a
`Signal` descriptor for each signal it finds in a class's own `QMetaObject`,
so the subclass's redeclaration simply shadows the inherited
broken-signature one through normal Python attribute/MRO lookup -- the same
mechanism that lets a Python subclass override a method by just redefining
it, no typesystem changes involved. `src/pyside6_scintilla/__init__.py`
imports the `Fixed` classes aliased to the public
`ScintillaEditBase`/`ScintillaEdit` names, so this is invisible from the
public API.

A few non-obvious wrinkles along the way:

- **`generate="no"` breaks header include order.** The first attempt set
  `generate="no"` on `ScintillaEditBase`/`ScintillaEdit` (since
  `ScintillaEditBaseFixed`/`ScintillaEditFixed` are the only Python-visible
  classes) but that drops the header from shiboken's "Bound library
  includes" list in the generated module header, which is sorted
  alphabetically -- losing `ScintillaEditBase.h` (which internally includes
  `ScintillaTypes.h` before `ScintillaStructures.h`) left `ScintillaStructures.h`
  included before `ScintillaTypes.h` with nothing to satisfy its need for
  `Position` first. **Fix**: keep `ScintillaEditBase`/`ScintillaEdit` fully
  bound (default `generate="yes"`) -- a `<modify-function remove="all"/>` on
  the broken signal overloads was tried too, on the assumption it'd avoid an
  ambiguous Python overload, but turned out to be a no-op (see the next
  point) and was dropped as dead weight.
- **`<modify-function>` doesn't apply to Qt signals.** Unlike ordinary
  methods, Qt signals are exposed to Python via PySide's own metaobject
  introspection at class-creation time, not via shiboken's
  typesystem-driven method generation -- so a `<modify-function
  signature="..." remove="all"/>` targeting a signal silently has no effect
  (confirmed: removing the dozen `remove="all"` rules entirely, the build
  and full test suite behaved identically). The fix above never needed
  them -- the shadowing happens purely through normal Python class
  attribute/MRO lookup, the same mechanism a Python subclass uses to
  override any other inherited attribute by just redefining it.
- **moc can't parse the vendored headers.** `ScintillaEditBaseFixed`/
  `ScintillaEditFixed` need `Q_OBJECT`, so CMake's `AUTOMOC` runs moc over
  `scintilla_signal_fixes.h` -- but moc's lightweight parser chokes on
  `ScintillaStructures.h`'s more intricate C++ once it's transitively
  included. moc only actually needs the base class names for the generated
  metaobject, not their full definitions. **Fix**: guard the real includes
  with `#ifdef Q_MOC_RUN` / `#else`, giving moc bare forward declarations of
  `ScintillaEditBase`/`ScintillaEdit` instead.
- **`EXPORT_IMPORT_API` only makes sense across a DLL boundary.** Vendored
  `ScintillaEditBase`/`ScintillaEdit` use this macro because they live in
  `scintilla_qt.dll` and are imported by `_pyside6_scintilla`. Applying the
  same macro to `ScintillaEditBaseFixed`/`ScintillaEditFixed` -- which are
  defined and compiled entirely *within* `_pyside6_scintilla`, no DLL
  boundary crossed -- resolved to `__declspec(dllimport)` on Windows and
  collided with moc's own definition of `staticMetaObject`
  (`error C2491: definition of dllimport static data member not allowed`).
  **Fix**: don't apply `EXPORT_IMPORT_API` to classes fully contained within
  one compiled module.
- **`<modify-function>` only applies to the declaring class.** shiboken
  ignores a `<modify-function>` rule registered against a subclass that
  merely *inherits* the method rather than redeclaring it. `get_doc()`'s
  `<define-ownership owner="target"/>` rule (giving Python ownership of the
  returned `ScintillaDocument`) has to stay on `<object-type
  name="ScintillaEdit">`, where `get_doc()` is actually declared --
  registering it on `ScintillaEditFixed` (a subclass that just inherits it)
  silently no-ops, and the document's signals kept firing even after its
  Python wrapper was dropped.

See `tests/test_scintilla_edit_base.py::test_modified_signal_reaches_python_slot`
for the regression test.

## Type stubs (`_pyside6_scintilla.pyi`)

`src/pyside6_scintilla/_pyside6_scintilla.pyi` (plus the PEP 561
`src/pyside6_scintilla/py.typed` marker) gives Pylance/mypy/ruff/pylint full
signatures and autocomplete for `Scintilla.*`/`ScintillaEditBase`. Unlike the
generated wrapper `.cpp`/`.h` files above, it's checked into git -- it
directly improves the local dev/IDE experience (e.g. for
`examples/simple_scintilla_base_edit/main.py`) and is picked up by
`wheel.packages` for published wheels, without adding a PySide6-importing
step to the CMake/CI build.

Regenerate it with `make stubs` (`tools/generate_pyi.py`) after rebuilding
the extension (`make install` or `uv sync --reinstall-package
pyside6-scintilla`), and whenever `bindings.xml`/`bindings.h` change the
public API surface. The script:

1. Runs `shiboken6-genpyi` against the built `_pyside6_scintilla` extension,
   working around two issues with the generic `shiboken6-genpyi` entry point
   on a PySide6-based binding (both detailed in the script's docstring):
   - it re-imports the extension from a bare path, which fails with "DLL
     load failed" unless `pyside6_scintilla` (and therefore `PySide6.QtWidgets`,
     see issue 3 above) has been imported first;
   - its `find_imports()` references a `PySide6` global that only
     `PySide6.support.generate_pyi` sets up -- the generic entry point raises
     `NameError: name 'PySide6' is not defined` without it.
2. Aliases `Scintilla.Position`/`sptr_t`/`uptr_t` to `int`. These are
   primitive typedefs (`<primitive-type>` in `bindings.xml`), not nested
   types, but genpyi emits unresolvable `Scintilla.Position`-style forward
   references for them (`ScintillaEditBase.event_command`'s signature,
   `NotificationData`'s `# type:` comments). If a future Scintilla release
   adds another such primitive typedef, genpyi will warn
   (`RuntimeWarning: ... UNRECOGNIZED: 'Scintilla.<name>'`) -- add it to
   `PRIMITIVE_ALIASES` in `tools/generate_pyi.py`.
3. Stitches the `# ...` doc comments from `src/scintilla/include/Scintilla.iface`
   into `Scintilla.Message` enum members as docstrings (per-member hover
   docs). `val`/enum-style features generally don't have these comments, so
   this mainly benefits `Message`.

`_pyside6_scintilla.pyi` is excluded from `ruff` (`extend-exclude` in
`pyproject.toml`) since it's machine-generated and not ruff-format-compliant.

## Lifetime & ownership

Audit of the wrapped types for memory-safety/lifetime surprises beyond
standard PySide6/Qt semantics (Phase 3 Tracer 6).

### `ScintillaEditBase`/`ScintillaEdit` as `QWidget`s

No caveat: standard Qt/PySide6 ownership rules apply. A parentless widget is
kept alive by Python references, and is destroyed -- its signals included --
once the last reference is dropped. A widget with a Qt parent is kept alive
by that parent regardless of Python references, and its signals (`notify`,
`command`, `macroRecord`, etc.) keep firing as long as the widget exists.

### `NotificationData` (delivered via `notify`)

`ScintillaEditBase.notify(NotificationData *pscn)` hands Python a wrapper
around a transient struct that Scintilla reuses across notifications.

- Reading a field during the handler call is safe -- e.g. `text` (a
  `const char *` in C++) is eagerly copied into an independent Python `str`
  at access time, valid for as long as you like afterwards.
- The `NotificationData` object itself must **not** be retained past the
  handler call: once later notifications overwrite the shared struct, its
  fields no longer reflect the notification it was originally delivered for
  (see `tests/test_scintilla_notify.py`).

Where available, prefer `ScintillaEditBase`'s typed per-notification signals
(`modified`, `charAdded`, `marginClicked`, etc.) instead -- e.g. `modified`'s
`text` parameter is already a `QByteArray`, with no such caveat.

### `ScintillaDocument` from `get_doc()`

Already documented on `ScintillaEdit.get_doc()`'s docstring and its reference
page: the returned `ScintillaDocument` has no Qt parent, so it's kept alive
only by your Python reference to it -- drop it and its `modified`/
`save_point`/etc. signals stop firing, even though the underlying document
buffer stays alive via Scintilla's own refcounting as long as an editor uses
it (see `tests/test_scintilla_document.py`).

## Updating to a new Scintilla release

Per [`docs/specs/mission.md`](specs/mission.md), Scintilla updates are deliberate and tested, not automatic.
When vendoring a new release tarball:

1. Replace `src/scintilla/` with the new release's contents (remove the old
   tree first so deleted upstream files are actually removed, then extract the
   new tarball with `--strip-components=1`). `.gitattributes`
   (`src/scintilla/** -text`) keeps the diff clean.
2. Diff `src/scintilla/qt/ScintillaEditBase/ScintillaEditBase.pro` against the
   previous release's copy. Update the source list in
   `src/scintilla_qt/CMakeLists.txt` to match any added/removed/renamed
   `.cxx`/`.h` files.
3. Check `src/scintilla/include/ScintillaTypes.h`,
   `ScintillaMessages.h`, and `ScintillaStructures.h` for new enums, typedefs,
   or struct fields that `ScintillaEditBase`'s public API (or `ScintillaEdit`,
   for Phase 2) now references. Add the corresponding `<primitive-type>`,
   `<enum-type>`, or `<value-type>` entries to `bindings.xml` -- this is
   iterative: run the build, read the shiboken error naming the missing type,
   add the minimal entry, repeat.
4. Re-check whether the duplicate-`Message`-enum workaround (issue 1 above)
   still applies as written. If Scintilla restructures its headers, the
   `#define <Type> ... / #include / #undef` pattern in `bindings.h` may need
   to move, target a different type, or no longer be needed -- the same
   pattern applies if a *different* enum/struct gets a forward-declaration
   collision in a future release.
5. Bump `__version__` in `src/pyside6_scintilla/__init__.py` from
   `X.Y.Z.N` to the new Scintilla release's `X.Y.Z.0` (binding-only changes
   against the same Scintilla release increment `N`).
6. Rebuild with `uv sync --reinstall-package pyside6-scintilla` (`uv sync`
   alone doesn't always pick up CMake/typesystem changes), then
   `uv run pytest` and `uv run ruff check .`. See [`docs/build.md`](build.md).
