Python bindings: update & adding new libraries
Automated bindings: introduction
The bindings are generated automatically thanks to a sophisticated generator, which is based on srcML.
The generator in itself is located in the litgen module provided by srcmlcpp.
Installing the generator
Step 1: install srcML
Get and install srcML from its official site.
Step 2: install the generator and the required libraries
You will need to install the following python packages (see requirements-dev.txt):
srcmlcpp @ git+https://github.com/pthom/srcmlcpp black mypy pytest opencv-contrib-python
Quick information about the generator
litgen
(aka "Literate Generator") is the package provided by srcmlcpp that will generate the python bindings.
Its source code is available here.
It is heavily configurable by a wide range of options.
See for examples the specific options for imgui bindings generation.
Folders structure
In order to work on bindings, it is essential to understand the folders structure inside Dear ImGui Bundle. Please study the dedicated doc.
Study of a bound library generation
Let’s take the example of the library ImCoolBar.
Tip
|
The processing of adding a new library from scratch is documented in Adding a new library to the bindings. It uses ImCoolBar as an example |
Here is how the generation works for the library. The library principal files are located in external/ImCoolBar:
external/ImCoolBar/ # Root folder for ImCoolBar
├── ImCoolBar/ # ImCoolBar submodule
│ ├── CMakeLists.txt # ImCoolBar code
│ ├── ImCoolbar.cpp
│ ├── ImCoolbar.h
│ ├── LICENSE
│ └── README.md
└── bindings/ # Scripts for the bindings generations & bindings
├── generate_imcoolbar.py # This script reads ImCoolbar.h and generates:
| # - binding C++ code in ./pybind_imcoolbar.cpp
| # - stubs in
| # bindings/imgui_bundle/im_cool_bar_pyi
├── im_cool_bar.pyi -> ../../../bindings/imgui_bundle/im_cool_bar.pyi # this is a symlink!
└── pybind_imcoolbar.cpp
The actual stubs are located here:
imgui_bundle/bindings/imgui_bundle/
├── im_cool_bar.pyi # Location of the stubs
├── __init__.pyi # Main imgui_bundle stub file, which loads im_cool_bar.pyi
├── __init__.py # Main imgui_bundle python module which loads
| # the actual im_cool_bar module
├── ...
And the library is referenced in a global generation script:
imgui_bundle/external/bindings_generation/
├── autogenerate_all.py # This script will call generate_imcoolbar.py (among many others)
├── all_external_libraries.py # ImCoolBar is referenced here
├── ...
Update existing bindings
The process for updating bindings for a given library is straightforward:
-
Update the library submodule in external/LIBNAME/LIBNAME
-
Run the generation script in external/LIBNAME/generate_LIBNAME.py
-
Compile and test python bindings (carefully study that nothing was broken)
-
Commit and push
For example with ImCoolBar, in order to update the bindings for ImCoolBar, one needs to run:
python external/ImCoolBar/bindings/generate_imcoolbar.py
Tip
|
This video demonstrates from starts to finish the process of updating imgui and its bindings (17 minutes). |
Adding a new library to the bindings
This example is based on the addition of ImCoolBar, which was added in Oct 2023.
Step 1: Reference the new library
Tip
|
All the modifications done in step 1 can be seen in this commit. |
Step 1-a: Add needed folders, files and submodules inside external/
Add the library as a submodule in external/lib_name/lib_name
If the library can be included without adaptations for inclusion inside ImGui Bundle, you can add it directly as a submodule.
mkdir external/ImCoolBar
git submodule add https://github.com/aiekick/ImCoolBar.git external/ImCoolBar/ImCoolBar
However, if it requires adaptations, you need to create a fork (it was the case for ImCoolBar): So, the following actions were done separately:
-
ImCoolBar was cloned into github.com/pthom/ImCoolBar.git
-
a branch imgui_bundle was created and pushed to github. It will contain the adaptations and bug corrections for imgui_bundle.
Then, we add this fork as a submodule.
git submodule add https://github.com/pthom/ImCoolBar.git external/ImCoolBar/ImCoolBar
cd external/ImCoolBar/ImCoolBar
git checkout imgui_bundle
cd -
Create the folder external/lib_name/bindings/
Copy the folder external/bindings_generation/bindings_generator_template
into external/lib_name/bindings/
cp -r external/bindings_generation/bindings_generator_template external/ImCoolBar/bindings
Rename files in external/lib_name/bindings
After having copied the template files, we need to rename them. In the example of ImCoolbar, we will rename them as follows:
mv external/ImCoolBar/bindings/generate_LIBNAME.py external/ImCoolBar/bindings/generate_imcoolbar.py
mv external/ImCoolBar/bindings/pybind_LIBNAME.cpp external/ImCoolBar/bindings/pybind_imcoolbar.cpp
# im_cool_bar will be the final name of the python module: imgui_bundle.im_cool_bar
mv external/ImCoolBar/bindings/LIBNAME.pyi external/ImCoolBar/bindings/im_cool_bar.pyi
Move external/ImCoolBar/bindings/im_cool_bar.pyi to bindings/imgui_bundle/
The stub file (*.pyi) must be inside bindings/imgui_bundle. In order to facilitate development, we will create a symlink to it inside external/ImCoolBar/bindings/
mv external/ImCoolBar/bindings/im_cool_bar.pyi bindings/imgui_bundle/im_cool_bar.pyi
cd external/ImCoolBar/bindings/
ln -s ../../../bindings/imgui_bundle/im_cool_bar.pyi .
cd -
Final folder structure
We end up with the following structure:
external/ImCoolBar/
├── ImCoolBar/ # Note that the submodule is inside
│ ├── CMakeLists.txt # external/ImCoolBar/ImCoolBar/ !!!
│ ├── ImCoolbar.cpp
│ ├── ImCoolbar.h
│ ├── LICENSE
│ └── README.md
└── bindings/
├── im_cool_bar.pyi # We will edit and rename those files later
├── generate_imcoolbar.py -> symlink to ../../../bindings/imgui_bundle/im_cool_bar.pyi
└── pybind_imcoolbar.cpp
Step 1-b: Update python generator manager
Update external/bindings_generation/all_external_libraries.py
Add a function that returns info about this new library:
def lib_imcoolbar() -> ExternalLibrary:
return ExternalLibrary(
name="ImCoolBar",
official_git_url="https://github.com/aiekick/ImCoolBar.git",
official_branch="master",
fork_git_url="https://github.com/pthom/ImCoolBar.git",
fork_branch="imgui_bundle"
)
ALL_LIBS = [
lib_imgui(), # must be first as it declare bindings used by the next ones
# ...
lib_imcoolbar(), # Add the lib here
# ...
Step 1-c: Update the C++ sources to include the new lib binding generation
In external/CMakeLists.txt: Add a cmake directive to compile the new library.
# If the library is "simple" to compile you can use `add_simple_external_library_with_sources`
add_simple_external_library_with_sources(imcoolbar ImCoolBar)
In external/bindings_generation/cpp/all_pybind_files.cmake:
add external/ImCoolBar/bindings/pybind_imcoolbar.cpp
Note
|
the script external/bindings_generation/autogenerate_all.py will also regenerate this file from scratch. |
In external/bindings_generation/cpp/pybind_imgui_bundle.cpp:
Add the bindings
// ... Near the start of the file, add a new function declaration
void py_init_module_imgui_command_palette(py::module& m);
void py_init_module_implot_internal(py::module& m);
void py_init_module_imcoolbar(py::module& m); // added this line
// ...
void py_init_module_imgui_bundle(py::module& m)
{
// ...
// At the end of py_init_module_imgui_bundle, register your new python module
auto module_imcooolbar = m.def_submodule("im_cool_bar"); // the python module will be known as imgui_bundle.im_cool_bar
py_init_module_imcoolbar(module_imcooolbar);
Now, run cmake.
Step 1-d: Edit and adapt the generation scripts
Edit the 3 files inside external/ImCoolBar/bindings
and replace occurrences of LIBNAME with appropriate values.
Step 1-e: Edit and adapt the imgui_bundle init scripts
In bindings/imgui_bundle/init.py, this line was added:
from imgui_bundle._imgui_bundle import im_cool_bar as im_cool_bar
In bindings/imgui_bundle/init.pyi, this line was added:
from . import im_cool_bar as im_cool_bar
Step 2: fine tune the generation options and write a demo
Tip
|
All the modifications done in step 2 can be seen in this commit. |
Step 2-a: Edit and run external/ImCoolBar/bindings/generate_imcoolbar.py
:
Edit and re-run it until the generated code fits the expected needs.
In the case of ImCoolBar, two simple changes were made:
def main():
# ...
# ...
# Configure options
options = litgen.LitgenOptions()
options.namespace_root__regex = "ImGui"
options.srcmlcpp_options.functions_api_prefixes = "IMGUI_API"
Each time you run the code generation, look at external/ImCoolBar/bindings/im_cool_bar.pyi
and external/ImCoolBar/bindings/pybind_imcoolbar.cpp
to see if they seem OK. Also run a compilation.
Step 2-b: Fix syntax issues in external/ImCoolBar/bindings/im_cool_bar.pyi
:
You can add some code before the autogenerated code to fix the syntax issues. For example, this was added:
import enum
from imgui_bundle.imgui import ImVec2, WindowFlags, WindowFlags_
ImCoolBarFlags = int
ImGuiWindowFlags = WindowFlags
ImGuiWindowFlags_None = WindowFlags_.none
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! AUTOGENERATED CODE !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# <litgen_stub> // Autogenerated code below! Do not edit!
#################### <generated_from:ImCoolbar.h> ####################
Step 2-c: Write a nice looking demo
It should demo the library, and act as a tutorial, in python and C++.