Metadata-Version: 2.4
Name: skbuild_conan
Version: 1.6.0
Summary: An extension for scikit-build to add C++-dependencies as easily as Python dependencies via conan. 
Author-email: "TU Braunschweig, IBR, Algorithms Group (Dominik Krupke)" <krupke@ibr.cs.tu-bs.de>
License: MIT License
        
        Copyright (c) 2023 Dominik Krupke
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/d-krupke/skbuild-conan
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Development Status :: 3 - Alpha
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: conan>=2.0.0
Requires-Dist: scikit-build>=0.17.3
Requires-Dist: setuptools
Requires-Dist: cmake>=3.23
Requires-Dist: ninja
Requires-Dist: colorama>=0.4.0
Dynamic: license-file

# skbuild-conan: A conan extension for scikit-build

![PyPI](https://img.shields.io/pypi/v/skbuild-conan)
![License](https://img.shields.io/github/license/d-krupke/skbuild-conan)

[PyBind11](https://github.com/pybind/pybind11) and
[scitkit-build](https://github.com/scikit-build/scikit-build)
enable us to easily write native C++-modules for Python.
However, you get problems if your C++-code has dependencies.
This extension tries to make defining C++-dependencies as easy
as defining the Python-dependencies. This way you can easily
add any C++-library that has a conan recipe to your Python-project.

This project originates from [our](https://www.ibr.cs.tu-bs.de/alg/) need to use complex
C++-libraries
in Python projects and missing any nice option to include C++-dependencies.
For a few projects, we wrote individual code to fetch the dependencies
or just added instructions on how to install them (which of course
can scare pure Python users).

**This project is currently just a nice interface to hacks we accumulated. We try to make it
as universal and robust as possible as we rely on this for multiple projects, but we are
working fast-paced and will quickly abandon this tool once there is something better.**

We (TU Braunschweig, Algorithms Group) are not affiliated with scikit-build or conan.

## Installation

For `skbuild-conan` to work, you need to have a full Python and C++ development environment.
This means that you need to have a compiler installed and the Python development files.
The Python development files are usually called `python3-dev` or `python3-devel` and
the compiler is usually called `gcc` or `clang`. On most Linux distributions, these
are installed by default.

You can simply add `"skbuild_conan",`
to `requires=[...]` in `pyproject.toml`.
E.g. The `pyproject.toml` could look like this

```toml
[build-system]
requires = [
    "conan>=2.0.0",
    "setuptools",
    "scikit-build>=0.17.3",
    "skbuild-conan",
    "cmake>=3.23",
    "ninja",
]
build-backend = "setuptools.build_meta"
```

If you want to use for example `setup.py build`, you need to
install `skbuild_conan` to your environment. You can do so
by ` pip install skbuild_conan`.

## Transparency and Logging

skbuild-conan provides comprehensive logging to help you understand what's happening during the build process.

### Using pip/setup.py verbosity flags (easiest)

skbuild-conan automatically respects standard `--verbose` and `--quiet` flags:

```bash
# Standard output (default)
pip install .

# Verbose output (shows all operations and phases)
pip install --verbose .

# Very verbose output (includes full conan output)
pip install -vv .

# Quiet output (errors only)
pip install --quiet .
```

The same works with `setup.py`:

```bash
python setup.py install --verbose
python setup.py build -vv
```

### Using environment variable (for more control)

You can also use the `SKBUILD_CONAN_LOG_LEVEL` environment variable:

```bash
# Minimal output (errors only)
export SKBUILD_CONAN_LOG_LEVEL=quiet
pip install .

# Standard output (recommended)
export SKBUILD_CONAN_LOG_LEVEL=normal  # This is the default
pip install .

# Detailed output (shows all operations)
export SKBUILD_CONAN_LOG_LEVEL=verbose
pip install .

# Debug output (includes full conan output)
export SKBUILD_CONAN_LOG_LEVEL=debug
pip install .
```

**Priority**: Command-line flags take precedence over the environment variable. If you set both, the `--verbose`/`--quiet` flags will be used.

After installation, a dependency report is generated at `.conan/<build_type_lowercase>/dependency-report.txt` (e.g. `.conan/release/dependency-report.txt`) showing:

- What dependencies were requested
- What versions were resolved
- Build configuration used
- Local recipes installed

This transparency helps with:

- Understanding exactly what's being built
- Debugging version conflicts
- Security auditing
- Reproducing builds

## Usage

The usage is very similar to scitkit-build (and setuptools).
We just added a few additional arguments to `setup()`.

See [how to use scikit-build](https://scikit-build.readthedocs.io/en/latest/usage.html#example-of-setup-py-cmakelists-txt-and-pyproject-toml)
first, as this is just a small extension to it.

The added options are

- `conanfile`: Path to the folder with the conanfile.[py|txt]. By default the root
  is assumed. The conanfile can be used to define the dependencies.
  Alternatively, you can also use `conan_requirements` to define
  the conan dependencies without a conanfile. This option is
  exclusive. If you define `conan_requirements`, this option is
  ignored.
- `conan_recipes`: List of paths to further conan recipes. The conan package index
  is far from perfect, so often you need to build your own recipes. You don't
  always want to upload those, so this argument gives you the option to integrate
  local recipes. Just the path to the folder containing the `conanfile.py`.
  Each recipe is exported into the cache on every build via
  `conan create --build=missing`, using the same `conan_profile_settings` as the
  rest of your dependencies. Conan skips the compile when a matching binary is
  already cached, so a repeated build only pays for the export.
- `conan_requirements`: Instead of providing a conanfile, you can simply state
  the dependencies here. E.g. `["fmt/[>=10.0.0]"]` to add fmt in version >=10.0.0.
- `conan_profile_settings`: Overwrite conan profile settings. You should use this to
  pin the C++ standard, e.g. `{"compiler.cppstd": "20"}` (see
  [Setting the C++ standard](#setting-the-c-standard)). Also necessary for
  ABI-problems, etc.
- `wrapped_setup`: The setup-method that is going to be wrapped. This would allow
  you to extend already extended setup functions. By default, it is the `setup`
  of `skbuild`, which extends the `setup` of `setuptools`.
- `conan_output_folder`: The folder where conan will write the generated files.
  No real reason to change it unless the default creates conflicts with some other
  tool.
- `cmake_args`: This is actually an argument of `skbuild` but we will extend it.
  It hands cmake custom arguments. We use it to tell cmake about the conan modules.
- `conan_profile`: The name of the conan profile to use. By default, it is
  `skbuild_conan_py`. This profile is created automatically and should work for
  most cases. If you need to change it, you can do so by editing
  `~/.conan2/profiles/skbuild_conan_py`.
- `conan_env`: Environment variables that are used for the conan calls. By
  default it will override `CC` and `CXX` with empty strings. This is necessary
  to work around problems with anaconda, but it should not cause any problems
  with other setups. You could define `CONAN_HOME` to `./conan/cache` to use
  a local cache and not install anything to the user space. A relative
  `CONAN_HOME` is resolved against the current working directory, because conan
  itself only accepts absolute paths. Note that a project-local cache is not
  shared with your other projects, so everything is downloaded and built again
  for this one.

An example usage could be as follows

```python
from skbuild_conan import setup
from setuptools import find_packages

setup(  # https://scikit-build.readthedocs.io/en/latest/usage.html#setup-options
    name="simple_skbuild_conan_example",
    version="0.1.1",
    packages=find_packages("src"),  # Include all packages in `./src`.
    package_dir={"": "src"},  # The root for our python package is in `./src`.
    python_requires=">=3.7",  # lowest python version supported.
    install_requires=[],  # Python Dependencies
    conan_requirements=["fmt/[>=10.0.0]"],  # C++ Dependencies
    conan_profile_settings={"compiler.cppstd": "17"},  # C++ standard
    cmake_minimum_required_version="3.23",
)
```

See [./examples/simple_skbuild_conan_example](./examples/simple_skbuild_conan_example)
for a full example.

## Setting the C++ standard

**Always pin the C++ standard your bindings are built against.**

```python
setup(
    ...,
    conan_profile_settings={"compiler.cppstd": "20"},
)
```

If you do not, conan uses the value it auto-detected via `conan profile detect`.
That value is the standard your compiler enables _without any flags_, which is
not the latest one it supports — it can be as low as `14`, and it differs between
platforms and compiler versions. This is a common cause of builds that work on
your machine but fail elsewhere: CGAL, for example, checks for C++17 and aborts
the build if the profile says `14`. Windows/MSVC and older GCC/Clang are the
usual offenders.

Pinning the standard also makes the build more reproducible: the standard is part
of the conan package id, so your dependencies are built and cached against the
standard you asked for rather than against whatever the machine happened to
detect.

Note that conan does not treat the package id as an exact requirement. If no
binary matches, its binary compatibility plugin looks for a _compatible_ one and
will accept, for example, a `gnu20` build for a `compiler.cppstd=20` request —
giving you the language extensions you just turned off, without an error. Pinning
the setting everywhere (which skbuild-conan now does for local recipes too) is
what keeps that fallback from being reached in the first place.

A few details worth knowing:

- **`"20"` vs `"gnu20"`**: the `gnu` prefix enables compiler extensions
  (`-std=gnu++20` instead of `-std=c++20`). Prefer the plain value unless you
  actually need the extensions. Valid values are e.g. `"11"`, `"14"`, `"17"`,
  `"20"`, `"23"` and their `gnu` variants.
- **Do not set `CMAKE_CXX_STANDARD` in your `CMakeLists.txt`.** The conan
  toolchain already sets it (plus `CMAKE_CXX_STANDARD_REQUIRED`) from
  `compiler.cppstd`. Overriding it after `project()` compiles your bindings
  against a different standard than your dependencies, which can lead to subtle
  ODR/ABI breakage. Conan will print a warning if you do:

  ```
  Warning: Standard CMAKE_CXX_STANDARD value defined in conan_toolchain.cmake
  to 20 has been modified to 17 by .../CMakeLists.txt
  ```

  If a target needs a _minimum_ standard, declare it on the target instead —
  this composes with a higher value from the toolchain rather than fighting it:

  ```cmake
  target_compile_features(_bindings PRIVATE cxx_std_17)
  ```

- The setting applies to everything skbuild-conan builds, including the local
  recipes you pass via `conan_recipes`.
- **Changing the standard later rebuilds your dependencies.** The standard is
  part of the package id, so conan has to build binaries it does not have yet.
  That is expected, and a one-off cost per standard you use.

All examples in [./examples](./examples) follow this pattern.

## Examples

If you do not have any C++-dependencies, you can just use [scikit-build](https://github.com/scikit-build/scikit-build) which also provides a [set of examples](https://github.com/scikit-build/scikit-build-sample-projects/tree/master/projects).

### Simple Example with fmt

The example in [./examples/simple_skbuild_conan_example](./examples/simple_skbuild_conan_example) provides a minimal example of how to use fmt in Python using PyBind11 and skbuild_conan.
fmt is a nice library for formatting strings and is used by many other libraries.
Python, of course, comes with extensive inbuilt string formatting, thus, this example is not very useful.
However, it is a good starting point to understand how to use skbuild_conan with a simple external library.

### Complex Example with CGAL: Using CGAL in Python

Sometimes, your dependencies are significantly more complex.
For example, you may want to do some geometry processing and use CGAL.
CGAL is a very complex library with many dependencies, but also the most powerful library for geometric operations and often the only choice for many problems.
CGAL has a conan recipe, but it took a while until it received updates for conan2, such that we wrote our own recipe.
In the meantime, the official recipe was updated, but for the sake of the example, we will use our own recipe.
In case you are faced with the problem of an outdated conan recipe (or none at all), you can use the same trick.

See [./examples/cgal_skbuild_conan_example](./examples/cgal_skbuild_conan_example) for an example of how to use CGAL via a custom conan recipe in Python using PyBind11 and skbuild_conan.

Note that there is also the [cgalpy](https://bitbucket.org/taucgl/cgal-python-bindings/) project by my friends at TAU (which I visited for a few months in 2022/2023), which is a nearly complete and efficient wrapper of CGAL.
It may need some more documentation, but Efi put a lot of thought into efficiency and configurability.

## Common problems

> Feel free to copy these comments. Attribution is appreciated but not necessary.

### ABI problems: Undefined symbole `...__cxx1112basic_stringIcSt11char_...`

This problem should be automatically fixed. Please open an issue if you still encounter it.

See [https://docs.conan.io/1/howtos/manage_gcc_abi.html](https://docs.conan.io/1/howtos/manage_gcc_abi.html) for more details.

### C++ standard problems: `requires C++17` / `no member named 'optional' in namespace 'std'`

If a dependency refuses to build with a message such as

```
ERROR: cgal/6.0.1: Invalid: cgal requires C++17. Your compiler is set to C++14.
```

or your own code fails to compile with errors about C++17/C++20 features not
existing, your conan profile has a lower `compiler.cppstd` than you expect.
The auto-detected profile uses the standard that is active without any compiler
flags, not the newest one the compiler supports.

Pin it explicitly in your `setup.py`:

```python
setup(
    ...,
    conan_profile_settings={"compiler.cppstd": "17"},  # or "20", "23", ...
)
```

See [Setting the C++ standard](#setting-the-c-standard) for details.

### glibcxx problems:

If you get an error such as

```
ImportError: /home/krupke/anaconda3/envs/mo310/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/krupke/anaconda3/envs/mo310/lib/python3.10/site-packages/samplns/cds/_cds_bindings.cpython-310-x86_64-linux-gnu.so)
```

you are probably using conda (good!) but need to update glibcxx. Install the latest version by

```sh
conda install -c conda-forge libstdcxx-ng
```

In some cases, this still is not enough, especially if you are using a very up to date rolling-release distribution, such as Arch Linux, or if you installed `libstdcxx-ng` some time ago.
This could lead to the system having a slightly newer version of glibcxx than conda.
First try to upgrade `libstdcxx-ng` with

```sh
conda upgrade -c conda-forge --all
```

If this does not help, you can try to install g++ (caveat: Linux only, Mac OS needs clang) into your conda environment and use it to compile the package.

```sh
conda install -c conda-forge gxx_linux-64  # This should enforce a modern g++ version.
conda install -c conda-forge cxx-compiler  # This should make sure that the compiler is used.
```

Note that just the second command first may install an outdated g++ version (at least I observed that it installed gcc11 instead of gcc13, messing up my whole environment as this is too old).
When compiling from source, you probably should delete the `_skbuild`-folder and do a proper uninstall of the previous installation first.

### conan problems

If you encounter problems with conan, you can try to delete the conan profile and let it be recreated.

```sh
rm ~/.conan2/profiles/skbuild_conan_py
```

Maybe you can also just take a look at the file and see if conan detected your compiler correctly.
A proper profile on Linux should for example look like this (different for other systems):

```
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu17
compiler.libcxx=libstdc++11
compiler.version=13
os=Linux
```

If the problem persists, you can try to delete the conan cache.

```sh
rm -rf ~/.conan2
```

### `fatal error: Python.h: No such file or directory`

This is a rare problem with your Python installation. Make sure that you have the development files installed.

```
sudo apt-get install python3-dev  # Ubuntu
sudo yum install python3-devel  # CentOS, RHEL
sudo dnf install python3-devel  # Fedora
sudo zypper in python3-devel  # OpenSUSE
sudo apk add python3-dev  # Alpine
sudo apk add python3-dev  # Cygwin
```

In case you have multiple Python versions installed, you may need to specify the Python version.

If you still encounter problems, please open an issue.

### MetadataPathFinder.invalidate_caches() missing 1 required positional argument: 'cls'

A very odd bug that we encountered with a distribution of Python 3.12 was the following:

```
...
TypeError: MetadataPathFinder.invalidate_caches() missing 1 required positional argument: 'cls'

ERROR: MetadataPathFinder.invalidate_caches() missing 1 required positional argument: 'cls'
```

The reason for this was surprisingly **a bug in the Python distribution of importlib itself**.
Someone forgot to add the `@classmethod` to the `MetadataPathFinder`.
Updating the Python distribution, in this case in conda via `conda update python` solved the problem.

## Contributing

We welcome contributions from the community! Whether you're fixing a bug, adding a feature, or improving documentation, your help is appreciated.

**Quick Start:**

- 📖 Read [CONTRIBUTING.md](CONTRIBUTING.md) for detailed contribution guidelines
- 🔧 See [DEVELOPMENT.md](DEVELOPMENT.md) for technical development documentation
- 🐛 Report bugs via [GitHub Issues](https://github.com/d-krupke/skbuild-conan/issues)
- 💡 Suggest features via [GitHub Discussions](https://github.com/d-krupke/skbuild-conan/discussions)

**Good First Issues:** Look for issues labeled [`good first issue`](https://github.com/d-krupke/skbuild-conan/labels/good%20first%20issue) to get started.

Please note that response times may vary as we prioritize based on available time and resources.

## Changelog

- _1.6.0_ C++ standard documentation plus a set of fixes around conan settings and
  local recipes.

  - Documented [setting the C++ standard](#setting-the-c-standard) via
    `conan_profile_settings={"compiler.cppstd": ...}`, with all examples updated to
    pin it. (#16)
  - The examples no longer set `CMAKE_CXX_STANDARD` in their `CMakeLists.txt`, as
    that overrides the value the conan toolchain derives from `compiler.cppstd`.
    They declare a per-target minimum with `target_compile_features` instead.
  - **Local recipes are now built with your `conan_profile_settings`.** Previously
    only `build_type` was forwarded to `conan create`, so a recipe was built with
    the profile defaults. Conan's binary compatibility fallback then resolved that
    mismatched binary for the subsequent `install` — asking for `compiler.cppstd=20`
    could silently get you a `gnu20` build, i.e. language extensions you had turned
    off.
  - **Local recipes are no longer skipped when their `name/version` is already in
    the cache.** That check ignored where the cached package came from, so a local
    recipe was silently replaced by an unrelated remote package of the same name —
    the bundled CGAL recipe is `cgal/6.0.1`, exactly the reference ConanCenter
    serves. It also ignored the settings of the cached binary, so changing
    `compiler.cppstd` never rebuilt it. Every recipe is now handed to
    `conan create --build=missing` on each run; conan skips the build itself when a
    matching binary exists, so a warm rebuild costs an export rather than a compile.
  - **`conan_env` is now actually applied to conan.** `ConanAPI()` was constructed
    before the environment override was installed, and it resolves `CONAN_HOME` and
    loads the configuration in its constructor — so the documented
    `conan_env={"CONAN_HOME": "./conan/cache"}` silently had no effect and the
    user-wide cache was used instead. A relative `CONAN_HOME` is now resolved
    against the current directory, since conan itself rejects one.
  - `build_type` passed via `conan_profile_settings` is now dropped with a warning
    instead of emitting a conan command line that sets it twice with conflicting
    values. Use the `--build-type` argument.
  - The examples now include `<fmt/format.h>` instead of the `<fmt/core.h>` that no
    longer provides `fmt::format` since fmt 12.
  - Developer tooling was modernized: `ruff` (lint + format, replacing the separate
    `pyupgrade` hook) and `ty` (type checking) run in pre-commit, with the rule set
    pinned in `pyproject.toml` so a local `ruff check` matches the hook. The
    `cmake-format` hook had its `additional_dependencies`/`types`/`files` keys at the
    repo level, where pre-commit ignores them; they now sit on the hook. Type hints
    across the package were updated to PEP 585/604 syntax (the `>=3.9` floor is kept
    via `from __future__ import annotations`).

- _1.5.0_ Support for conan's `cmake_layout` in conanfiles. Previously, using `[layout] cmake_layout` in a `conanfile.txt` (or `cmake_layout(self)` in `conanfile.py`) caused a "conan_toolchain.cmake not found" error because the generators are placed under `build/{BuildType}/generators/` instead of directly in the output folder. (#7)
- _1.4.0_ Major transparency and usability improvements:
  - **Structured logging** with configurable verbosity levels (quiet/normal/verbose/debug)
  - **Auto-detect verbosity** from pip/setup.py --verbose/-v flags
  - **Dependency resolution reports** showing what was installed and why
  - **Context-aware error messages** with specific remediation suggestions
  - **Input validation** to catch configuration errors early
  - **Automatic retry** for network operations with exponential backoff
  - **Version compatibility checks** with warnings for known issues
  - **Cross-platform color support** via colorama
- _1.3.1_ Replace `pkg_resources` with `importlib.metadata` to fix compatibility with setuptools v76+.
- _1.3.0_ The Debug/Release will propagate to the conan profile. (thanks to @xandox)
- _1.2.0_ Workaround for Windows and MSVC found by Ramin Kosfeld (TU Braunschweig).
- _1.1.1_ Fixing problem if the conan default profile has been renamed via environment variable.
- _1.1.0_ conan is now called directly. This is kind of hacky, but circumvents problems with conan not being in the path if only installed for build.
- _1.0.0_ Custom conan profile and workaround for anaconda problem.
- _0.2.0_ Improved logging.
- _0.1.4_ Fixing problem with paths that contain spaces. Switching back to manual versioning.
- _0.1.3_ Fixing bug if no settings are given.
- _0.1.2_ Moved workaround into setup to make it more explicit.
- _0.1.1_ First tested and apparently working version.
