* First draft a subinterpreter embedding API
* Move subinterpreter tests to their own file
* Migrate subinterpreter tests to use the new embedded class.
* Add a test for moving subinterpreters across threads for destruction
And find a better way to make that work.
* Code organization
* Add a test which shows demostrates how gil_scoped interacts with sub-interpreters
* Add documentation for embeded sub-interpreters
* Some additional docs work
* Add some convenience accessors
* Add some docs cross references
* Sync some things that were split out into #5665
* Update subinterpreter docs example to not use the CPython api
* Fix pip test
* style: pre-commit fixes
* Fix MSVC warnings
I am surprised other compilers allowed this code with a deleted move ctor.
* Add some sub-headings to the docs
* Oops, make_unique is C++14 so remove it from the tests.
* I think this fixes the EndInterpreter issues on all versions.
It just has to be ifdef'd because it is slightly broken on 3.12, working well on 3.13, and kind of crashy on 3.14beta. These two verion ifdefs solve all the issues.
* Add a note about exceptions.
They contain Python object references and acquire the GIL, that means they are a danger with subinterpreters!
* style: pre-commit fixes
* Add try/catch to docs examples to match the tips
* Python 3.12 is very picky about this first PyThreadState
Try special casing the destruction on the same thread.
* style: pre-commit fixes
* Missed a rename in a ifdef block
* I think this test is causing problems in 3.12, so try ifdefing it to see if the problems go away.
* style: pre-commit fixes
* Document the 3.12 constraints with a warning
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add cpptest to the clang-tidy job
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* noexcept move operations
* Update include/pybind11/subinterpreter.h
std::memset
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* `container: silkeh/clang:18-bookworm` in .github/workflows/format.yml
* clang-tidy auto-fix (trivial, in test only)
* Disable `performance-enum-size` (noisy, low value)
* Temporarily turn off 3 diagnostics (to be tackled one-by-one).
* Add explicit `switch` `default` to resolve clang-tidy `bugprone-switch-missing-default-case`
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
tests/test_numpy_dtypes.cpp:212:5: warning: switching on non-enum value without default case may not cover all cases [bugprone-switch-missing-default-case]
* Add clang-17 and clang-18 testing.
* Add `NOLINTNEXTLINE(clang-analyzer-optin.core.EnumCastOutOfRange)` in test_tagbased_polymorphic.cpp
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
tests/test_tagbased_polymorphic.cpp:77:40: warning: The value '150' provided to the cast expression is not in the valid range of values for 'Kind' [clang-analyzer-optin.core.EnumCastOutOfRange]
* Fix inconsistent pybind11/eigen/tensor.h behavior:
This existing comment in pybind11/eigen/tensor.h
```
// move, take_ownership don't make any sense for a ref/map:
```
is at odds with the `delete src;` three lines up.
In real-world client code `take_ownership` will not exist (unless the client code is untested and unused). I.e. the `delete` is essentially only useful to avoid leaks in the pybind11 unit tests.
While upgrading to clang-tidy 18, the warning below appeared. Apparently it is produced during LTO, and it appears difficult to suppress. Regardless, the best way to resolve this is to remove the `delete` and to simply make the test objects `static` in the unit test code.
________
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________
```
lto-wrapper: warning: using serial compilation of 3 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
In function ‘cast_impl’,
inlined from ‘cast’ at /mounted_pybind11/include/pybind11/eigen/tensor.h:414:25,
inlined from ‘operator()’ at /mounted_pybind11/include/pybind11/eigen/../pybind11.h:296:40,
inlined from ‘_FUN’ at /mounted_pybind11/include/pybind11/eigen/../pybind11.h:267:21:
/mounted_pybind11/include/pybind11/eigen/tensor.h:475:17: warning: ‘operator delete’ called on unallocated object ‘<anonymous>’ [-Wfree-nonheap-object]
475 | delete src;
| ^
/mounted_pybind11/include/pybind11/eigen/../pybind11.h: In function ‘_FUN’:
/mounted_pybind11/include/pybind11/eigen/../pybind11.h:297:75: note: declared here
297 | std::move(args_converter).template call<Return, Guard>(cap->f),
| ^
```
* Disable `bugprone-chained-comparison`: this clang-tidy check is incompatible with the Catch2 `REQUIRE` macro (26 warnings like the one below).
________
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________
```
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:9: warning: chained comparison 'v0 <= v1 == v2' may generate unintended results, use parentheses to specify order of evaluation or a logical operator to separate comparison expressions [bugprone-chained-comparison]
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:24: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:47: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:9: note: operand 'v0' is here
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:24: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:47: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:17: note: operand 'v1' is here
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:90: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:70: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~
/mounted_pybind11/tests/test_embed/test_interpreter.cpp:127:24: note: operand 'v2' is here
127 | REQUIRE(ret == 42);
| ^
/build/tests/catch/catch.hpp:17670:90: note: expanded from macro 'REQUIRE'
17670 | #define REQUIRE( ... ) INTERNAL_CATCH_TEST( "REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
| ^~~~~~~~~~~
/build/tests/catch/catch.hpp:2710:70: note: expanded from macro 'INTERNAL_CATCH_TEST'
2710 | catchAssertionHandler.handleExpr( Catch::Decomposer() <= __VA_ARGS__ ); \
| ^~~~~~~~~~~
```
* Add 8 `// NOLINT(bugprone-empty-catch)`
* Resolve clang-tidy `bugprone-multi-level-implicit-pointer-conversion` warnings.
________
Debian clang version 18.1.8 (++20240718080534+3b5b5c1ec4a3-1~exp1~20240718200641.143)
Target: x86_64-pc-linux-gnu
________
```
pybind11/detail/internals.h:556:53: warning: multilevel pointer conversion from 'internals **' to 'const void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/detail/type_caster_base.h:431:20: warning: multilevel pointer conversion from 'void **' to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:904:81: warning: multilevel pointer conversion from '_object *const *' to 'const void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:1989:39: warning: multilevel pointer conversion from 'typename vectorize_arg<const double *>::type *' (aka 'const double **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/numpy.h:1989:39: warning: multilevel pointer conversion from 'typename vectorize_arg<const VectorizeTestClass *>::type *' (aka 'const VectorizeTestClass **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/stl/filesystem.h:75:44: warning: multilevel pointer conversion from 'PyObject **' (aka '_object **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
pybind11/stl/filesystem.h:83:42: warning: multilevel pointer conversion from 'PyObject **' (aka '_object **') to 'void *', please use explicit cast [bugprone-multi-level-implicit-pointer-conversion]
```
* Introduce `PYBIND11_REINTERPRET_CAST_VOID_PTR_IF_NOT_PYPY` to resolve PyPy build errors:
```
In file included from /Users/runner/work/pybind11/pybind11/tests/test_stl.cpp:18:
/Users/runner/work/pybind11/pybind11/include/pybind11/stl/filesystem.h:75:17: error: no matching function for call to 'PyPyUnicode_FSConverter'
if (PyUnicode_FSConverter(buf, reinterpret_cast<void *>(&native)) != 0) {
^~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:969:31: note: expanded from macro 'PyUnicode_FSConverter'
^~~~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:970:17: note: candidate function not viable: cannot convert argument of incomplete type 'void *' to 'struct _object **' for 2nd argument
PyAPI_FUNC(int) PyUnicode_FSConverter(struct _object *arg0, struct _object **arg1);
^
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:969:31: note: expanded from macro 'PyUnicode_FSConverter'
^
In file included from /Users/runner/work/pybind11/pybind11/tests/test_stl.cpp:18:
/Users/runner/work/pybind11/pybind11/include/pybind11/stl/filesystem.h:83:17: error: no matching function for call to 'PyPyUnicode_FSDecoder'
if (PyUnicode_FSDecoder(buf, reinterpret_cast<void *>(&native)) != 0) {
^~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:971:29: note: expanded from macro 'PyUnicode_FSDecoder'
^~~~~~~~~~~~~~~~~~~~~
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:972:17: note: candidate function not viable: cannot convert argument of incomplete type 'void *' to 'struct _object **' for 2nd argument
PyAPI_FUNC(int) PyUnicode_FSDecoder(struct _object *arg0, struct _object **arg1);
^
/Users/runner/hostedtoolcache/PyPy/3.10.14/x64/include/pypy3.10/pypy_decl.h:971:29: note: expanded from macro 'PyUnicode_FSDecoder'
^
```
* clang-tidy auto-fix
* Fix silly oversight.
* chore: update clang-tidy to 15
* Add git
* Add NOLINTNEXTLINE for assignment in if
* Update CONTRIBUTING.md
* Add NOLINTNEXTLINE where needed
* Add one more NOLINTNEXTLINE
* stl_bind: make more readable
* Another missing NOLINTNEXTLINE
* Match style elsewhere
* Apply reviewer suggestion. Mark false positive
* Revert "Systematically add `-DCMAKE_VERBOSE_MAKEFILE=ON` to obtain full command lines related to `-Wodr` (#4398)"
This reverts commit ff42f5254a.
* Set `env: VERBOSE: 1` as suggested by @henryiii
* Set `env: VERBOSE: 1` also in all other .yml files using cmake
* Updating and slightly enhancing instructions for running clang-tidy.
* Making documented commands identical to workflow commands. Adding comment in workflow file pointing to documentation.
* feat: setup.py redesign and helpers
* refactor: simpler design with two outputs
* refactor: helper file update and Windows support
* fix: review points from @YannickJadoul
* refactor: fixes to naming and more docs
* feat: more customization points
* feat: add entry point pybind11-config
* refactor: Try Extension-focused method
* refactor: rename alt/inplace to global
* fix: allow usage with git modules, better docs
* feat: global as an extra (@YannickJadoul's suggestion)
* feat: single version location
* fix: remove the requirement that setuptools must be imported first
* fix: some review points from @wjacob
* fix: use .in, add procedure to docs
* refactor: avoid monkeypatch copy
* docs: minor typos corrected
* fix: minor points from @YannickJadoul
* fix: typo on Windows C++ mode
* fix: MSVC 15 update 3+ have c++14 flag
See <https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=vs-2019>
* docs: discuss making SDists by hand
* ci: use pep517.build instead of manual setup.py
* refactor: more comments from @YannickJadoul
* docs: updates from @ktbarrett
* fix: change to newly recommended tool instead of pep517.build
This was intended as a proof of concept; build seems to be the correct replacement.
See https://github.com/pypa/pep517/pull/83
* docs: updates from @wjakob
* refactor: dual version locations
* docs: typo spotted by @wjakob
