Commit Graph

1600 Commits

Author SHA1 Message Date
Henry Schreiner
cddec2bb1d fix: limit warning issue (#5807)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-08-21 16:07:26 -04:00
Henry Schreiner
adb5603f60 chore: rename generic slots variable (#5793)
* fix: better compatibility with Qt

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

style: pre-commit fixes

* refactor: use mod_def_slots as a name instead

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-08-21 11:06:30 -04:00
Justen Di Ruscio
c1bf55a211 Fix subinterpreter exception handling SEGFAULT (#5795)
* check for current exception, not uncaught_exceptions

* remove all in-flight exception handling from ~subinterpreter_scoped_activate
2025-08-15 00:03:03 -07:00
Justen Di Ruscio
94c8250818 inline get_interpreter_state_uncheccked inline function (#5789) 2025-08-11 19:25:09 -07:00
Ralf W. Grosse-Kunstleve
a5665e3aca fix: type_caster_enum_type for pointer types (#5694) (#5776)
Add pointer overload to type_caster_enum_type::cast method to handle
enum pointer casting. Fixes compilation error when returning pointers
to enum types from bound functions.

Experiment for validation:

Temporarily undo the changes in include/pybind11/cast.h:

```
g++ -o pybind11/tests/test_native_enum.os -c -std=c++20 -fPIC -fvisibility=hidden -O0 -g -Wall -Wextra -Wconversion -Wcast-qual -Wdeprecated -Wundef -Wnon-virtual-dtor -Wunused-result -Werror -funsigned-char -Wpedantic -isystem /usr/include/python3.12 -isystem /usr/include/eigen3 -DPYBIND11_SMART_HOLDER_PADDING_ON -DPYBIND11_STRICT_ASSERTS_CLASS_HOLDER_VS_TYPE_CASTER_MIX -DPYBIND11_ENABLE_TYPE_CASTER_ODR_GUARD_IF_AVAILABLE -DPYBIND11_TEST_BOOST -Ipybind11/include -I/home/rgrossekunst/forked/pybind11/include -I/home/rgrossekunst/clone/pybind11/include /home/rgrossekunst/forked/pybind11/tests/test_native_enum.cpp
In file included from /home/rgrossekunst/forked/pybind11/include/pybind11/native_enum.h:10,
                 from /home/rgrossekunst/forked/pybind11/tests/test_native_enum.cpp:1:
/home/rgrossekunst/forked/pybind11/include/pybind11/cast.h: In instantiation of ‘static pybind11::handle pybind11::detail::type_caster_enum_type<EnumType>::cast(SrcType&&, pybind11::return_value_policy, pybind11::handle) [with SrcType = const test_native_enum::color*; EnumType = test_native_enum::color]’:
/home/rgrossekunst/forked/pybind11/include/pybind11/pybind11.h:429:40:   required from ‘void pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = test_submodule_native_enum(pybind11::module_&)::<lambda()>; Return = const test_native_enum::color*; Args = {}; Extra = {pybind11::name, pybind11::scope, pybind11::sibling}]’
/home/rgrossekunst/forked/pybind11/include/pybind11/pybind11.h:274:19:   required from ‘pybind11::cpp_function::cpp_function(Func&&, const Extra& ...) [with Func = test_submodule_native_enum(pybind11::module_&)::<lambda()>; Extra = {pybind11::name, pybind11::scope, pybind11::sibling}; <template-parameter-1-3> = void]’
/home/rgrossekunst/forked/pybind11/include/pybind11/pybind11.h:1384:22:   required from ‘pybind11::module_& pybind11::module_::def(const char*, Func&&, const Extra& ...) [with Func = test_submodule_native_enum(pybind11::module_&)::<lambda()>; Extra = {}]’
/home/rgrossekunst/forked/pybind11/tests/test_native_enum.cpp:139:10:   required from here
/home/rgrossekunst/forked/pybind11/include/pybind11/cast.h:70:32: error: invalid ‘static_cast’ from type ‘const test_native_enum::color*’ to type ‘pybind11::detail::type_caster_enum_type<test_native_enum::color>::Underlying’ {aka ‘unsigned int’}
   70 |             return native_enum(static_cast<Underlying>(src)).release();
      |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/rgrossekunst/forked/pybind11/include/pybind11/cast.h: In instantiation of ‘static pybind11::handle pybind11::detail::type_caster_enum_type<EnumType>::cast(SrcType&&, pybind11::return_value_policy, pybind11::handle) [with SrcType = test_native_enum::color*; EnumType = test_native_enum::color]’:
/home/rgrossekunst/forked/pybind11/include/pybind11/pybind11.h:429:40:   required from ‘void pybind11::cpp_function::initialize(Func&&, Return (*)(Args ...), const Extra& ...) [with Func = test_submodule_native_enum(pybind11::module_&)::<lambda()>; Return = test_native_enum::color*; Args = {}; Extra = {pybind11::name, pybind11::scope, pybind11::sibling}]’
/home/rgrossekunst/forked/pybind11/include/pybind11/pybind11.h:274:19:   required from ‘pybind11::cpp_function::cpp_function(Func&&, const Extra& ...) [with Func = test_submodule_native_enum(pybind11::module_&)::<lambda()>; Extra = {pybind11::name, pybind11::scope, pybind11::sibling}; <template-parameter-1-3> = void]’
/home/rgrossekunst/forked/pybind11/include/pybind11/pybind11.h:1384:22:   required from ‘pybind11::module_& pybind11::module_::def(const char*, Func&&, const Extra& ...) [with Func = test_submodule_native_enum(pybind11::module_&)::<lambda()>; Extra = {}]’
/home/rgrossekunst/forked/pybind11/tests/test_native_enum.cpp:143:10:   required from here
/home/rgrossekunst/forked/pybind11/include/pybind11/cast.h:70:32: error: invalid ‘static_cast’ from type ‘test_native_enum::color*’ to type ‘pybind11::detail::type_caster_enum_type<test_native_enum::color>::Underlying’ {aka ‘unsigned int’}
```
2025-08-06 17:11:45 -04:00
b-pass
c5e8fec920 Make function record subinterpreter safe (#5771)
* Make function_record type subinterpreter safe

* Get rid of static state in implicit conversion

* style: pre-commit fixes

* Fix lambda

* Bump ABI because we added an internals member

* Set __module__ on the type instance to get rid of DepricationWarning

* Work around internal compiler error in CUDA by not using typedef

hopefully

* Make clang-tidy happy

* Use the same __module__ as pybind11_static_property

* style: pre-commit fixes

* Oops, find-replace error

* style: pre-commit fixes

* Move the once initialization to happen more behind the scenes

* Oops, need those casts...

* Undo implicit conversion change, will do a separate PR

* Use local_internals for function_record pointer to avoid ABI bump

* style: pre-commit fixes

* Get rid of this auto for readability

* Change back to using unqualified tp_name, set __module__ attribute, explicitly add Py_TPFLAGS_HEAPTYPE → does not resolve DeprecationWarning :-(

* Revert "Change back to using unqualified tp_name, set __module__ attribute, explicitly add Py_TPFLAGS_HEAPTYPE → does not resolve DeprecationWarning :-("

This reverts commit 9ccd6de9b7.

* Add Py_TPFLAGS_HEAPTYPE to be explicit (more readable).

* Remove obsolete PYBIND11_WARNING_DISABLE_...

* Make tp_plainname_impl, tp_qualname_impl more DRY

* Change PYBIND11_INTERNAL_MODULE_NAME → PYBIND11_DUMMY_MODULE_NAME

* Add a long comment to explain the tp_qualname_impl workaround.

* Rename local_internals::function_record → function_record_py_type

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2025-08-04 05:00:33 -07:00
b-pass
0db3d59fdf Allow multiphase modules to be re-imported (#5782)
* Failing unit test

* Potential fix for the issue of re-importing a multi-phase module

- When a module is successfully imported and exec'd, save its handle in a dict in the interpreter state
- Use a special Py_mod_create slot to look in the cache and return the cached handle if it is in the cache
- Don't re-run the user exec function if the module is in the interpreter's cache (implying it was already successfully imported)

* Oops, need to inline these.

* Clang-Tidy fixes

* Oops, debug code

* Add xfail for this GraalPy bug

* Remove static from these function defs, it was a cut-and-paste error in the first place.

* Fix test comment

* Proper error handling

* Oops

* Split up this line, but still just ignore failure .. if the module doesn't have the right properties to check the cache then just allow exec to run.

* Clean up - already looked up the name, just use that.

* Some compilers complain if the pointer isn't taken here, weird.

* Allow attribute errors to be thrown here, will be converted to import errors by the exception handler.

* Remove bogus incref, unconditionally expect a __spec__.name on the module

* Add PR to test comment

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-08-02 19:48:38 -07:00
b-pass
780ec11360 Make implicitly_convertable sub-interpreter and free-threading safe (#5777)
* Replace static bool with thread-specific-storage to make this code sub-interpreter and free-threading safe.

* Make sure there is only one tss value in existence for this.

The previous code had multiple (one for every type pair, as this is a template function), which may have posed a problem for some platforms.

* Make set_flag in implicitly_convertible() non-copyable/movable

set_flag is an RAII guard for a thread-specific reentrancy flag.
Copying or moving it would risk double-resetting or rearming the flag,
breaking the protection. Disable copy/move constructors and assignment
operators to make this explicit.

* Minor cleanup to avoid venturing into UB territory.

* Experiment: Disable `~thread_specific_storage()` body when using GraalPy.

* Try the suggestion to only call TSS_free if the python interpreter is still active.

* Add IsFinalizing check

* Put this back to having a per-template-instance static

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
2025-08-01 20:04:44 -07:00
b-pass
9af8adb712 Subinterpreter creation concurrency issues in 3.12 (#5779)
* Seems like 3.12 has concurrency issues when creating a subinterpreter, easiest workaround is just to lock during it.

* Only need this for PER_INTERPRETER_GIL
2025-07-30 21:31:54 -07:00
Paul Fultz II
7f5eea432e Check for __cpp_lib_remove_cvref as well (#5761)
C++20 can be enabled while the C++ runtime is still much older so use the feature macro to check for it. 

For example, we are using the latest clang with c++23 on SLES, while the gcc version is 7.
2025-07-24 10:01:18 -07:00
Mike Jarvis
49d19fef68 Implement binary version of make_index_sequence (#5751)
* Implement binary version of make_index_sequence

* style: pre-commit fixes

* typo

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-07-24 09:56:32 -07:00
moi15moi
316273efc8 fix: don't force -fvisibility=hidden on Windows (#5757)
Fix https://github.com/pybind/pybind11/discussions/5750
2025-07-18 09:51:33 -07:00
Thomas Braun
c4ee83c498 fix:: initialize_generic compiler warning about nullptr dereference (#5756)
When compiling an application using pybind11 3.0.0, GCC 13.3.0 and
python 3.11.13 the following warning is emitted [1]:

  In function 'PyObject* PyCFunction_GET_SELF(PyObject*)',
      inlined from 'void pybind11::cpp_function::initialize_generic(unique_function_record&&, const char*, const std::type_info* const*, pybind11::size_t)' at /opt/conda/lib/python3.11/site-packages/pybind11/include/pybind11/pybind11.h:605:30:
  /opt/conda/include/python3.11/cpython/methodobject.h:50:16: error: potential null pointer dereference [-Werror=null-dereference]
     50 |         return _Py_NULL;
        |                ^~~~~~~~

It stems form the fact that PyCFunction_GET_SELF can return a nullptr.
Let's fail in this case.

[1]: https://gitlab.com/tango-controls/pytango/-/jobs/10671972312#L570
2025-07-18 09:50:33 -07:00
Ralf W. Grosse-Kunstleve
422990f842 chore: get back to work (after v3.0.0 release) (#5747)
* Change version on master to v3.0.1a0

* Fix up PYBIND11_VERSION_PATCH
2025-07-11 13:25:25 -07:00
Ralf W. Grosse-Kunstleve
ed5057ded6 chore: prepare for 3.0.0 (final) (#5746)
* Update docs/changelog.md and change version to v3.0.0 (final)

* [skip ci] Add `|SPEC 4 — Using and Creating Nightly Wheels|` badge in main README.rst
2025-07-10 09:16:14 -07:00
Ralf W. Grosse-Kunstleve
4dc4aca2e1 [skip ci] Explain: conduit feature only covers from-Python-to-C++ conversions (#5740) 2025-06-20 13:23:26 -07:00
Henry Schreiner
03d8f48750 docs: fix typo in multiple_interpreters (#5738)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-06-19 11:46:03 -04:00
Henry Schreiner
ea3e33e40d chore: prepare for 3.0.0rc4 (#5736)
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-06-19 10:51:52 -04:00
Henry Schreiner
cf3d1a75a2 feat: numpy scalars (#5726) 2025-06-18 19:40:31 -04:00
b-pass
f2c0ab83bc Fix TSan warning in sub-interpreter test (#5729) 2025-06-17 20:50:30 -07:00
Ralf W. Grosse-Kunstleve
86e82ddbc2 Add support for shared_ptr<const T> in py::init() with smart_holder (#5731)
* Add overload to enable `.def(py::init(&rtrn_shcp))`. Also uncomment `.def(py::init(&rtrn_uqcp))` and `.def(py::init(&rtrn_udcp))`, which happen to work already (not sure what change in the past made those work).

* Introduce `construct_from_shared_ptr()` helper for DRY-ness.
2025-06-17 12:16:56 -07:00
Ralf W. Grosse-Kunstleve
365d41a4ba Eliminate cross-DSO RTTI reliance in smart_holder functionality (for platforms like macOS). (#5728)
* Revert PR #5700 production code change (pybind11/detail/struct_smart_holder.h).

```
git checkout b19489145b2c7a117138632d624809dfb3b380bb~1 include/pybind11/detail/struct_smart_holder.h
```

* Introduce `get_internals().get_memory_guarded_delete()`

* [skip ci] Only pass around `memory::get_guarded_delete` function pointer.

* [skip ci] Change a variable name for internal consistency. Add 3 x NOTE: PYBIND11_INTERNALS_VERSION needs to be bumped if changes are made to this struct.

* Add comment: get_internals().get_memory_guarded_delete does not need with_internals()

* Traverse all DSOs to find memory::guarded_delete with matching RTTI.

* Add nullptr check to dynamic_cast overload.

Suggested by ChatGPT for these reasons:

* Prevents runtime RTTI lookups on nullptr.

* Helps avoid undefined behavior if users pass in nulls from failed casts or optional paths.

* Ensures consistent return value semantics and no accidental access to vtable.

* Improve smart_holder unique_ptr deleter compatibility checks across DSOs:

* Replace RTTI-based detection of std::default_delete<T> with a constexpr check to avoid RTTI reliance

* Add type_info_equal_across_dso_boundaries() fallback using type_info::name() for RTTI equality across macOS DSOs

* Rename related flags and functions for clarity (e.g., builtin → std_default)

* Improves ABI robustness and clarity of ownership checks in smart_holder

* Trivial renaming for internal consistency: builtin_delete → std_default_delete

* Add get_trampoline_self_life_support to detail::type_info (passes local testing).

* Polish previous commit slightly.

* [skip ci] Store memory::get_guarded_delete in `detail::type_info` instead of `detail::internals` (no searching across DSOs required).

* Revert change suggested by ChatGPT. After double-checking, ChatGPT agrees this isn't needed.

* Minor polishing.
2025-06-17 12:14:50 -07:00
Rosdf
7ed76e2d50 fix: add support for const-only smart pointers (#5718)
* add support for const pointers in smart pointers

* use c++11 compatible code

* add template parameter in test

* Make the const-removal clearly visible. This simplifies the production code changes significantly.

For background see: https://claude.ai/share/4085d9ab-a859-44cc-bb56-450e472f817a

* test without leaks

* add namespace for test

* rename test

* fix test compilation

* using namespace test_const_only_smart_ptr;

* fix smartptr in test

* smaller test body

* move test

* style: pre-commit fixes

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-11 12:32:26 -04:00
Henry Schreiner
5d32ed76c0 docs: prepare for RC 3 (#5713)
* docs: prepare for RC 3

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* docs: one more rc 3 update

* chore: bump to 3.0.0rc3

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-06-05 11:14:33 -04:00
Xuehai Pan
c786d34f60 fix: handle null py::handle and add tests for py::scoped_critical_section (#5706)
* chore: handle null for `py::scoped_critical_section`

* test: add tests for `py::scoped_critical_section`

* test: use assert instead of REQUIRE

* feat: enable faulthandler for pytest

* chore: use `__has_include(<barrier>)`

* fix: fix segmentation fault in test

* fix: test critical_section for no-gil only

* test: run new tests only

* test: ensure non-empty test selection

* fix: fix test critical_section

* fix: change Python 3.14.0b1/2 xfail tests to non-strict

* test: trigger gc manually

* test: mark xfail to `DynamicClass`

* Use `namespace test_scoped_critical_section_ns` (standard approach to guard against name clashes).

* Simplify changes in pybind11/critical_section.h and add test_nullptr_combinations()

* test: disable Python devmode in pytest

* test: add comprehensive comments for the tests

* test: add a summary comment for tests

* refactor: simpler impl

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-06-04 11:45:26 -04:00
b-pass
c7026d0d1c fix!: modify the internals pointer-to-pointer implementation to not use thread_local (#5709)
* Refactor internals to use a holder that manages the PP

* Refactor internals to use a holder that manages the PP

* Fix cleanup/destruction issues.

* Fix one more destruction issue

Should now just be able to delete the internals PP on destruction

* Make clang-tidy happy

* Try to fix exception translators issue on certain platforms

Also fix a couple more pedantic warings

* Fix test, after internals is free'd it can come back at the same address

So instead, just make sure it was zero'd and don't try to compare the addresses.

Also a little code cleanup

* Comment tweak [skip ci]

* Switch to ifdef instead of if

* Re-enable subinterpreters in iOS

* style: pre-commit fixes

* Oops, this snuck in on merge

* fix: bump ABI version to 10

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-06-03 16:02:44 -04:00
Peter Steneteg
b19489145b fix: expose required symbol using clang (#5700)
* test: Added test case for visibility of common symbols across shared libraries

* style: pre-commit fixes

* tests: cmake target name fix

* tests: Added visibility test to ci

* tests: set the default visibility to hidden

* prototype/proof-of-concept fix: PYBIND11_EXPORT_GUARDED_DELETE

* Fix silly oversight: actually use PYBIND11_EXPORT_GUARDED_DELETE

* Update struct_smart_holder.h

* style: pre-commit fixes

* Update include/pybind11/detail/struct_smart_holder.h

* Update struct_smart_holder.h

* ci: fix addition to reusable-standard.yml

* Update CMakeLists.txt

* refactor: rename tests to test_cross_module_rtti

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2025-06-03 16:02:19 -04:00
Henry Schreiner
9295c4a568 fix: follow rest of pybind11 closer with PYBIND11_HAS_SUBINTERPRETER_SUPPORT (#5710)
* fix: follow rest of pybind11 closer with PYBIND11_HAS_SUBINTERPRETER_SUPPORT

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* style: pre-commit fixes

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-06-02 16:08:40 -04:00
Michael Carlstrom
c2b32b1e3b fix(types): type hints from future python versions (#5693)
* fix future type hints

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* style: pre-commit fixes

* remove unused var

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* remove union_helper

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* fix speelling error

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* base case for union_concat

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* style: pre-commit fixes

* add case for one descr

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* weakref and final test

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* Add acrpss_version_type_hint_checker

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* cleanup

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* style: pre-commit fixes

* remove test.pyi

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* use new unions and add fixture

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* timohl suggested cleanup

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* style: pre-commit fixes

* add missing auto

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>

* style: pre-commit fixes

* move operator| def

---------

Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-31 23:47:20 -04:00
Henry Schreiner
e4873e8f59 fix: allow subinterp support to be disabled
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-05-31 15:44:42 -04:00
Henry Schreiner
a1b1972241 chore: prepare for 3.0.0rc2 (#5698)
* chore: prepare for 3.0.0rc2

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Update docs/changelog.md

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-05-29 01:16:32 -04:00
b-pass
03b4a9e56f fix: don't destruct module objects in atexit (#5688) 2025-05-25 18:56:26 -04:00
Henry Schreiner
1dd85ef42a chore: bump maximum clang tested to 20 (#5692) 2025-05-25 10:54:05 -04:00
Henry Schreiner
d7769de533 feat: scoped_critical_section (#5684)
* feat: scoped_critical_section

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* refactor: pull out to file

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* style: pre-commit fixes

* fix: GIL code in some compilers

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: move to correct spot

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-23 17:41:45 -04:00
Xuehai Pan
e3883dd5d5 refactor: use CPython macros to construct PYBIND11_VERSION_HEX (#5683)
* refactor: use CPython macros to construct `PYBIND11_VERSION_HEX`

* docs: update release guide

* tests: add test to keep version values in sync

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* style: pre-commit fixes

* test: update version test

* test: update version test

* test: update version test

* chore: update code comments

* Update docs/release.rst

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-23 17:12:46 -04:00
Xuehai Pan
98bd78f063 chore: rename macro PYBIND11_SUBINTERPRETER_SUPPORT -> PYBIND11_HAS_SUBINTERPRETER_SUPPORT to meet naming convention (#5682) 2025-05-22 10:16:57 -07:00
b-pass
07d028f218 feat: Embeded sub-interpreters (#5666)
* First draft a subinterpreter embedding API

* Move subinterpreter tests to their own file

* Migrate subinterpreter tests to use the new embedded class.

* Add a test for moving subinterpreters across threads for destruction

And find a better way to make that work.

* Code organization

* Add a test which shows demostrates how gil_scoped interacts with sub-interpreters

* Add documentation for embeded sub-interpreters

* Some additional docs work

* Add some convenience accessors

* Add some docs cross references

* Sync some things that were split out into #5665

* Update subinterpreter docs example to not use the CPython api

* Fix pip test

* style: pre-commit fixes

* Fix MSVC warnings

I am surprised other compilers allowed this code with a deleted move ctor.

* Add some sub-headings to the docs

* Oops, make_unique is C++14 so remove it from the tests.

* I think this fixes the EndInterpreter issues on all versions.

It just has to be ifdef'd because it is slightly broken on 3.12, working well on 3.13, and kind of crashy on 3.14beta.  These two verion ifdefs solve all the issues.

* Add a note about exceptions.

They contain Python object references and acquire the GIL, that means they are a danger with subinterpreters!

* style: pre-commit fixes

* Add try/catch to docs examples to match the tips

* Python 3.12 is very picky about this first PyThreadState

Try special casing the destruction on the same thread.

* style: pre-commit fixes

* Missed a rename in a ifdef block

* I think this test is causing problems in 3.12, so try ifdefing it to see if the problems go away.

* style: pre-commit fixes

* Document the 3.12 constraints with a warning

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* ci: add cpptest to the clang-tidy job

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* noexcept move operations

* Update include/pybind11/subinterpreter.h

std::memset

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
2025-05-20 18:17:48 -04:00
b-pass
9e6fe464be chore: consolidate common code in PYBIND11_MODULE and PYBIND11_EMBEDDED_MODULE (#5670)
* Move common code from PYBIND11_MODULE and PYBIND11_EMBEDDED_MODULE into one macro

The difference between the two is really small, almost all of the code was the same.

* Use a macro to make the forward decl correct and reduce code duplication

* Oops, extra semicolon

* Split up the macro into two so we can avoid the redundant decl
2025-05-19 20:34:26 -04:00
gentlegiantJGC
67424358f4 fix(types): add typing and collections.abc module prefix (#5663)
* Fix Python 3.8 type hints and add module prefix

These type hints are invalid in Python 3.8.
Add `typing.` prefix to remove ambiguity.

* style: pre-commit fixes

* Add module prefix to Union

* Rename macros

* Improve comment

* Comment out 3.8 type hint macros

Fixing this issue in Python 3.8 will require updating lots of tests. This can be added in a further pull request.

* Add Iterable module prefix

* Add module prefix to Iterator

* Add module prefix to Callable

* Re-add accidentally deleted brackets

* Add module prefix to Optional

* Add module prefix to Final

* Add module prefix to ClassVar

* Add module prefix to TypeGuard

* Add module prefix to TypeIs

* Add module prefix to NoReturn

* Add module prefix to Never

* Add module prefix to Literal

* Add module prefix to Callable

* Add module prefix to Sequence

* Add module prefix to Iterator

* style: pre-commit fixes

* Remove type hint macros

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-19 11:54:31 -04:00
Ralf W. Grosse-Kunstleve
4587d33cf9 docs: prepare for v3.0.0rc1 (#5589)
* Squashed prepv300/manuscript — 30b9c268aeb98308ea42aaccfd5fe454e173c6fc — 2025-03-30 14:56:03 -0700 [skip ci]

[Browse prepv300/manuscript tree](30b9c268ae)

[Browse prepv300/manuscript commits](30b9c268ae/)

* docs: update changelog

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* docs: upgrade guide CMake suggestions

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Explain type_caster_enum_type_enabled, copyable_holder_caster_shared_ptr_with_smart_holder_support_enabled, move_only_holder_caster_unique_ptr_with_smart_holder_support_enabled in Upgrade guide.

* Add a small section for py::bind_vector, py::bind_map & py::smart_holder

* Fix tiny oversight: Reference back to the current release v2.13 (not v2.12)

* Remove sentence: Using self._pybind11_conduit_v1_() ... should keep extension compatibility.

This isn't true, because we also modernized `PYBIND11_PLATFORM_ABI_ID`
(which I believe was absolutely necessary). I think it'll be too complicated
to explain that here, and there is a mention in the Upgrade guide.

* Changelog: combine #4953 and #5439

* Trivial whitespace/formatting fixes/enhancements.

* chore: add more to deprecation page

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* docs: update for recent additions

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* docs: fixes and set rc1 version

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: support rc versions

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Undo erroneous copilot change: We need to use `detail::enable_if_t`, for compatibility with C++11 and C++14.

* Empty lines cleanup.

* Rewording of "CMake support now defaults to ..." paragraph.

* Add missing backticks in upgrade guide.

* Try :ref:deprecated instead of :doc:deprecated

* docs: last bit of polish

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Piggy-back trivial whitespace cleanup that was missed in PR #5669

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-05-19 09:02:11 -04:00
Xuehai Pan
6bf25d1e39 feat: add semi-public API: pybind11::detail::is_holder_constructed (#5669)
* Add semi-public API: `pybind11::detail::is_holder_constructed`

* Update docs for `pybind11::custom_type_setup`

* Update tests in `test_custom_type_setup.cpp`

* Apply suggestions from code review
2025-05-18 21:11:56 -04:00
b-pass
9afc9c4ffb feat: change PYBIND11_EMBEDDED_MODULE to multiphase init (#5665)
* Move embedded modules to multiphase init

So that they too can support multi-interpreter and nogil tags

* Update the multiple interpreter test for embedded module changes

* Add a note to embedded module docs about the new tags

* Oops, missed a warning pop

* Remove unused variable

* Update ci.yml

* Fix this embedded GIL test for free-threading

* Oops, need to use ptr() here

* This test created a subinterpreter when PYBIND11_SUBINTERPRETER_SUPPORT was off

So the fix is really this test should not be run in these older versions at all.

The hang was a GIL issue between the subinterpreters during pybind11::exception::what().

* fix: standard mutex for 3.13t

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2025-05-18 09:39:26 -04:00
Henry Schreiner
094343c74a fix: support Python 3.14 (#5646)
* ci: support Python 3.14

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: Python 3.14 name change

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: fix expected output to handle Python 3.14

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: tighten CLI and add color on 3.14+

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: ignore failure on 3.14.0b1

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: support Python 3.14.0b1 with interperters

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Update test_multiple_interpreters.py

* Update test_multiple_interpreters.py

* fix: new breakage for 3.14 fixed

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: handle empty annotations 3.14

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: Python 3.14 may not create the annotations dict

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: use PyUnstable_IsImmortal

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: use sys._is_immortal

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: ignore large values for refcount too

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* style: pre-commit fixes

* ci: enable all free-threaded builds

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: patch for embed

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Revert "fix: patch for embed"

This reverts commit c4226a0671.

* ci: drop new 3.xt additions

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: logic issue, also add some comments

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Update include/pybind11/pytypes.h

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-16 21:58:43 -04:00
gentlegiantJGC
6aa3b335f4 fix(types): Buffer type hint (#5662)
* Fix Buffer type hint

collections.abc.Buffer was added in Python 3.12.
The previous behaviour should be used prior to this version.

* Fix comment

* Fix indentation

* style: pre-commit fixes

* Fix test

* Add missing import

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-05-14 10:41:08 -04:00
b-pass
95e8f89be1 Support for sub-interpreters (#5564)
* Allow per-interpreter internals/local_internals

* Significant rewrite to avoid using thread_locals as much as possible.

Since we can avoid them by checking this atomic, the cmake config conditional shouldn't be necessary.

The slower path (with thread_locals and extra checks) only comes in when a second interpreter is actually instanciated.

* Add a test for per-interpreter GIL

Uses two extra threads to demonstrate that neither shares a GIL.

* Fix for nonconforming std::atomic constructors on some compilers

* style: pre-commit fixes

* Fix initializer to make MSVC happy.

* Switch to gil_scoped_acquire_simple, get rid of old copy of it from internals.h

* Use the PyThreadState's interp member rather than the thread state itself.

* Be more explicit about the type of the internalspp

* Suggested renamings and rewordings

* Rename find_internals_pp and change it to take in the state dict reference

* Use the old raise_from instead of pybind11_fail

* Move most of the internals initialization into its constructor.

* Move round_up_to_next_pow2 function upwards

* Remove redundant forward decl

* Add a python-driven subinterpreter test

* Disable the python subinterpreter test on emscripten

Can't load the native-built cpp modules.

* Switch the internals pointer pointer to a unique_ptr pointer

* Spelling

* Fix clang-tidy warning, compare pointer to nullptr

* Rename get_interpreter_counter to get_num_interpreters_seen

* Try simplifying the test's cmake set_target_properties

* Replace mod_* tags with a single tag w/enum

Update tests accordingly

* Add a test for shared-GIL (legacy) subinterpreters

* Update test to work around differences in the various versions of interpreters modules

* Fix unused parameter

* Rename tests and associated test modules.

* Switch get_internals_pp to a template function

* Rename curtstate to cur_tstate

* refactor: use simpler names

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* style: pre-commit fixes

* fix: return class, not enum

Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Have to join these threads to make sure they are totally done before the test returns.

* Wrap module_def initialization in a static so it only happens once.

If it happens concurrently in multiple threads, badness ensues....

* style: pre-commit fixes

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
2025-05-14 00:59:44 -07:00
Henry Schreiner
853bafa0a2 fix: use original dict (#5658)
* fix: use original dict

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* refactor: handle unset dict just in case

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-05-13 14:09:05 -04:00
Noah Oblath
74b5242713 feat: add py::potentially_slicing_weak_ptr(handle) function (#5624)
* Added weak_ptr test (currently failing)

* Cleanup

* [skip ci] Simplify test case.

* Add test_class_sp_trampoline_weak_ptr.cpp,py (using std::shared_ptr as holder). Tweak test_class_sh_trampoline_weak_ptr.py to pass, with `# THIS NEEDS FIXING` comment.

* Resolve clang-tidy errors

```
/__w/pybind11/pybind11/tests/test_class_sh_trampoline_weak_ptr.cpp:23:43: error: the parameter 'sp' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param,-warnings-as-errors]
   23 |     void set_wp(std::shared_ptr<VirtBase> sp) { wp = sp; }
      |                                           ^
      |                 const                    &
4443 warnings generated.
```

```
/__w/pybind11/pybind11/tests/test_class_sp_trampoline_weak_ptr.cpp:23:43: error: the parameter 'sp' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param,-warnings-as-errors]
   23 |     void set_wp(std::shared_ptr<VirtBase> sp) { wp = sp; }
      |                                           ^
      |                 const                    &
4430 warnings generated.
```

* PYPY, GRAALPY: skip last part of test_weak_ptr_base

* Rename test_weak_ptr_base → test_weak_ptr_owner

* Add SpOwner, test_with_sp_owner, test_with_sp_and_wp_owners

* Modify py::trampoline_self_life_support semantics: if trampoline class does not inherit from this class, preserve established Inheritance Slicing behavior.

rwgk reached this point with the help of ChatGPT:

* https://chatgpt.com/share/68056498-7d94-8008-8ff0-232e2aba451c

The only production code change in this commit is:

```
diff --git a/include/pybind11/detail/type_caster_base.h b/include/pybind11/detail/type_caster_base.h
index d4f9a41e..f3d45301 100644
--- a/include/pybind11/detail/type_caster_base.h
+++ b/include/pybind11/detail/type_caster_base.h
@@ -776,6 +776,14 @@ struct load_helper : value_and_holder_helper {
                 if (released_ptr) {
                     return std::shared_ptr<T>(released_ptr, type_raw_ptr);
                 }
+                auto *self_life_support
+                    = dynamic_raw_ptr_cast_if_possible<trampoline_self_life_support>(type_raw_ptr);
+                if (self_life_support == nullptr) {
+                    std::shared_ptr<void> void_shd_ptr = hld.template as_shared_ptr<void>();
+                    std::shared_ptr<T> to_be_released(void_shd_ptr, type_raw_ptr);
+                    vptr_gd_ptr->released_ptr = to_be_released;
+                    return to_be_released;
+                }
                 std::shared_ptr<T> to_be_released(
                     type_raw_ptr, shared_ptr_trampoline_self_life_support(loaded_v_h.inst));
                 vptr_gd_ptr->released_ptr = to_be_released;
```

* Remove debug printf in include/pybind11/pybind11.h

* Resolve MSVC error

```
    11>D:\a\pybind11\pybind11\tests\test_class_sp_trampoline_weak_ptr.cpp(44,50): error C2220: the following warning is treated as an error [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
    11>D:\a\pybind11\pybind11\tests\test_class_sp_trampoline_weak_ptr.cpp(44,50): warning C4458: declaration of 'sp' hides class member [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
             D:\a\pybind11\pybind11\tests\test_class_sp_trampoline_weak_ptr.cpp(54,31):
             see declaration of 'pybind11_tests::class_sp_trampoline_weak_ptr::SpOwner::sp'
```

* [skip ci] Undo the production code change under 4638e017b6

Also undo the corresponding test change in test_class_sh_trampoline_weak_ptr.py

But keep all extra debugging code for now.

* [skip ci] Introduce lambda in `WpOwner::set_wp` bindings, but simply cast to `std::shared_ptr<VirtBase>` for now.

* Add `py::potentially_slicing_shared_ptr()`

* Add `type_id<T>()` to `py::potentially_slicing_shared_ptr()` error message and add test.

* test_potentially_slicing_shared_ptr.cpp,py (for smart_holder only)

* Generalize test_potentially_slicing_shared_ptr.cpp,py for testing with smart_holder and std::shared_ptr as holder.

* Add back test_potentially_slicing_shared_ptr_not_convertible_error(), it got lost accidentally in commit 56d23dc478

* Add simple trampoline state assertions.

* Resolve clang-tidy errors.

```
/__w/pybind11/pybind11/tests/test_potentially_slicing_shared_ptr.cpp:30:9: error: 'magic_token' should be initialized in a member initializer of the constructor [cppcoreguidelines-prefer-member-initializer,-warnings-as-errors]
   29 |     trampoline_is_alive_simple(const trampoline_is_alive_simple &other) {
      |                                                                         : magic_token(other.magic_token)
   30 |         magic_token = other.magic_token;
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/__w/pybind11/pybind11/tests/test_potentially_slicing_shared_ptr.cpp:33:9: error: 'magic_token' should be initialized in a member initializer of the constructor [cppcoreguidelines-prefer-member-initializer,-warnings-as-errors]
   32 |     trampoline_is_alive_simple(trampoline_is_alive_simple &&other) noexcept {
      |                                                                             : magic_token(other.magic_token)
   33 |         magic_token = other.magic_token;
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

* Add a (long) C++ comment for py::potentially_slicing_shared_ptr<>()

* [skip ci] Add new "Avoiding Inheritance Slicing and ``std::weak_ptr`` surprises" section in advanced/classes.rst

* [skip ci] Add introductory comment to test_potentially_slicing_shared_ptr.py

* Minimal (!) changes to have py::potentially_slicing_weak_ptr<T>(handle) as the public API. For CI testing, before changing the names around more widely, and the documentation.

* Rename test_potentially_slicing_shared_ptr.cpp,py → test_potentially_slicing_weak_ptr.cpp,py

* Update docs/advanced/classes.rst and C++ comments → potentially_slicing_weak_ptr

* Write "shared_ptr" instead of just "pointer" in a couple places in docs/advanced/classes.rst

* Add comment for force_potentially_slicing_shared_ptr in type_caster_base.h, to make a direct connection py::potentially_slicing_weak_ptr

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2025-05-13 10:07:33 -04:00
Henry Schreiner
d7d782c5e0 fix(regression): support embedded submodule (#5650)
* tests: add test for embedded submodule

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: adjust

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Update test_interpreter.cpp

* Update test_interpreter.cpp

* Update test_interpreter.cpp

* Update test_interpreter.cpp

* Update pybind11.h

* fix: end instead of endif

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Update include/pybind11/pybind11.h

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
2025-05-12 05:32:36 -04:00
Ralf W. Grosse-Kunstleve
c125cc789c Collect all #define PYBIND11_HAS_... in pybind11/detail/common.h (#5647) 2025-05-08 23:46:01 -07:00
Xuehai Pan
002c05b17e fix: handle MSVC warning C4866: compiler may not enforce left-to-right evaluation order (#5641) 2025-05-06 21:40:54 -04:00