* feat: scoped_critical_section
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* refactor: pull out to file
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* style: pre-commit fixes
* fix: GIL code in some compilers
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: move to correct spot
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* refactor: use CPython macros to construct `PYBIND11_VERSION_HEX`
* docs: update release guide
* tests: add test to keep version values in sync
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* style: pre-commit fixes
* test: update version test
* test: update version test
* test: update version test
* chore: update code comments
* Update docs/release.rst
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* First draft a subinterpreter embedding API
* Move subinterpreter tests to their own file
* Migrate subinterpreter tests to use the new embedded class.
* Add a test for moving subinterpreters across threads for destruction
And find a better way to make that work.
* Code organization
* Add a test which shows demostrates how gil_scoped interacts with sub-interpreters
* Add documentation for embeded sub-interpreters
* Some additional docs work
* Add some convenience accessors
* Add some docs cross references
* Sync some things that were split out into #5665
* Update subinterpreter docs example to not use the CPython api
* Fix pip test
* style: pre-commit fixes
* Fix MSVC warnings
I am surprised other compilers allowed this code with a deleted move ctor.
* Add some sub-headings to the docs
* Oops, make_unique is C++14 so remove it from the tests.
* I think this fixes the EndInterpreter issues on all versions.
It just has to be ifdef'd because it is slightly broken on 3.12, working well on 3.13, and kind of crashy on 3.14beta. These two verion ifdefs solve all the issues.
* Add a note about exceptions.
They contain Python object references and acquire the GIL, that means they are a danger with subinterpreters!
* style: pre-commit fixes
* Add try/catch to docs examples to match the tips
* Python 3.12 is very picky about this first PyThreadState
Try special casing the destruction on the same thread.
* style: pre-commit fixes
* Missed a rename in a ifdef block
* I think this test is causing problems in 3.12, so try ifdefing it to see if the problems go away.
* style: pre-commit fixes
* Document the 3.12 constraints with a warning
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ci: add cpptest to the clang-tidy job
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* noexcept move operations
* Update include/pybind11/subinterpreter.h
std::memset
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* Move common code from PYBIND11_MODULE and PYBIND11_EMBEDDED_MODULE into one macro
The difference between the two is really small, almost all of the code was the same.
* Use a macro to make the forward decl correct and reduce code duplication
* Oops, extra semicolon
* Split up the macro into two so we can avoid the redundant decl
* Fix Python 3.8 type hints and add module prefix
These type hints are invalid in Python 3.8.
Add `typing.` prefix to remove ambiguity.
* style: pre-commit fixes
* Add module prefix to Union
* Rename macros
* Improve comment
* Comment out 3.8 type hint macros
Fixing this issue in Python 3.8 will require updating lots of tests. This can be added in a further pull request.
* Add Iterable module prefix
* Add module prefix to Iterator
* Add module prefix to Callable
* Re-add accidentally deleted brackets
* Add module prefix to Optional
* Add module prefix to Final
* Add module prefix to ClassVar
* Add module prefix to TypeGuard
* Add module prefix to TypeIs
* Add module prefix to NoReturn
* Add module prefix to Never
* Add module prefix to Literal
* Add module prefix to Callable
* Add module prefix to Sequence
* Add module prefix to Iterator
* style: pre-commit fixes
* Remove type hint macros
* style: pre-commit fixes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Squashed prepv300/manuscript — 30b9c268aeb98308ea42aaccfd5fe454e173c6fc — 2025-03-30 14:56:03 -0700 [skip ci]
[Browse prepv300/manuscript tree](30b9c268ae)
[Browse prepv300/manuscript commits](30b9c268ae/)
* docs: update changelog
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* docs: upgrade guide CMake suggestions
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Explain type_caster_enum_type_enabled, copyable_holder_caster_shared_ptr_with_smart_holder_support_enabled, move_only_holder_caster_unique_ptr_with_smart_holder_support_enabled in Upgrade guide.
* Add a small section for py::bind_vector, py::bind_map & py::smart_holder
* Fix tiny oversight: Reference back to the current release v2.13 (not v2.12)
* Remove sentence: Using self._pybind11_conduit_v1_() ... should keep extension compatibility.
This isn't true, because we also modernized `PYBIND11_PLATFORM_ABI_ID`
(which I believe was absolutely necessary). I think it'll be too complicated
to explain that here, and there is a mention in the Upgrade guide.
* Changelog: combine #4953 and #5439
* Trivial whitespace/formatting fixes/enhancements.
* chore: add more to deprecation page
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* docs: update for recent additions
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* docs: fixes and set rc1 version
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: support rc versions
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Undo erroneous copilot change: We need to use `detail::enable_if_t`, for compatibility with C++11 and C++14.
* Empty lines cleanup.
* Rewording of "CMake support now defaults to ..." paragraph.
* Add missing backticks in upgrade guide.
* Try :ref:deprecated instead of :doc:deprecated
* docs: last bit of polish
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Piggy-back trivial whitespace cleanup that was missed in PR #5669
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
* Move embedded modules to multiphase init
So that they too can support multi-interpreter and nogil tags
* Update the multiple interpreter test for embedded module changes
* Add a note to embedded module docs about the new tags
* Oops, missed a warning pop
* Remove unused variable
* Update ci.yml
* Fix this embedded GIL test for free-threading
* Oops, need to use ptr() here
* This test created a subinterpreter when PYBIND11_SUBINTERPRETER_SUPPORT was off
So the fix is really this test should not be run in these older versions at all.
The hang was a GIL issue between the subinterpreters during pybind11::exception::what().
* fix: standard mutex for 3.13t
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* ci: support Python 3.14
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: Python 3.14 name change
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: fix expected output to handle Python 3.14
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: tighten CLI and add color on 3.14+
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: ignore failure on 3.14.0b1
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: support Python 3.14.0b1 with interperters
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update test_multiple_interpreters.py
* Update test_multiple_interpreters.py
* fix: new breakage for 3.14 fixed
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: handle empty annotations 3.14
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: Python 3.14 may not create the annotations dict
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: use PyUnstable_IsImmortal
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: use sys._is_immortal
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* tests: ignore large values for refcount too
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* style: pre-commit fixes
* ci: enable all free-threaded builds
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: patch for embed
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Revert "fix: patch for embed"
This reverts commit c4226a0671.
* ci: drop new 3.xt additions
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* fix: logic issue, also add some comments
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update include/pybind11/pytypes.h
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add documentation (advanced/misc) for mod_gil_not_used and multiple_interpreters options
* Add some comparison of free-threading vs sub-interpreters
* Formatting
* Reword for clarity
* style: pre-commit fixes
* Suggested changes, including a long-winded explaination of parallelism in python today
* Slight formatting updates and GPT suggested clarifications
* Better section title
* Update docs/advanced/misc.rst
Copilot suggestion
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update docs/advanced/misc.rst
Copilot suggestion
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update docs/advanced/misc.rst
* Add a deadlock warning, and minor corrections
* Rewrite the example to avoid free-threading risky behaviors
* Oops, missed a line in the example
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Fix Buffer type hint
collections.abc.Buffer was added in Python 3.12.
The previous behaviour should be used prior to this version.
* Fix comment
* Fix indentation
* style: pre-commit fixes
* Fix test
* Add missing import
* style: pre-commit fixes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Allow per-interpreter internals/local_internals
* Significant rewrite to avoid using thread_locals as much as possible.
Since we can avoid them by checking this atomic, the cmake config conditional shouldn't be necessary.
The slower path (with thread_locals and extra checks) only comes in when a second interpreter is actually instanciated.
* Add a test for per-interpreter GIL
Uses two extra threads to demonstrate that neither shares a GIL.
* Fix for nonconforming std::atomic constructors on some compilers
* style: pre-commit fixes
* Fix initializer to make MSVC happy.
* Switch to gil_scoped_acquire_simple, get rid of old copy of it from internals.h
* Use the PyThreadState's interp member rather than the thread state itself.
* Be more explicit about the type of the internalspp
* Suggested renamings and rewordings
* Rename find_internals_pp and change it to take in the state dict reference
* Use the old raise_from instead of pybind11_fail
* Move most of the internals initialization into its constructor.
* Move round_up_to_next_pow2 function upwards
* Remove redundant forward decl
* Add a python-driven subinterpreter test
* Disable the python subinterpreter test on emscripten
Can't load the native-built cpp modules.
* Switch the internals pointer pointer to a unique_ptr pointer
* Spelling
* Fix clang-tidy warning, compare pointer to nullptr
* Rename get_interpreter_counter to get_num_interpreters_seen
* Try simplifying the test's cmake set_target_properties
* Replace mod_* tags with a single tag w/enum
Update tests accordingly
* Add a test for shared-GIL (legacy) subinterpreters
* Update test to work around differences in the various versions of interpreters modules
* Fix unused parameter
* Rename tests and associated test modules.
* Switch get_internals_pp to a template function
* Rename curtstate to cur_tstate
* refactor: use simpler names
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* style: pre-commit fixes
* fix: return class, not enum
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Have to join these threads to make sure they are totally done before the test returns.
* Wrap module_def initialization in a static so it only happens once.
If it happens concurrently in multiple threads, badness ensues....
* style: pre-commit fixes
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
* Added weak_ptr test (currently failing)
* Cleanup
* [skip ci] Simplify test case.
* Add test_class_sp_trampoline_weak_ptr.cpp,py (using std::shared_ptr as holder). Tweak test_class_sh_trampoline_weak_ptr.py to pass, with `# THIS NEEDS FIXING` comment.
* Resolve clang-tidy errors
```
/__w/pybind11/pybind11/tests/test_class_sh_trampoline_weak_ptr.cpp:23:43: error: the parameter 'sp' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param,-warnings-as-errors]
23 | void set_wp(std::shared_ptr<VirtBase> sp) { wp = sp; }
| ^
| const &
4443 warnings generated.
```
```
/__w/pybind11/pybind11/tests/test_class_sp_trampoline_weak_ptr.cpp:23:43: error: the parameter 'sp' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param,-warnings-as-errors]
23 | void set_wp(std::shared_ptr<VirtBase> sp) { wp = sp; }
| ^
| const &
4430 warnings generated.
```
* PYPY, GRAALPY: skip last part of test_weak_ptr_base
* Rename test_weak_ptr_base → test_weak_ptr_owner
* Add SpOwner, test_with_sp_owner, test_with_sp_and_wp_owners
* Modify py::trampoline_self_life_support semantics: if trampoline class does not inherit from this class, preserve established Inheritance Slicing behavior.
rwgk reached this point with the help of ChatGPT:
* https://chatgpt.com/share/68056498-7d94-8008-8ff0-232e2aba451c
The only production code change in this commit is:
```
diff --git a/include/pybind11/detail/type_caster_base.h b/include/pybind11/detail/type_caster_base.h
index d4f9a41e..f3d45301 100644
--- a/include/pybind11/detail/type_caster_base.h
+++ b/include/pybind11/detail/type_caster_base.h
@@ -776,6 +776,14 @@ struct load_helper : value_and_holder_helper {
if (released_ptr) {
return std::shared_ptr<T>(released_ptr, type_raw_ptr);
}
+ auto *self_life_support
+ = dynamic_raw_ptr_cast_if_possible<trampoline_self_life_support>(type_raw_ptr);
+ if (self_life_support == nullptr) {
+ std::shared_ptr<void> void_shd_ptr = hld.template as_shared_ptr<void>();
+ std::shared_ptr<T> to_be_released(void_shd_ptr, type_raw_ptr);
+ vptr_gd_ptr->released_ptr = to_be_released;
+ return to_be_released;
+ }
std::shared_ptr<T> to_be_released(
type_raw_ptr, shared_ptr_trampoline_self_life_support(loaded_v_h.inst));
vptr_gd_ptr->released_ptr = to_be_released;
```
* Remove debug printf in include/pybind11/pybind11.h
* Resolve MSVC error
```
11>D:\a\pybind11\pybind11\tests\test_class_sp_trampoline_weak_ptr.cpp(44,50): error C2220: the following warning is treated as an error [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
11>D:\a\pybind11\pybind11\tests\test_class_sp_trampoline_weak_ptr.cpp(44,50): warning C4458: declaration of 'sp' hides class member [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
D:\a\pybind11\pybind11\tests\test_class_sp_trampoline_weak_ptr.cpp(54,31):
see declaration of 'pybind11_tests::class_sp_trampoline_weak_ptr::SpOwner::sp'
```
* [skip ci] Undo the production code change under 4638e017b6
Also undo the corresponding test change in test_class_sh_trampoline_weak_ptr.py
But keep all extra debugging code for now.
* [skip ci] Introduce lambda in `WpOwner::set_wp` bindings, but simply cast to `std::shared_ptr<VirtBase>` for now.
* Add `py::potentially_slicing_shared_ptr()`
* Add `type_id<T>()` to `py::potentially_slicing_shared_ptr()` error message and add test.
* test_potentially_slicing_shared_ptr.cpp,py (for smart_holder only)
* Generalize test_potentially_slicing_shared_ptr.cpp,py for testing with smart_holder and std::shared_ptr as holder.
* Add back test_potentially_slicing_shared_ptr_not_convertible_error(), it got lost accidentally in commit 56d23dc478
* Add simple trampoline state assertions.
* Resolve clang-tidy errors.
```
/__w/pybind11/pybind11/tests/test_potentially_slicing_shared_ptr.cpp:30:9: error: 'magic_token' should be initialized in a member initializer of the constructor [cppcoreguidelines-prefer-member-initializer,-warnings-as-errors]
29 | trampoline_is_alive_simple(const trampoline_is_alive_simple &other) {
| : magic_token(other.magic_token)
30 | magic_token = other.magic_token;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/__w/pybind11/pybind11/tests/test_potentially_slicing_shared_ptr.cpp:33:9: error: 'magic_token' should be initialized in a member initializer of the constructor [cppcoreguidelines-prefer-member-initializer,-warnings-as-errors]
32 | trampoline_is_alive_simple(trampoline_is_alive_simple &&other) noexcept {
| : magic_token(other.magic_token)
33 | magic_token = other.magic_token;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```
* Add a (long) C++ comment for py::potentially_slicing_shared_ptr<>()
* [skip ci] Add new "Avoiding Inheritance Slicing and ``std::weak_ptr`` surprises" section in advanced/classes.rst
* [skip ci] Add introductory comment to test_potentially_slicing_shared_ptr.py
* Minimal (!) changes to have py::potentially_slicing_weak_ptr<T>(handle) as the public API. For CI testing, before changing the names around more widely, and the documentation.
* Rename test_potentially_slicing_shared_ptr.cpp,py → test_potentially_slicing_weak_ptr.cpp,py
* Update docs/advanced/classes.rst and C++ comments → potentially_slicing_weak_ptr
* Write "shared_ptr" instead of just "pointer" in a couple places in docs/advanced/classes.rst
* Add comment for force_potentially_slicing_shared_ptr in type_caster_base.h, to make a direct connection py::potentially_slicing_weak_ptr
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Strictly enforce: trampoline must inherit from trampoline_self_life_support when used in combination with smart_holder
* Simplify test_class_sh_trampoline_basic.cpp,py (only one Abase is needed now)
* Replace obsolete sophisticated `throw value_error()` with a simple `assert()`
* Strictly enforce: trampoline should inherit from trampoline_self_life_support only if used in combination with smart_holder
* Resolve clang-tidy error
```
/__w/pybind11/pybind11/tests/test_class_sh_trampoline_basic.cpp:35:46: error: the parameter 'obj' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param,-warnings-as-errors]
35 | int AddInCppSharedPtr(std::shared_ptr<Abase> obj, int other_val) {
| ^
| const &
```
* Disable new static_assert if PYBIND11_RUN_TESTING_WITH_SMART_HOLDER_AS_DEFAULT_BUT_NEVER_USE_IN_PRODUCTION_PLEASE is defined.
* CI: Fail on any warnings with clang 18
* CI: Fail on any warnings with gcc 13
* Fix cmake's try_compile warning
* Guard redundant declarations
* ci.yml: fix syntax error
* Use PYBIND11_WARNING macros
* Fix more redundant declarations
... introduced with merge of smart_holder branch
* fix missing pythonic type hints for native_enum
* Fix __qualname__ in native_enum
* Rename `enum class native` → `func_sig_rendering`. Add to `ENUM_TYPES_AND_MEMBERS`. Move new code around to fit in more organically.
"native" is used in so many places here, it would be very difficult to pin-point where the specific enum type is used.
Similarly, "value" is difficult to pin-point. Changed to "nested_value".
---------
Co-authored-by: Bryn Lloyd <12702862+dyollb@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Updated STL type hints use support collections.abc
* Updated array_caster to match numpy/eigen typing.Annotated stlye
* Added support for Mapping, Set and Sequence derived from collections.abc.
* Fixed merge of typing.SupportsInt in new tests
* Integrated collections.abc checks into convertible check functions.
* Changed type hint of py::buffer to collections.abc.Buffer
* Changed convertible check function names
* Added comments to convertible check functions
* Removed checks for methods that are already required by the abstract base class
* Improved mapping caster test using more compact a1b2c3 variable
* Renamed and refactored sequence, mapping and set test classes to reuse implementation
* Added tests for mapping and set casters for noconvert mode
* Added tests for sequence caster for noconvert mode
* ci: speed up ci a bit by thinning out matrix
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Update configure.yml
* Update configure.yml
* Update configure.yml
* ci: drop third build
* ci: fix minor issues
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* ci: trim a bit more
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
