* fix: strdup args added after initialize_generic in def_property_static (gh-5976)
`def_property_static` calls `process_attributes::init` on already-initialized
function records (after `initialize_generic`'s strdup loop has run).
Args added at this stage (e.g. "self" via `append_self_arg_if_needed`) remain
as string literals, so `destruct()` would call `free()` on them.
Fix by strdup'ing name/descr of any args appended by the late
`process_attributes::init` call. Root cause introduced by gh-5486.
Made-with: Cursor
* Partially revert gh-6010: remove py_is_finalizing() workarounds
Now that the root cause (free of string literals in def_property_static,
gh-5976) is fixed in the previous commit, the py_is_finalizing() guards
introduced in gh-6010 are no longer needed:
- tp_dealloc_impl: remove early return during finalization (was leaking
all function records instead of properly destroying them)
- destruct(): remove guard around arg.value.dec_ref()
- common.h: remove py_is_finalizing() helper (no remaining callers)
The genuine fix from gh-6010 (PyObject_Free + Py_DECREF ordering in
tp_dealloc_impl) is retained.
Made-with: Cursor
* test: add embedding test for py::enum_ across interpreter restart (gh-5976)
py::enum_ is the primary trigger for gh-5976 because its constructor
creates properties via def_property_static / def_property_readonly_static,
which call process_attributes::init on already-initialized function records.
Yet none of the existing embedding tests used py::enum_ at all.
Add an PYBIND11_EMBEDDED_MODULE with py::enum_ and a test case that imports
it, finalize/reinitializes the interpreter, and re-imports it. This exercises
the def_property_static code path that was fixed in the preceding commit.
Note: on Python 3.14.2 (and likely 3.12+), tp_dealloc_impl is not called
during Py_FinalizeEx for function record PyObjects — they simply leak because
types are effectively immortalized. As a result, this test cannot trigger the
original free()-on-string-literal crash on this Python version. However, it
remains valuable as a regression guard: on Python builds where finalization
does clean up function records (or if CPython changes this behavior), the
test would catch the crash. It also verifies that py::enum_ survives
interpreter restart correctly, which was previously untested.
Made-with: Cursor
* test: skip enum restart test on Python 3.12 (pre-existing crash)
Made-with: Cursor
* Add test_standalone_enum_module.py, standalone_enum_module.cpp
* Make standalone_enum_module.cpp more similar to #5976 reproducer. Also fix clang-tidy error.
* This crashes when testing locally:
( cd /wrk/forked/pybind11/tests && PYTHONPATH=/wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/lib /wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/TestVenv/bin/python3 -m pytest test_standalone_enum_module.py )
============================= test session starts ==============================
platform linux -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0
installed packages of interest: build==1.4.2 numpy==2.4.3 scipy==1.17.1
C++ Info: 13.3.0 C++20 __pybind11_internals_v12_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1__ PYBIND11_SIMPLE_GIL_MANAGEMENT=False
rootdir: /wrk/forked/pybind11/tests
configfile: pytest.ini
plugins: timeout-2.4.0, xdist-3.8.0
collected 1 item
test_standalone_enum_module.py F [100%]
=================================== FAILURES ===================================
________________________ test_enum_import_exit_no_crash ________________________
def test_enum_import_exit_no_crash():
# Modeled after reproducer under issue #5976
> env.check_script_success_in_subprocess(
f"""
import sys
sys.path.insert(0, {os.path.dirname(env.__file__)!r})
import standalone_enum_module as m
assert m.SomeEnum.__class__.__name__ == "pybind11_type"
""",
rerun=1,
)
test_standalone_enum_module.py:10:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"'
def check_script_success_in_subprocess(code: str, *, rerun: int = 8) -> None:
"""Runs the given code in a subprocess."""
import os
import subprocess
import sys
import textwrap
if ANDROID or IOS or sys.platform.startswith("emscripten"):
pytest.skip("Requires subprocess support")
code = textwrap.dedent(code).strip()
try:
for _ in range(rerun): # run flakily failing test multiple times
subprocess.check_output(
[sys.executable, "-c", code],
cwd=os.getcwd(),
stderr=subprocess.STDOUT,
text=True,
)
except subprocess.CalledProcessError as ex:
> raise RuntimeError(
f"Subprocess failed with exit code {ex.returncode}.\n\n"
f"Code:\n"
f"```python\n"
f"{code}\n"
f"```\n\n"
f"Output:\n"
f"{ex.output}"
) from None
E RuntimeError: Subprocess failed with exit code -6.
E
E Code:
E ```python
E import sys
E sys.path.insert(0, '/wrk/forked/pybind11/tests')
E import standalone_enum_module as m
E assert m.SomeEnum.__class__.__name__ == "pybind11_type"
E ```
E
E Output:
E munmap_chunk(): invalid pointer
_ = 0
code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"'
os = <module 'os' (frozen)>
rerun = 1
subprocess = <module 'subprocess' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/subprocess.py'>
sys = <module 'sys' (built-in)>
textwrap = <module 'textwrap' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/textwrap.py'>
env.py:68: RuntimeError
=========================== short test summary info ============================
FAILED test_standalone_enum_module.py::test_enum_import_exit_no_crash - Runti...
============================== 1 failed in 0.23s ===============================
ERROR: completed_process.returncode=1
* Add "Added in PR #6015" comments, for easy reference back to this PR
* test: use PYBIND11_CATCH2_SKIP_IF for Python 3.12 enum restart skip
Replace #if/#else/#endif preprocessor guard with runtime
PYBIND11_CATCH2_SKIP_IF so the test is always compiled and
shows [ SKIPPED ] in output on Python 3.12.
Made-with: Cursor
* fix: suppress MSVC C4127 in PYBIND11_CATCH2_SKIP_IF macro
The constant condition in PYBIND11_CATCH2_SKIP_IF triggers MSVC
warning C4127 (conditional expression is constant), which becomes
a build error under /WX.
Made-with: Cursor
Replace `static thread_specific_storage<int>` with `thread_local bool`
in the implicit conversion reentrancy guard. Since implicitly_convertible
is a template function, each unique <InputType, OutputType> pair created
its own TSS key via PyThread_tss_create(). Projects with hundreds of
modules and many implicit conversions could exhaust PTHREAD_KEYS_MAX
(1024 on Linux, 512 on macOS), especially on Python 3.12+ where CPython
itself consumes more TSS keys for subinterpreter support.
thread_local bool is safe here because:
- bool is trivially destructible, so it works on all C++11 platforms
including older macOS (the concern that motivated the TSS approach in
PR #5777 applied only to types with non-trivial destructors needing
__cxa_thread_atexit runtime support)
- Each thread gets its own copy, so it is thread-safe for free-threading
- Subinterpreter sharing is benign: the guard prevents recursive implicit
conversions on the same thread regardless of which interpreter is active
- The v3.0.0 code already used thread_local bool under Py_GIL_DISABLED
This effectively reverts the core change from PR #5777 while keeping
the non-copyable/non-movable set_flag guard.
Made-with: Cursor
* Strip noexcept from cpp17 function type bindings
* Fix a bug and increase test coverage
* Does this fix it?
* Silence clang-tidy issue
* Simplify method adapter with macro and add missing rvalue adaptors + tests
* Supress clang-tidy errors
* Improve test coverage
* Add additional static assert
* Try to resolve MSVC C4003 warning
* Simplify method adaptor into 2 template instatiations with enable_if_t
* Fix ambiguous STL template
* Close remaining qualifier consistency gaps for member pointer bindings.
A production-code review after #2234 showed that ref-qualified member pointers were still inconsistently handled across def_buffer, vectorize, and overload_cast, so this adds the missing overloads with focused tests for each newly-supported signature.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Clarify why def_buffer/vectorize omit rvalue-qualified overloads.
These comments were added while reviewing the qualifier coverage follow-up, to document that buffer/vectorized calls operate on existing Python-owned instances and should not move-from self.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add compile-only overload_cast guard for ref-qualified methods.
This was added as a maintenance follow-up to the qualifier-consistency work, so future changes that introduce overload_cast ambiguity or wrong ref/noexcept resolution fail at compile time.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Refactor overload_cast_impl qualifier overloads with a macro.
As part of the qualifier-consistency maintenance follow-up, this reduces duplication in overload_cast_impl while preserving the same ref/noexcept coverage and keeping pedantic-clean macro expansion.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Expose __cpp_noexcept_function_type to Python tests and use explicit skip guards.
This replaces hasattr-based optional assertions with skipif-gated noexcept-only tests so skipped coverage is visible in pytest output while keeping non-noexcept checks always active.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Add static_assert in method_adaptor to guard that T is a member function pointer.
Suggested by @Skylion007 in PR #5992 review comment [T007].
Made-with: Cursor
* automatic clang-format change (because of #6002)
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* gh-5991: Fix segfault during finalization related to function_record
This patch was developed with assistance from Claude Code Opus 4.6
Here's Claude's explanation of the crash mechanism and some reasoning for the difficulty to repro:
`tp_dealloc_impl` calls `cpp_function::destruct` which:
1. Calls `std::free()` on function_record string members (`name`, `doc`, `signature`)
2. Calls `arg.value.dec_ref()` on default argument values
3. Calls `delete rec` on the function_record
But it never calls `PyObject_Free(self)` or `Py_DECREF(Py_TYPE(self))`, which are
required for heap types.
During `_Py_Finalize`, final GC collects the heap types (which survive module dict
clearing via `tp_mro` self-references). This triggers a massive cascade:
`type_dealloc → property_dealloc → meth_dealloc → tp_dealloc_impl → destruct`.
At scale (~1,200+ function_records), the volume of `delete`/`free` calls corrupts
heap metadata, causing subsequent `std::free()` to receive garbage pointers → SEGV.
* Add detail::py_is_finalizing() wrapper to deduplicate version-guarded #ifdef blocks
Also fixes clang-tidy readability-implicit-bool-conversion warnings.
Made-with: Cursor
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Add failback implementation of `PyCriticalSection_BeginMutex` for Python 3.13t
* Add comment for Python version
* Use `_PyCriticalSection_BeginSlow`
* Add forward declaration
* Fix forward declaration
* Remove always true condition `defined(PY_VERSION_HEX)`
* Detect musllinux
* Add manylinux test
* Use direct mutex locking for Python 3.13t
`_PyCriticalSection_BeginSlow` is a private CPython function not exported
on Linux. For Python < 3.14.0rc1, use direct `mutex.lock()`/`mutex.unlock()`
instead of critical section APIs.
* Empty commit to trigger CI
* Empty commit to trigger CI
* Empty commit to trigger CI
* Run apt update before apt install
* Remove unnecessary prefix
* Add manylinux test with Python 3.13t
* Simplify pycritical_section with std::unique_lock fallback for Python < 3.14
* Fix potential deadlock in make_iterator_impl for Python 3.13t
Refactor pycritical_section into a unified class with internal version
checks instead of using a type alias fallback. Skip locking in
make_iterator_impl for Python < 3.14.0rc1 to avoid deadlock during
type registration, as pycritical_section cannot release the mutex
during Python callbacks without PyCriticalSection_BeginMutex.
* Add reference for xfail message
* Fix race condition with py::make_key_iterator in free threading
The creation of the iterator class needs to be synchronized.
* style: pre-commit fixes
* Use PyCriticalSection_BeginMutex instead of recursive mutex
* style: pre-commit fixes
* Make pycritical_section non-copyable and non-movable
The pycritical_section class is a RAII wrapper that manages a Python
critical section lifecycle:
- Acquires the critical section in the constructor via
PyCriticalSection_BeginMutex
- Releases it in the destructor via PyCriticalSection_End
- Holds a reference to a pymutex
Allowing copy or move operations would be dangerous:
1. Copy: Both the original and copied objects would call
PyCriticalSection_End on the same PyCriticalSection object in their
destructors, leading to double-unlock and undefined behavior.
2. Move: The moved-from object's destructor would still run and attempt
to end the critical section, while the moved-to object would also try
to end it, again causing double-unlock.
This follows the same pattern used by other RAII lock guards in the
codebase, such as gil_scoped_acquire and gil_scoped_release, which also
explicitly delete copy/move operations to prevent similar issues.
By explicitly deleting these operations, we prevent accidental misuse
and ensure the critical section is properly managed by a single RAII
object throughout its lifetime.
* Drop Python 3.13t support from CI
Python 3.13t was experimental, while Python 3.14t is not. This PR
uses PyCriticalSection_BeginMutex which is only available in Python
3.14+, making Python 3.13t incompatible with the changes.
Removed all Python 3.13t CI jobs:
- ubuntu-latest, 3.13t (standard-large matrix)
- macos-15-intel, 3.13t (standard-large matrix)
- windows-latest, 3.13t (standard-large matrix)
- manylinux job testing 3.13t
This aligns with the decision to drop Python 3.13t support as
discussed in PR #5971.
* Add Python 3.13 (default) replacement jobs for removed 3.13t jobs
After removing Python 3.13t support (incompatible with PyCriticalSection_BeginMutex
which requires Python 3.14+), we're adding replacement jobs using Python 3.13
(default) to maintain test coverage in key dimensions:
1. ubuntu-latest, Python 3.13: C++20 + DISABLE_HANDLE_TYPE_NAME_DEFAULT_IMPLEMENTATION
- Replaces: ubuntu-latest, 3.13t with same config
- Maintains coverage for this specific configuration combination
2. macos-15-intel, Python 3.13: C++11
- Replaces: macos-15-intel, 3.13t with same config
- Maintains macOS coverage for Python 3.13
3. manylinux (musllinux), Python 3.13: GIL testing
- Replaces: manylinux, 3.13t job
- Maintains manylinux/musllinux container testing coverage
These additions are proposed to get feedback on which jobs should be kept
to maintain appropriate test coverage without the experimental 3.13t builds.
* ci: run in free-threading mode a bit more on 3.14
* Revert "ci: run in free-threading mode a bit more on 3.14"
This reverts commit 91189c9242.
Reason: https://github.com/pybind/pybind11/pull/5971#issuecomment-3831321903
* Reapply "ci: run in free-threading mode a bit more on 3.14"
This reverts commit f3197de975.
After #5972 is/was merged, tests should pass (already tested under #5980).
See also https://github.com/pybind/pybind11/pull/5972#discussion_r2752674989
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
* Appease MSVC Warning C4866: compiler may not enforce left-to-right evaluation order
* Remove const qualifier
* Reword comment to be self-explanatory without PR context
The previous comment referenced the MSVC warning but didn't explain
why the code is structured as two statements. The revised comment
clarifies the intent: fetching the value first ensures well-defined
evaluation order.
* chore(deps): switch typos to mirror repo
Switch from crate-ci/typos to adhtruong/mirrors-typos because
pre-commit autoupdate confuses tags in the upstream repo, selecting
the mutable `v1` tag instead of pinned versions like `v1.41.0`.
See https://github.com/crate-ci/typos/issues/390
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Make argument_vector re-usable for other types.
* Attempt to collect args into array for vectorcall
* Revert "Attempt to collect args into array for vectorcall"
This reverts commit 418a034195.
* Implement vectorcall args collector
* pre-commit fixes
* Checkpoint in moving to METH_FASTCALL
* pre-commit fixes
* Use the names tuple directly, cleaner code and less reference counting
* Fix unit test, the code now holds more references
It cannot re-use the incoming tuple as before, because it is no longer a tuple at all. So a new tuple must be created, which then holds references for each member.
* Make clangtidy happy
* Oops, _v is C++14
* style: pre-commit fixes
* Minor code cleanup
* Fix signed conversions
* Fix args expansion
This would be easier with `if constexpr`
* style: pre-commit fixes
* Code cleanup
* fix(tests): Install multiple-interpreter test modules into wheel
The `mod_per_interpreter_gil`, `mod_shared_interpreter_gil`, and
`mod_per_interpreter_gil_with_singleton` modules were being built
but not installed into the wheel when using scikit-build-core
(SKBUILD=true). This caused iOS (and potentially Android) CIBW
tests to fail with ModuleNotFoundError.
Root cause analysis:
- The main test targets have install() commands (line 531)
- The PYBIND11_MULTIPLE_INTERPRETERS_TEST_MODULES were missing
equivalent install() commands
- For regular CMake builds, this wasn't a problem because
LIBRARY_OUTPUT_DIRECTORY places the modules next to pybind11_tests
- For wheel builds, only targets with explicit install() commands
are included in the wheel
This issue was latent until commit fee2527d changed the test imports
from `pytest.importorskip()` (graceful skip) to direct `import`
statements (hard failure), which exposed the missing modules.
Failing tests:
- test_multiple_interpreters.py::test_independent_subinterpreters
- test_multiple_interpreters.py::test_dependent_subinterpreters
Error: ModuleNotFoundError: No module named 'mod_per_interpreter_gil'
* tests: Pin numpy 2.4.0 for Python 3.14 CI tests
Add numpy==2.4.0 requirement for Python 3.14 (both default and
free-threaded builds). NumPy 2.4.0 is the first version to provide
official PyPI wheels for Python 3.14:
- numpy-2.4.0-cp314-cp314-manylinux_2_27_x86_64...whl (default)
- numpy-2.4.0-cp314-cp314t-manylinux_2_27_x86_64...whl (free-threaded)
Previously, CI was skipping all numpy-dependent tests for Python 3.14
because PIP_ONLY_BINARY was set and no wheels were available:
SKIPPED [...] test_numpy_array.py:8: could not import 'numpy':
No module named 'numpy'
With this change, the full numpy test suite will run on Python 3.14,
providing better test coverage for the newest Python version.
Note: Using exact pin (==2.4.0) rather than compatible release (~=2.4.0)
to ensure reproducible CI results with the first known-working version.
* tests: Add verbose flag to CIBW pytest command
Add `-v` to the pytest command in tests/pyproject.toml to help
diagnose hanging tests in CIBW jobs (particularly iOS).
This will show each test name as it runs, making it easier to
identify which specific test is hanging.
* tests: Skip subinterpreter tests on iOS, add pytest timeout
- Add `IOS` platform constant to `tests/env.py` for consistency with
existing `ANDROID`, `LINUX`, `MACOS`, `WIN`, `FREEBSD` constants.
- Skip `test_multiple_interpreters.py` module on iOS. Subinterpreters
are not supported in the iOS simulator environment. These tests were
previously skipped implicitly because the modules weren't installed
in the wheel; now that they are (commit 6ed6d5a8), we need an
explicit skip.
- Change pytest timeout from 0 (disabled) to 120 seconds. This provides
a safety net to catch hanging tests before the CI job times out after
hours. Normal test runs complete in 33-55 seconds total (~1100 tests),
so 120 seconds per test is very generous.
- Add `-v` flag for verbose output to help diagnose any future issues.
* More cleanups in argument vector, per comments.
* Per Cursor, move all versions to Vectorcall since it has been supported since 3.8.
This means getting rid of simple_collector, we can do the same with a constexpr if in the unpacking_collector.
* Switch to a bool vec for the used_kwargs flag...
This makes more sense and saves a sort, and the small_vector implementation means it will actually take less space than a vector of size_t elements.
The most common case is that all kwargs are used.
* Fix signedness for clang
* Another signedness issue
* tests: Disable pytest-timeout for Pyodide (no signal.setitimer)
Pyodide runs in a WebAssembly sandbox without POSIX signals, so
`signal.setitimer` is not available. This causes pytest-timeout to
crash with `AttributeError: module 'signal' has no attribute 'setitimer'`
when timeout > 0.
Override the test-command for Pyodide to keep timeout=0 (disabled).
* Combine temp storage and args into one vector
It's a good bit faster at the cost of this one scary reinterpret_cast.
* Phrasing
* Delete incorrect comment
At 6, the struct is 144 bytes (not 128 bytes as the comment said).
* Fix push_back
* Update push_back in argument_vector.h
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* style: pre-commit fixes
* Use real types for these instead of object
They can be null if you "steal" a null handle.
* refactor: Replace small_vector<object> with ref_small_vector for explicit ownership
Introduce `ref_small_vector` to manage PyObject* references in `unpacking_collector`,
replacing the previous `small_vector<object>` approach.
Primary goals:
1. **Maintainability**: The previous implementation relied on
`sizeof(object) == sizeof(PyObject*)` and used a reinterpret_cast to
pass the object array to vectorcall. This coupling to py::object's
internal layout could break if someone adds a debug field or other
member to py::handle/py::object in the future.
2. **Readability**: The new `push_back_steal()` vs `push_back_borrow()`
API makes reference counting intent explicit at each call site,
rather than relying on implicit py::object semantics.
3. **Intuitive code**: Storing `PyObject*` directly and passing it to
`_PyObject_Vectorcall` without casts is straightforward and matches
what the C API expects. No "scary" reinterpret_cast needed.
Additional benefits:
- `PyObject*` is trivially copyable, simplifying vector operations
- Batch decref in destructor (tight loop vs N individual object destructors)
- Self-documenting ownership semantics
Design consideration: We considered folding the ref-counting functionality
directly into `small_vector` via template specialization for `PyObject*`.
We decided against this because:
- It would give `small_vector<PyObject*, N>` a different interface than the
generic `small_vector<T, N>` (steal/borrow vs push_back)
- Someone might want a non-ref-counting `small_vector<PyObject*, N>`
- The specialization behavior could surprise users expecting uniform semantics
A separate `ref_small_vector` type makes the ref-counting behavior explicit
and self-documenting, while keeping `small_vector` generic and predictable.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
* Don't allow keep_alive or call_guard on properties
The def_property family blindly ignore the keep_alive and call_guard arguments passed to them making them confusing to use.
This adds a static_assert if either is passed to make it clear it doesn't work.
I would prefer this to be a compiler warning but I can't find a way to do that. Is that even possible?
* style: pre-commit fixes
* Re-run tests
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Type hint make_tuple / fix *args/**kwargs return type
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* add back commented out panic
* ignore return std move clang
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* fix for mingmw
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* added missing case
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
---------
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* Centralize readable function signatures to avoid duplication
This seems to reduce size costs of adding enum_-specific implementations of dunder methods, but also should provide a nice to have size optimization for programs that use pybind11 in general.
* gate disabling of -Wdeprecated-redundant-constexpr-static-def to clang 17+
* fix gating to include Apple Clang 15
* Make GCC happy with types
* fix apple clang gating again. suppress -Wdeprecated for GCC
* Gate warning suppressions to C++17. Suppress -Wdeprecated for clang as well.
* hopefully fix last straggler CI job
* attempt to address readability review feedback from @rwgk
* drop warning suppressions and instead just gate compilation the pre-C++17 compat code
* Add fast_type_map, use it authoritatively for local types and as a hint for global types
nanobind has a similar two-level lookup strategy, added and explained
by
b515b1f7f2
In this PR I've ported this approach to pybind11. To avoid an ABI
break, I've kept the fast maps to the `local_internals`. I think this
should be safe because any particular module should see its
`local_internals` reset at least as often as the global `internals`,
and misses in the fast "hint" map for global types fall back to the
global `internals`.
Performance seems to have improved. Using my patched fork of
pybind11_benchmark
(https://github.com/swolchok/pybind11_benchmark/tree/benchmark-updates,
specifically commit hash b6613d12607104d547b1c10a8145d1b3e9937266), I
run bench.py and observe the MyInt case. Each time, I do 3 runs and
just report all 3.
master, Mac: 75.9, 76.9, 75.3 nsec/loop
this PR, Mac: 73.8, 73.8, 73.6 nsec/loop
master, Linux box: 188, 187, 188 nsec/loop
this PR, Linux box: 164, 165, 164 nsec/loop
Note that the "real" percentage improvement is larger than implied by the
above because master does not yet include #5824.
* simplify unsafe_reset_local_internals in test
* pre-implement PYBIND11_INTERNALS_VERSION 12
* use PYBIND11_INTERNALS_VERSION 12 on Python 3.14 per suggestion
* Implement reviewer comments: revert PY_VERSION_HEX change, fix REVIEW comment, add two-level lookup comments. ci.yml coming separately
* Use the inplace build to smoke test ABI bump?
* [skip ci] Remove "smoke" from comment. This is full testing, just only on a few platforms.
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Avoid heap allocation for function calls with a small number of arguments
We don't have access to llvm::SmallVector or similar, but given the
limited subset of the `std::vector` API that
`function_call::args{,_convert}` need and the "reserve-then-fill"
usage pattern, it is relatively straightforward to implement custom
containers that get the job done.
Seems to improves time to call the collatz function in
pybind/pybind11_benchmark significantly; numbers are a little noisy
but there's a clear improvement from "about 60 ns per call" to "about
45 ns per call" on my machine (M4 Max Mac), as measured with
`timeit.repeat('collatz(4)', 'from pybind11_benchmark import
collatz')`.
* clang-tidy
* more clang-tidy
* clang-tidy NOLINTBEGIN/END instead of NOLINTNEXTLINE
* forgot to increase inline size after removing std::variant
* constexpr arg_vector_small_size, use move instead of swap to hopefully clarify second_pass_convert
* rename test_embed to test_low_level
* rename test_low_level to test_with_catch
* Be careful to NOINLINE slow paths
* rename array/vector members to iarray/hvector. Move comment per request. Add static_asserts for our untagged union implementation per request.
* drop is_standard_layout assertions; see https://github.com/pybind/pybind11/pull/5824#issuecomment-3308616072
* Make function_record type subinterpreter safe
* Get rid of static state in implicit conversion
* style: pre-commit fixes
* Fix lambda
* Bump ABI because we added an internals member
* Set __module__ on the type instance to get rid of DepricationWarning
* Work around internal compiler error in CUDA by not using typedef
hopefully
* Make clang-tidy happy
* Use the same __module__ as pybind11_static_property
* style: pre-commit fixes
* Oops, find-replace error
* style: pre-commit fixes
* Move the once initialization to happen more behind the scenes
* Oops, need those casts...
* Undo implicit conversion change, will do a separate PR
* Use local_internals for function_record pointer to avoid ABI bump
* style: pre-commit fixes
* Get rid of this auto for readability
* Change back to using unqualified tp_name, set __module__ attribute, explicitly add Py_TPFLAGS_HEAPTYPE → does not resolve DeprecationWarning :-(
* Revert "Change back to using unqualified tp_name, set __module__ attribute, explicitly add Py_TPFLAGS_HEAPTYPE → does not resolve DeprecationWarning :-("
This reverts commit 9ccd6de9b7.
* Add Py_TPFLAGS_HEAPTYPE to be explicit (more readable).
* Remove obsolete PYBIND11_WARNING_DISABLE_...
* Make tp_plainname_impl, tp_qualname_impl more DRY
* Change PYBIND11_INTERNAL_MODULE_NAME → PYBIND11_DUMMY_MODULE_NAME
* Add a long comment to explain the tp_qualname_impl workaround.
* Rename local_internals::function_record → function_record_py_type
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Failing unit test
* Potential fix for the issue of re-importing a multi-phase module
- When a module is successfully imported and exec'd, save its handle in a dict in the interpreter state
- Use a special Py_mod_create slot to look in the cache and return the cached handle if it is in the cache
- Don't re-run the user exec function if the module is in the interpreter's cache (implying it was already successfully imported)
* Oops, need to inline these.
* Clang-Tidy fixes
* Oops, debug code
* Add xfail for this GraalPy bug
* Remove static from these function defs, it was a cut-and-paste error in the first place.
* Fix test comment
* Proper error handling
* Oops
* Split up this line, but still just ignore failure .. if the module doesn't have the right properties to check the cache then just allow exec to run.
* Clean up - already looked up the name, just use that.
* Some compilers complain if the pointer isn't taken here, weird.
* Allow attribute errors to be thrown here, will be converted to import errors by the exception handler.
* Remove bogus incref, unconditionally expect a __spec__.name on the module
* Add PR to test comment
* style: pre-commit fixes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Replace static bool with thread-specific-storage to make this code sub-interpreter and free-threading safe.
* Make sure there is only one tss value in existence for this.
The previous code had multiple (one for every type pair, as this is a template function), which may have posed a problem for some platforms.
* Make set_flag in implicitly_convertible() non-copyable/movable
set_flag is an RAII guard for a thread-specific reentrancy flag.
Copying or moving it would risk double-resetting or rearming the flag,
breaking the protection. Disable copy/move constructors and assignment
operators to make this explicit.
* Minor cleanup to avoid venturing into UB territory.
* Experiment: Disable `~thread_specific_storage()` body when using GraalPy.
* Try the suggestion to only call TSS_free if the python interpreter is still active.
* Add IsFinalizing check
* Put this back to having a per-template-instance static
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
When compiling an application using pybind11 3.0.0, GCC 13.3.0 and
python 3.11.13 the following warning is emitted [1]:
In function 'PyObject* PyCFunction_GET_SELF(PyObject*)',
inlined from 'void pybind11::cpp_function::initialize_generic(unique_function_record&&, const char*, const std::type_info* const*, pybind11::size_t)' at /opt/conda/lib/python3.11/site-packages/pybind11/include/pybind11/pybind11.h:605:30:
/opt/conda/include/python3.11/cpython/methodobject.h:50:16: error: potential null pointer dereference [-Werror=null-dereference]
50 | return _Py_NULL;
| ^~~~~~~~
It stems form the fact that PyCFunction_GET_SELF can return a nullptr.
Let's fail in this case.
[1]: https://gitlab.com/tango-controls/pytango/-/jobs/10671972312#L570
* Revert PR #5700 production code change (pybind11/detail/struct_smart_holder.h).
```
git checkout b19489145b2c7a117138632d624809dfb3b380bb~1 include/pybind11/detail/struct_smart_holder.h
```
* Introduce `get_internals().get_memory_guarded_delete()`
* [skip ci] Only pass around `memory::get_guarded_delete` function pointer.
* [skip ci] Change a variable name for internal consistency. Add 3 x NOTE: PYBIND11_INTERNALS_VERSION needs to be bumped if changes are made to this struct.
* Add comment: get_internals().get_memory_guarded_delete does not need with_internals()
* Traverse all DSOs to find memory::guarded_delete with matching RTTI.
* Add nullptr check to dynamic_cast overload.
Suggested by ChatGPT for these reasons:
* Prevents runtime RTTI lookups on nullptr.
* Helps avoid undefined behavior if users pass in nulls from failed casts or optional paths.
* Ensures consistent return value semantics and no accidental access to vtable.
* Improve smart_holder unique_ptr deleter compatibility checks across DSOs:
* Replace RTTI-based detection of std::default_delete<T> with a constexpr check to avoid RTTI reliance
* Add type_info_equal_across_dso_boundaries() fallback using type_info::name() for RTTI equality across macOS DSOs
* Rename related flags and functions for clarity (e.g., builtin → std_default)
* Improves ABI robustness and clarity of ownership checks in smart_holder
* Trivial renaming for internal consistency: builtin_delete → std_default_delete
* Add get_trampoline_self_life_support to detail::type_info (passes local testing).
* Polish previous commit slightly.
* [skip ci] Store memory::get_guarded_delete in `detail::type_info` instead of `detail::internals` (no searching across DSOs required).
* Revert change suggested by ChatGPT. After double-checking, ChatGPT agrees this isn't needed.
* Minor polishing.
* Fix Python 3.8 type hints and add module prefix
These type hints are invalid in Python 3.8.
Add `typing.` prefix to remove ambiguity.
* style: pre-commit fixes
* Add module prefix to Union
* Rename macros
* Improve comment
* Comment out 3.8 type hint macros
Fixing this issue in Python 3.8 will require updating lots of tests. This can be added in a further pull request.
* Add Iterable module prefix
* Add module prefix to Iterator
* Add module prefix to Callable
* Re-add accidentally deleted brackets
* Add module prefix to Optional
* Add module prefix to Final
* Add module prefix to ClassVar
* Add module prefix to TypeGuard
* Add module prefix to TypeIs
* Add module prefix to NoReturn
* Add module prefix to Never
* Add module prefix to Literal
* Add module prefix to Callable
* Add module prefix to Sequence
* Add module prefix to Iterator
* style: pre-commit fixes
* Remove type hint macros
* style: pre-commit fixes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Move embedded modules to multiphase init
So that they too can support multi-interpreter and nogil tags
* Update the multiple interpreter test for embedded module changes
* Add a note to embedded module docs about the new tags
* Oops, missed a warning pop
* Remove unused variable
* Update ci.yml
* Fix this embedded GIL test for free-threading
* Oops, need to use ptr() here
* This test created a subinterpreter when PYBIND11_SUBINTERPRETER_SUPPORT was off
So the fix is really this test should not be run in these older versions at all.
The hang was a GIL issue between the subinterpreters during pybind11::exception::what().
* fix: standard mutex for 3.13t
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
* Allow per-interpreter internals/local_internals
* Significant rewrite to avoid using thread_locals as much as possible.
Since we can avoid them by checking this atomic, the cmake config conditional shouldn't be necessary.
The slower path (with thread_locals and extra checks) only comes in when a second interpreter is actually instanciated.
* Add a test for per-interpreter GIL
Uses two extra threads to demonstrate that neither shares a GIL.
* Fix for nonconforming std::atomic constructors on some compilers
* style: pre-commit fixes
* Fix initializer to make MSVC happy.
* Switch to gil_scoped_acquire_simple, get rid of old copy of it from internals.h
* Use the PyThreadState's interp member rather than the thread state itself.
* Be more explicit about the type of the internalspp
* Suggested renamings and rewordings
* Rename find_internals_pp and change it to take in the state dict reference
* Use the old raise_from instead of pybind11_fail
* Move most of the internals initialization into its constructor.
* Move round_up_to_next_pow2 function upwards
* Remove redundant forward decl
* Add a python-driven subinterpreter test
* Disable the python subinterpreter test on emscripten
Can't load the native-built cpp modules.
* Switch the internals pointer pointer to a unique_ptr pointer
* Spelling
* Fix clang-tidy warning, compare pointer to nullptr
* Rename get_interpreter_counter to get_num_interpreters_seen
* Try simplifying the test's cmake set_target_properties
* Replace mod_* tags with a single tag w/enum
Update tests accordingly
* Add a test for shared-GIL (legacy) subinterpreters
* Update test to work around differences in the various versions of interpreters modules
* Fix unused parameter
* Rename tests and associated test modules.
* Switch get_internals_pp to a template function
* Rename curtstate to cur_tstate
* refactor: use simpler names
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* style: pre-commit fixes
* fix: return class, not enum
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Have to join these threads to make sure they are totally done before the test returns.
* Wrap module_def initialization in a static so it only happens once.
If it happens concurrently in multiple threads, badness ensues....
* style: pre-commit fixes
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
* Strictly enforce: trampoline must inherit from trampoline_self_life_support when used in combination with smart_holder
* Simplify test_class_sh_trampoline_basic.cpp,py (only one Abase is needed now)
* Replace obsolete sophisticated `throw value_error()` with a simple `assert()`
* Strictly enforce: trampoline should inherit from trampoline_self_life_support only if used in combination with smart_holder
* Resolve clang-tidy error
```
/__w/pybind11/pybind11/tests/test_class_sh_trampoline_basic.cpp:35:46: error: the parameter 'obj' is copied for each invocation but only used as a const reference; consider making it a const reference [performance-unnecessary-value-param,-warnings-as-errors]
35 | int AddInCppSharedPtr(std::shared_ptr<Abase> obj, int other_val) {
| ^
| const &
```
* Disable new static_assert if PYBIND11_RUN_TESTING_WITH_SMART_HOLDER_AS_DEFAULT_BUT_NEVER_USE_IN_PRODUCTION_PLEASE is defined.
* fix missing pythonic type hints for native_enum
* Fix __qualname__ in native_enum
* Rename `enum class native` → `func_sig_rendering`. Add to `ENUM_TYPES_AND_MEMBERS`. Move new code around to fit in more organically.
"native" is used in so many places here, it would be very difficult to pin-point where the specific enum type is used.
Similarly, "value" is difficult to pin-point. Changed to "nested_value".
---------
Co-authored-by: Bryn Lloyd <12702862+dyollb@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Change PYBIND11_MODULE to use multi-phase init
Use slots to specify advanced init options (currently just Py_GIL_NOT_USED).
Adds a new function `initialize_multiphase_module_def` to module_, similar to the existing (unchanged) create_extension_module.
* Avoid dynamic allocation and non-trivial destruction
... by using an std::array for the slots.
* Oops, stray cut and paste character
* Remove assignment from placement new, change size fo slots array
* style: pre-commit fixes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* init
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* remove import
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* remove uneeded function
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* style: pre-commit fixes
* Add missing import
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* style: pre-commit fixes
* Fix type behind detailed_message_enabled flag
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* Fix type behind detailed_message_enabled flag
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* Add io_name comment
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* Extra loops to single function
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* style: pre-commit fixes
* Remove unneeded forward declaration
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* Switch variable name away from macro
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* Switch variable name away from macro
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* Switch variable name away from macro
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* clang-tidy
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* remove stack import
* Fix bug in std::function Callable type
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* style: pre-commit fixes
* remove is_annotation argument
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* style: pre-commit fixes
* Update function name and arg names
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
* style: pre-commit fixes
---------
Signed-off-by: Michael Carlstrom <rmc@carlstrom.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Pure `git merge --squash smart_holder` (no manual interventions).
* Remove ubench/ directory.
* Remove include/pybind11/smart_holder.h
* [ci skip] smart_ptrs.rst updates [WIP/unfinished]
* [ci skip] smart_ptrs.rst updates continued; also updating classes.rst, advanced/classes.rst
* Remove README_smart_holder.rst
* Restore original README.rst from master
* [ci skip] Minimal change to README.rst, to leave a hint that this is pybind11v3
* [ci skip] Work in ChatGPT suggestions.
* Change macro name to PYBIND11_RUN_TESTING_WITH_SMART_HOLDER_AS_DEFAULT_BUT_NEVER_USE_IN_PRODUCTION_PLEASE
* Add a note pointing to the holder reinterpret_cast.
* Incorporate suggestion by @virtuald: https://github.com/pybind/pybind11/pull/5542#discussion_r1967000989
* Systematically change most py::class_ to py::classh under docs/
* Remove references to README_smart_holder.rst
This should have been part of commit eb550d03d3.
* [ci skip] Fix minor oversight (``class_`` -> ``py::class_``) noticed by chance.
* [ci skip] Resolve suggestion by @virtuald
https://github.com/pybind/pybind11/pull/5542#discussion_r1969940605
* [ci skip] Apply suggestions by @timohl (thanks!)
* https://github.com/pybind/pybind11/pull/5542#discussion_r1970714551
* https://github.com/pybind/pybind11/pull/5542#discussion_r1971315329
* https://github.com/pybind/pybind11/pull/5542#discussion_r1971322821
* Replace `classh : class_` inhertance with `using`, as suggested by @henryiii
https://github.com/pybind/pybind11/pull/5542#issuecomment-2689034104
* Revert "Systematically change most py::class_ to py::classh under docs/"
This reverts commit ac9d31e13f.
* docs: focus on py::smart_holder instead of py::classh
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
* Restore minor general fixes that got lost when ac9d31e13f was reverted.
* Remove `- smart_holder` from list of branches in all .github/workflows
* Extend classh note to explain whitespace noise motivation.
* Suggest `py::smart_holder` for "most situations for safety"
* Add back PYBIND11_HAS_INTERNALS_WITH_SMART_HOLDER_SUPPORT
This define was
* introduced with https://github.com/pybind/pybind11/pull/5286
* removed with https://github.com/pybind/pybind11/pull/5531
It is has been in use here:
* f02a2b7653/pybind11_protobuf/native_proto_caster.h (L89-L101)
Currently pybind11 unit tests for the two holder caster backwards compatibility traits
* `copyable_holder_caster_shared_ptr_with_smart_holder_support_enabled`
* `move_only_holder_caster_unique_ptr_with_smart_holder_support_enabled`
are missing.
* Add py::trampoline_self_life_support to all trampoline examples under docs/.
Address suggestion by @timohl:
* https://github.com/pybind/pybind11/pull/5542#issuecomment-2686452062
Add to the "please think twice" note: the overhead for safety is likely in the noise.
Also fix a two-fold inconsistency introduced by revert-commit 1e646c91b4:
1.
py::trampoline_self_life_support is mentioned in a note, but is missing in the example right before.
2.
The section starting with
To enable safely passing a ``std::unique_ptr`` to a trampoline object between
is obsolete.
* Fix whitespace accident (indentation) introduced with 1e646c91b4
Apparently the mis-indentation was introduced when resolving merge conflicts for what became 1e646c91b4
* WHITESPACE CHANGES ONLY in README.rst (list of people that made significant contributions)
* Add Ethan Steinberg to list of people that made significant contributions (for completeness, unrelated to smart_holder work).
* [ci skip] Add to list of people that made significant contributions: major and/or influential contributors to smart_holder branch
* #2904 by @rhaschke was merged on Mar 16, 2021
* #3012 by @rhaschke was merged on May 28, 2021
* #3039 by @jakobandersen was merged on Jun 29, 2021
* #3048 by @Skylion007 was merged on Jun 18, 2021
* #3588 by @virtuald was merged on Jan 3, 2022
* #3633 by @wangxf123456 was merged on Jan 25, 2022
* #3635 by @virtuald was merged on Jan 26, 2022
* #3645 by @wangxf123456 was merged on Jan 25, 2022
* #3796 by @wangxf123456 was merged on Mar 10, 2022
* #3807 by @wangxf123456 was merged on Mar 18, 2022
* #3838 by @wangxf123456 was merged on Apr 15, 2022
* #3929 by @tomba was merged on May 7, 2022
* #4031 by @wangxf123456 was merged on Jun 27, 2022
* #4343 by @wangxf123456 was merged on Nov 18, 2022
* #4381 by @wangxf123456 was merged on Dec 5, 2022
* #4539 by @wangxf123456 was merged on Feb 28, 2023
* #4609 by @wangxf123456 was merged on Apr 6, 2023
* #4775 by @wangxf123456 was merged on Aug 3, 2023
* #4921 by @iwanders was merged on Nov 7, 2023
* #4924 by @iwanders was merged on Nov 6, 2023
* #5401 by @msimacek was merged on Oct 8, 2024
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: Ivor Wanders <iwanders@users.noreply.github.com>
Co-authored-by: Jakob Lykke Andersen <Jakob@caput.dk>
Co-authored-by: Michael Šimáček <michael.simacek@oracle.com>
Co-authored-by: Robert Haschke <rhaschke@users.noreply.github.com>
Co-authored-by: Tomi Valkeinen <tomi.valkeinen@iki.fi>
Co-authored-by: Xiaofei Wang <6218006+wangxf123456@users.noreply.github.com>
---------
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: Ivor Wanders <iwanders@users.noreply.github.com>
Co-authored-by: Jakob Lykke Andersen <Jakob@caput.dk>
Co-authored-by: Michael Šimáček <michael.simacek@oracle.com>
Co-authored-by: Robert Haschke <rhaschke@users.noreply.github.com>
Co-authored-by: Tomi Valkeinen <tomi.valkeinen@iki.fi>
Co-authored-by: Xiaofei Wang <6218006+wangxf123456@users.noreply.github.com>
* Backport of https://github.com/google/pybind11clif/pull/30088 (main PR) and https://github.com/google/pybind11clif/pull/30092 (minor fixes).
Note for completeness:
These are identical to the current versions on the pybind11clif main branch (@ commit 4841661df5daf26ecdedaace54e64d0782e63f64):
* test_class_release_gil_before_calling_cpp_dtor.cpp
* test_class_release_gil_before_calling_cpp_dtor.py
* Fix potential data race in test_class_release_gil_before_calling_cpp_dtor.cpp
The original intent was to let the singleton leak, but making that tread-safe is slightly more involved than this solution. It's totally fine in this case if the RegistryType destructor runs on process teardown.
See also: https://github.com/pybind/pybind11/pull/5522#issuecomment-2661068351
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgk@google.com>
* Added rework of arg/return typing
* Changed `Path` to `pathlib.Path` for compatibility with pybind11-stubgen
* Removed old arg/return type hint implementation
* Added noconvert support for arg/return type hints
* Added commented failing tests for Literals with special characters
* Added return_descr/arg_descr for correct typing in typing::Callable
* Fixed clang-tidy issues
* Changed io_name to have explicit return type (for C++11 support)
* style: pre-commit fixes
* Added support for nested callables
* Fixed missing include
* Fixed is_return_value constructor call
* Fixed clang-tidy issue
* Uncommented test cases for special characters in literals
* Moved literal tests to correct test case
* Added escaping of special characters in typing::Literal
* Readded mistakenly deleted bracket
* Moved sanitize_string_literal to correct namespace
* Added test for Literal with `!` and changed StringLiteral template param name
* Added test for Literal with multiple and repeated special chars
* Simplified string literal sanitization function
* Added test for `->` in literal
* Added test for `->` with io_name
* Removed unused parameter name to prevent warning
* Added escaping of `-` in literal to prevent processing of `->`
* Fixed wrong computation of sanitized string literal length
* Added cast to prevent error with MSVC
* Simplified special character check
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix module type hint
"module" is not a valid python value.
The correct type hint for a module object is "types.ModuleType" which has existed since at least Python 2.6
* Added module type hint test
* style: pre-commit fixes
* Remove doc function
* Fixed type hint
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Added arg/return type handling.
* Added support for nested arg/return type in py::typing::List
* Added support for arg/return type in stl/filesystem
* Added tests for arg/return type in stl/filesystem and py::typing::List
* Added arg/return name to more py::typing classes
* Added arg/return type to Callable[...]
* Added tests for typing container classes (also nested)
* Changed typing classes to avoid using C++14 auto return type deduction.
* Fixed clang-tidy errors.
* Changed Enable to SFINAE
* Added test for Tuple[T, ...]
* Added RealNumber with custom caster for testing typing classes.
* Added tests for Set, Iterable, Iterator, Union, and Optional
* Added tests for Callable
* Fixed Callable with ellipsis test
* Changed TypeGuard/TypeIs to use return type (being the narrower type) + Tests
* Added test for use of fallback type name with stl vector
* Updated documentation.
* Fixed unnecessary constructor call in test.
* Fixed reference counting in example type caster.
* Fixed clang-tidy issues.
* Fix for clang-tidy
* Updated cast method to use pybind11 API rather than Python C API in custom caster example
* Updated load to use pybind11 API rather than Python C API in custom caster example
* Changed test of arg/return name to use pybind11 API instead of Python C API
* Updated code in adcanced/cast example and improved documentation text
* Fixed references in custom type caster docs
* Fixed wrong logical and operator in test
* Fixed wrong logical operator in doc example
* Added comment to test about `float` vs `float | int`
* Updated std::filesystem::path docs in cast/overview section
* Remove one stray dot.
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* fix: make PYBIND11_WARNING_POP actually pop clang diagnostics
* fix: ignore -Wgnu-zero-variadic-macro-arguments on clang
* Revert "fix: ignore -Wgnu-zero-variadic-macro-arguments on clang"
This reverts commit d310959bf5.
* C++20 modernization: Use `__VA_OPT__(, ) __VA_ARGS__` in `PYBIND11_DECLARE_HOLDER_TYPE()`
* Disable `__VA_OPT__(, ) __VA_ARGS__` usage for MSVC (it is unclear to rwgk why it does not work).
This is the beginning of the error message:
```
D:\a\pybind11\pybind11\tests\test_smart_ptr.cpp(285,1): error C2143: syntax error: missing ')' before ',' [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
D:\a\pybind11\pybind11\tests\test_smart_ptr.cpp(285,1): error C2059: syntax error: ')' [D:\a\pybind11\pybind11\build\tests\pybind11_tests.vcxproj]
```
* Add `PYBIND11_WARNING_DISABLE_CLANG("-Wgnu-zero-variadic-macro-arguments")` in test_smart_ptr.cpp
This is the error message:
```
/__w/pybind11/pybind11/tests/test_smart_ptr.cpp:287:51: error: must specify at least one argument for '...' parameter of variadic macro [-Werror,-Wgnu-zero-variadic-macro-arguments]
PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>)
^
/__w/pybind11/pybind11/include/pybind11/cast.h:885:13: note: macro 'PYBIND11_DECLARE_HOLDER_TYPE' defined here
^
```
* Also add `PYBIND11_WARNING_DISABLE_CLANG("-Wgnu-zero-variadic-macro-arguments")` in test_virtual_functions.cpp
* Also add `PYBIND11_WARNING_DISABLE_CLANG("-Wgnu-zero-variadic-macro-arguments")` in test_embed/test_interpreter.cpp
* Undo all changes except the original push -> pop fix.
* 1. Add `PYBIND11_WARNING_DISABLE_CLANG("-Wgnu-zero-variadic-macro-arguments")` near the top of pybind11/pybind11.h; 2. Change `PYBIND11_DECLARE_HOLDER_TYPE` macro to side-step the only remaining clang warning-as-error (this is still needed even for clang 18).
Alternatively the warning suppression could be moved into pybind11/cast.h, but this commit limits the warning suppression to smaller scope within include/pybind11.
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* feat: allow annotate methods with `pos_only` when only have the `self` argument
* chore(typing): make arguments for auto-generated dunder methods positional-only
* docs: add more comments to improve readability
* style: fix nit suggestions
* Add test_self_only_pos_only() in tests/test_methods_and_attributes
* test: add docstring tests for generated dunder methods
* test: remove failed tests
* fix(test): run `gc.collect()` three times for refcount tests
---------
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
* Fix data race all_type_info_populate in free-threading mode
Description:
- fixed data race all_type_info_populate in free-threading mode
- added test
For example, we have 2 threads entering `all_type_info`.
Both enter `all_type_info_get_cache`` function and
there is a first one which inserts a tuple (type, empty_vector) to the map
and second is waiting. Inserting thread gets the (iter_to_key, True) and non-inserting thread
after waiting gets (iter_to_key, False).
Inserting thread than will add a weakref and will then call into `all_type_info_populate`.
However, non-inserting thread is not entering `if (ins.second) {` clause and
returns `ins.first->second;`` which is just empty_vector.
Finally, non-inserting thread is failing the check in `allocate_layout`:
```c++
if (n_types == 0) {
pybind11_fail(
"instance allocation failed: new instance has no pybind11-registered base types");
}
```
* style: pre-commit fixes
* Addressed PR comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Initial support for GraalPy
* Mark tests that currently fail on GraalPy with xfail
* Add graalpy to CI
* Limit test deps on graalpy to available binary wheels
* Skip cmake test installed_function on GraalPy
CMake won't find libpython on GraalPy, it either fails or silently picks
CPython's libpython.
* Factor out setting function docstrings into a macro
* Try to narrow down skipped tests
