Commit Graph

3247 Commits

Author SHA1 Message Date
Dustin Spicuzza
ab392bd845 fix: avoid copy constructor instantiation in shared_ptr fallback cast (#6028)
* tests: add regressions for shared_ptr reference_internal fallback

* fix: avoid copy constructor instantiation in shared_ptr fallback cast

* Remove stray empty line

* tests: rename PyTorch shared_ptr regression test files

* refactor: add cast_non_owning helper for reference-like casts

Name the non-owning generic cast path so callers do not have to rediscover that
reference-like policies must pass null copy/move constructor callbacks. This
keeps the shared_ptr reference_internal fallback self-documenting and points
future maintainers toward the safe API.

Made-with: Cursor

* tests: guard deprecated-copy warning probes with __has_warning

Use __has_warning for the Clang-only regression test so older compiler jobs skip
unsupported warning groups instead of failing with -Wunknown-warning-option. A
simple __clang_major__ >= 13 guard would be shorter, but it bakes in a version
cutoff; __has_warning is slightly more verbose while being more robust to
vendor builds, backports, and future packaging differences.

Made-with: Cursor

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2026-04-19 10:01:51 +07:00
dependabot[bot]
ad5bc9e80e chore(deps): bump the actions group with 2 updates (#6027) 2026-04-19 10:01:51 +07:00
Ralf W. Grosse-Kunstleve
1b49908389 docs: add v3.0.3 ~and v3.1.0~ changelog updates. (#6023)
git amend: v3.1.0 section removed

* [skip ci] docs: add v3.0.3 and v3.1.0 changelog updates.

Made-with: Cursor

* [skip ci] docs: set v3.0.3 release date: March 31, 2026
v3.0.3
2026-03-30 20:38:29 -07:00
Ralf W. Grosse-Kunstleve
c066c762ba fix: add missing PYBIND11_ALWAYS_INLINE macro for v3.0.3 backports
Cherry-picking #5992 without #5887 left this macro undefined and broke builds, so this restores the expected inline helper definition with the smallest possible change.

Made-with: Cursor
2026-03-30 12:33:20 -07:00
Ralf W. Grosse-Kunstleve
76efcb3715 Bump version from v3.0.2 → v3.0.3 2026-03-30 12:22:21 -07:00
dependabot[bot]
0f41635768 chore(deps): bump pygments from 2.17.2 to 2.20.0 in /docs (#6024)
Bumps [pygments](https://github.com/pygments/pygments) from 2.17.2 to 2.20.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.17.2...2.20.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-version: 2.20.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-30 12:11:15 -07:00
dependabot[bot]
54a4615ed6 chore(deps): bump requests from 2.32.4 to 2.33.0 in /docs (#6013)
Bumps [requests](https://github.com/psf/requests) from 2.32.4 to 2.33.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.4...v2.33.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.33.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-29 23:11:57 -07:00
pre-commit-ci[bot]
d12037cd9d chore(deps): update pre-commit hooks (#6002)
* chore(deps): update pre-commit hooks

updates:
- [github.com/pre-commit/mirrors-clang-format: v21.1.8 → v22.1.0](https://github.com/pre-commit/mirrors-clang-format/compare/v21.1.8...v22.1.0)
- [github.com/astral-sh/ruff-pre-commit: v0.14.14 → v0.15.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.14...v0.15.4)
- [github.com/adhtruong/mirrors-typos: v1.42.3 → v1.44.0](https://github.com/adhtruong/mirrors-typos/compare/v1.42.3...v1.44.0)
- [github.com/PyCQA/pylint: v4.0.4 → v4.0.5](https://github.com/PyCQA/pylint/compare/v4.0.4...v4.0.5)
- [github.com/python-jsonschema/check-jsonschema: 0.36.1 → 0.37.0](https://github.com/python-jsonschema/check-jsonschema/compare/0.36.1...0.37.0)

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-03-29 23:11:57 -07:00
dependabot[bot]
64eecc4cd3 chore(deps): bump the actions group with 3 updates (#6000) 2026-03-29 23:11:57 -07:00
Ralf W. Grosse-Kunstleve
c0ea8fcd23 Fix heap-buffer-overflow in pythonbuf with undersized buffers (#6019)
* Fix heap-buffer-overflow in pythonbuf with undersized buffers (gh-5886)

The _sync() UTF-8 remainder logic can leave pptr() past the end of
the allocated buffer when buf_size < 4: after moving up to 3 bytes
of an incomplete UTF-8 sequence to the front, pbump(remainder) pushes
pptr() beyond epptr() and the buffer boundary.  The next overflow()
then writes out of bounds.

Fix by clamping the buffer size to a minimum of 4 in the constructor,
ensuring the maximum UTF-8 remainder (3 bytes) plus the overflow slot
(1 byte) always fits within the allocated buffer.

Made-with: Cursor

* Avoid C++14 ODR-use linker error for minimum_buffer_size

std::max takes arguments by const&, which ODR-uses the static constexpr
member and requires an out-of-line definition in C++14. Replace with a
ternary expression that uses the value without taking its address.

Made-with: Cursor
2026-03-29 23:08:30 -07:00
Ralf W. Grosse-Kunstleve
8d621d614a fix: detect virtual inheritance in add_base to prevent pointer offset crash (#6017)
Virtual inheritance places the base subobject at a dynamic offset, but
load_impl Case 2a uses reinterpret_cast which assumes a fixed offset.
This caused segfaults when dispatching inherited methods through virtual
bases (e.g. SftVirtDerived2::name()).

Add an is_static_downcastable SFINAE trait that detects whether
static_cast<Derived*>(Base*) is valid. When it is not (virtual
inheritance), set multiple_inheritance = true in add_base to force the
implicit_casts path, which correctly adjusts pointers at runtime.

Remove the workaround .def("name", &SftVirtDerived2::name) from
test_smart_ptr.cpp that was papering over the issue.

Made-with: Cursor
2026-03-29 23:08:30 -07:00
Ralf W. Grosse-Kunstleve
b6a616e526 fix: strdup "self" arg in def_property_static, partially revert #6010 (gh-5976) (#6015)
* fix: strdup args added after initialize_generic in def_property_static (gh-5976)

`def_property_static` calls `process_attributes::init` on already-initialized
function records (after `initialize_generic`'s strdup loop has run).
Args added at this stage (e.g. "self" via `append_self_arg_if_needed`) remain
as string literals, so `destruct()` would call `free()` on them.

Fix by strdup'ing name/descr of any args appended by the late
`process_attributes::init` call. Root cause introduced by gh-5486.

Made-with: Cursor

* Partially revert gh-6010: remove py_is_finalizing() workarounds

Now that the root cause (free of string literals in def_property_static,
gh-5976) is fixed in the previous commit, the py_is_finalizing() guards
introduced in gh-6010 are no longer needed:

- tp_dealloc_impl: remove early return during finalization (was leaking
  all function records instead of properly destroying them)
- destruct(): remove guard around arg.value.dec_ref()
- common.h: remove py_is_finalizing() helper (no remaining callers)

The genuine fix from gh-6010 (PyObject_Free + Py_DECREF ordering in
tp_dealloc_impl) is retained.

Made-with: Cursor

* test: add embedding test for py::enum_ across interpreter restart (gh-5976)

py::enum_ is the primary trigger for gh-5976 because its constructor
creates properties via def_property_static / def_property_readonly_static,
which call process_attributes::init on already-initialized function records.
Yet none of the existing embedding tests used py::enum_ at all.

Add an PYBIND11_EMBEDDED_MODULE with py::enum_ and a test case that imports
it, finalize/reinitializes the interpreter, and re-imports it. This exercises
the def_property_static code path that was fixed in the preceding commit.

Note: on Python 3.14.2 (and likely 3.12+), tp_dealloc_impl is not called
during Py_FinalizeEx for function record PyObjects — they simply leak because
types are effectively immortalized. As a result, this test cannot trigger the
original free()-on-string-literal crash on this Python version. However, it
remains valuable as a regression guard: on Python builds where finalization
does clean up function records (or if CPython changes this behavior), the
test would catch the crash. It also verifies that py::enum_ survives
interpreter restart correctly, which was previously untested.

Made-with: Cursor

* test: skip enum restart test on Python 3.12 (pre-existing crash)

Made-with: Cursor

* Add test_standalone_enum_module.py, standalone_enum_module.cpp

* Make standalone_enum_module.cpp more similar to #5976 reproducer. Also fix clang-tidy error.

* This crashes when testing locally:

( cd /wrk/forked/pybind11/tests && PYTHONPATH=/wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/lib /wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/TestVenv/bin/python3 -m pytest test_standalone_enum_module.py )

============================= test session starts ==============================
platform linux -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0
installed packages of interest: build==1.4.2 numpy==2.4.3 scipy==1.17.1
C++ Info: 13.3.0 C++20 __pybind11_internals_v12_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1__ PYBIND11_SIMPLE_GIL_MANAGEMENT=False
rootdir: /wrk/forked/pybind11/tests
configfile: pytest.ini
plugins: timeout-2.4.0, xdist-3.8.0
collected 1 item

test_standalone_enum_module.py F                                         [100%]

=================================== FAILURES ===================================
________________________ test_enum_import_exit_no_crash ________________________

    def test_enum_import_exit_no_crash():
        # Modeled after reproducer under issue #5976
>       env.check_script_success_in_subprocess(
            f"""
            import sys
            sys.path.insert(0, {os.path.dirname(env.__file__)!r})
            import standalone_enum_module as m
            assert m.SomeEnum.__class__.__name__ == "pybind11_type"
            """,
            rerun=1,
        )

test_standalone_enum_module.py:10:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"'

    def check_script_success_in_subprocess(code: str, *, rerun: int = 8) -> None:
        """Runs the given code in a subprocess."""
        import os
        import subprocess
        import sys
        import textwrap

        if ANDROID or IOS or sys.platform.startswith("emscripten"):
            pytest.skip("Requires subprocess support")

        code = textwrap.dedent(code).strip()
        try:
            for _ in range(rerun):  # run flakily failing test multiple times
                subprocess.check_output(
                    [sys.executable, "-c", code],
                    cwd=os.getcwd(),
                    stderr=subprocess.STDOUT,
                    text=True,
                )
        except subprocess.CalledProcessError as ex:
>           raise RuntimeError(
                f"Subprocess failed with exit code {ex.returncode}.\n\n"
                f"Code:\n"
                f"```python\n"
                f"{code}\n"
                f"```\n\n"
                f"Output:\n"
                f"{ex.output}"
            ) from None
E           RuntimeError: Subprocess failed with exit code -6.
E
E           Code:
E           ```python
E           import sys
E           sys.path.insert(0, '/wrk/forked/pybind11/tests')
E           import standalone_enum_module as m
E           assert m.SomeEnum.__class__.__name__ == "pybind11_type"
E           ```
E
E           Output:
E           munmap_chunk(): invalid pointer

_          = 0
code       = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"'
os         = <module 'os' (frozen)>
rerun      = 1
subprocess = <module 'subprocess' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/subprocess.py'>
sys        = <module 'sys' (built-in)>
textwrap   = <module 'textwrap' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/textwrap.py'>

env.py:68: RuntimeError
=========================== short test summary info ============================
FAILED test_standalone_enum_module.py::test_enum_import_exit_no_crash - Runti...
============================== 1 failed in 0.23s ===============================

ERROR: completed_process.returncode=1

* Add "Added in PR #6015" comments, for easy reference back to this PR

* test: use PYBIND11_CATCH2_SKIP_IF for Python 3.12 enum restart skip

Replace #if/#else/#endif preprocessor guard with runtime
PYBIND11_CATCH2_SKIP_IF so the test is always compiled and
shows [ SKIPPED ] in output on Python 3.12.

Made-with: Cursor

* fix: suppress MSVC C4127 in PYBIND11_CATCH2_SKIP_IF macro

The constant condition in PYBIND11_CATCH2_SKIP_IF triggers MSVC
warning C4127 (conditional expression is constant), which becomes
a build error under /WX.

Made-with: Cursor
2026-03-29 23:08:30 -07:00
Ralf W. Grosse-Kunstleve
0184c0212f Harden PYBIND11_MODULE_PYINIT and get_internals() against crashes during module init (#6018)
* Wrap ensure_internals() in try-catch in PYBIND11_MODULE_PYINIT

Previously, ensure_internals() was called without exception handling
in the PyInit_* function (PYBIND11_MODULE_PYINIT), while the same call
in PYBIND11_MODULE_EXEC was already wrapped in try-catch. On MSVC,
a C++ exception propagating through the extern "C" PyInit_* boundary
is undefined behavior, which can manifest as an access violation
instead of a clean error message. This is a potential contributor to
crashes like gh-5993. Wrap the entire PyInit body in try/catch using
the existing PYBIND11_CATCH_INIT_EXCEPTIONS pattern.

Made-with: Cursor

* Add nullptr guards in get_internals() for better crash diagnostics

Add explicit null checks after get_pp() and create_pp_content_once()
in get_internals(), calling pybind11_fail() with descriptive messages.
These guards convert potential null-pointer dereferences (which produce
unhelpful access-violation crashes, especially on Windows) into clear
runtime_error messages that can be caught and reported as ImportError
by the try-catch added in the previous commit.

Made-with: Cursor
2026-03-29 23:08:30 -07:00
Ralf W. Grosse-Kunstleve
ec875b63f0 Fix TSS key exhaustion in implicitly_convertible() (gh-5975) (#6020)
Replace `static thread_specific_storage<int>` with `thread_local bool`
in the implicit conversion reentrancy guard. Since implicitly_convertible
is a template function, each unique <InputType, OutputType> pair created
its own TSS key via PyThread_tss_create(). Projects with hundreds of
modules and many implicit conversions could exhaust PTHREAD_KEYS_MAX
(1024 on Linux, 512 on macOS), especially on Python 3.12+ where CPython
itself consumes more TSS keys for subinterpreter support.

thread_local bool is safe here because:
- bool is trivially destructible, so it works on all C++11 platforms
  including older macOS (the concern that motivated the TSS approach in
  PR #5777 applied only to types with non-trivial destructors needing
  __cxa_thread_atexit runtime support)
- Each thread gets its own copy, so it is thread-safe for free-threading
- Subinterpreter sharing is benign: the guard prevents recursive implicit
  conversions on the same thread regardless of which interpreter is active
- The v3.0.0 code already used thread_local bool under Py_GIL_DISABLED

This effectively reverts the core change from PR #5777 while keeping
the non-copyable/non-movable set_flag guard.

Made-with: Cursor
2026-03-29 23:08:30 -07:00
Ralf W. Grosse-Kunstleve
bb786681c4 Fix static_pointer_cast build failure with virtual inheritance in holder_caster_foreign_helpers.h (#6014)
* Add regression test for #5989: static_pointer_cast fails with virtual inheritance

When a class uses virtual inheritance and its holder type is shared_ptr,
passing a shared_ptr of the derived type as a method argument triggers
a compilation error because static_pointer_cast cannot downcast through
a virtual base (dynamic_pointer_cast is needed instead).

Made-with: Cursor

* Fix #5989: use dynamic_pointer_cast for virtual inheritance in esft downcast

Replace the unconditional static_pointer_cast in set_via_shared_from_this
with a SFINAE-dispatched esft_downcast helper that falls back to
dynamic_pointer_cast when static_cast through a virtual base is ill-formed.

Also add a workaround in the test binding (.def("name") on SftVirtDerived2)
for a separate pre-existing issue with inherited method dispatch through
virtual bases.

Made-with: Cursor
2026-03-29 23:08:29 -07:00
Aaron Gokaslan
3cb5a763c1 fix: bind noexcept and ref-qualified methods from unregistered base classes (#5992)
* Strip noexcept from cpp17 function type bindings

* Fix a bug and increase test coverage

* Does this fix it?

* Silence clang-tidy issue

* Simplify method adapter with macro and add missing rvalue adaptors + tests

* Supress clang-tidy errors

* Improve test coverage

* Add additional static assert

* Try to resolve MSVC C4003 warning

* Simplify method adaptor into 2 template instatiations with enable_if_t

* Fix ambiguous STL template

* Close remaining qualifier consistency gaps for member pointer bindings.

A production-code review after #2234 showed that ref-qualified member pointers were still inconsistently handled across def_buffer, vectorize, and overload_cast, so this adds the missing overloads with focused tests for each newly-supported signature.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Clarify why def_buffer/vectorize omit rvalue-qualified overloads.

These comments were added while reviewing the qualifier coverage follow-up, to document that buffer/vectorized calls operate on existing Python-owned instances and should not move-from self.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add compile-only overload_cast guard for ref-qualified methods.

This was added as a maintenance follow-up to the qualifier-consistency work, so future changes that introduce overload_cast ambiguity or wrong ref/noexcept resolution fail at compile time.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Refactor overload_cast_impl qualifier overloads with a macro.

As part of the qualifier-consistency maintenance follow-up, this reduces duplication in overload_cast_impl while preserving the same ref/noexcept coverage and keeping pedantic-clean macro expansion.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Expose __cpp_noexcept_function_type to Python tests and use explicit skip guards.

This replaces hasattr-based optional assertions with skipif-gated noexcept-only tests so skipped coverage is visible in pytest output while keeping non-noexcept checks always active.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Add static_assert in method_adaptor to guard that T is a member function pointer.

Suggested by @Skylion007 in PR #5992 review comment [T007].

Made-with: Cursor

* automatic clang-format change (because of #6002)

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-03-29 23:08:29 -07:00
Dustin Spicuzza
c0cfa96555 Fix crash in def_readwrite for non-smart-holder properties of smart-holder classes (v2) (#6008)
* Add tests that cause crash in def_readwrite

- Occurs with non-smart-holder property of smart-holder class

* Fix crash in def_readwrite for non-smart-holder properties of smart-holder classes

* Use default policy

* Address PR comments

* Add test for cast error path

* style: pre-commit fixes

* Revert "Use default policy"

This reverts commit b299f32104.

* Disable test_shared_ptr_return_for_unique_ptr_holder when PYBIND11_TEST_SMART_HOLDER=ON

* Add counterexample

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2026-03-29 23:08:29 -07:00
Marco Garten
463168a901 Resolve ambiguous factory spec. with nvcc+GCC (#6011)
Explicitly specify the 4th template parameter in the
single-factory partial specialization of `factory` to
disambiguate it from the dual-factory specialization
when compiled with nvcc + GCC 14. Fixes #5565.

Co-authored-by: Oz <oz-agent@warp.dev>
2026-03-29 23:08:29 -07:00
Yury Matveev
16ef2e302c Fix memory leak: clear managed dict in pybind11_object_dealloc on Python 3.13+ (#5999)
* fix: clear managed dict in pybind11_object_dealloc on Python 3.13+

On Python 3.14, PyObject_GC_Del (tp_free) no longer implicitly clears
the managed dict of objects with Py_TPFLAGS_MANAGED_DICT. Without an
explicit PyObject_ClearManagedDict() call before tp_free(), objects
stored in the __dict__ of py::dynamic_attr() instances have their
refcounts permanently abandoned, causing memory leaks — capsule
destructors for numpy arrays (and other objects) never run.

Adds a regression test: stores a py::capsule in the __dict__ of a
DynamicClass instance and asserts the capsule destructor is called
when the instance is deleted.

* [tests]: mark test_dynamic_attr_dealloc_frees_dict_contents to be strict=False xfail on PYPY

* [docs]: clarify Python version comments in pybind11_object_dealloc

Distinguish between when the API is available (3.13+, where
PyObject_ClearManagedDict was introduced) and when the leak actually
manifests (3.14+, where tp_free stopped implicitly clearing the
managed dict).

---------

Co-authored-by: Yury Matveev <yury.matveev@desy.de>
2026-03-29 23:08:29 -07:00
Itamar Oren
7db5e992b4 gh-5991: Fix segfault during finalization related to function_record (#6010)
* gh-5991: Fix segfault during finalization related to function_record

This patch was developed with assistance from  Claude Code Opus 4.6

Here's Claude's explanation of the crash mechanism and some reasoning for the difficulty to repro:

`tp_dealloc_impl` calls `cpp_function::destruct` which:
1. Calls `std::free()` on function_record string members (`name`, `doc`, `signature`)
2. Calls `arg.value.dec_ref()` on default argument values
3. Calls `delete rec` on the function_record

But it never calls `PyObject_Free(self)` or `Py_DECREF(Py_TYPE(self))`, which are
required for heap types.

During `_Py_Finalize`, final GC collects the heap types (which survive module dict
clearing via `tp_mro` self-references). This triggers a massive cascade:
`type_dealloc → property_dealloc → meth_dealloc → tp_dealloc_impl → destruct`.

At scale (~1,200+ function_records), the volume of `delete`/`free` calls corrupts
heap metadata, causing subsequent `std::free()` to receive garbage pointers → SEGV.

* Add detail::py_is_finalizing() wrapper to deduplicate version-guarded #ifdef blocks

Also fixes clang-tidy readability-implicit-bool-conversion warnings.

Made-with: Cursor

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2026-03-29 23:08:29 -07:00
Henry Schreiner
fb1438351f chore: some minor CPython API cleanup (#6005)
* chore: use PyType_GetFlags

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

* chore: use public VectorCall in 3.9+

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

---------

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
2026-03-29 23:08:29 -07:00
Malcolm Smith
1bf20ea7d9 Re-enable Android tests in CIBW workflow (#6001)
* Re-enable Android tests in CIBW workflow

* Skip subprocess tests on Android

* Remove Android workarounds no longer necessary with current cibuildwheel version

* Skip more subprocess tests on Android
2026-03-29 23:08:29 -07:00
Yaakov Selkowitz
a896f61573 Move tomlkit dependency to dev group (#5990)
tomlkit is used only in the packaging tests which are not ordinarily run as
part of the normal workflow of a user or downstream packager.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2026-03-29 23:08:29 -07:00
Ralf W. Grosse-Kunstleve
f6198fb209 [ci skip] fix(tests): make async callback test deterministic (#5986)
Replace the fixed sleep in test_async_callbacks with a bounded wait for all expected callback results, so detached worker scheduling no longer causes sporadic CI failures.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-03-29 23:08:29 -07:00
Ralf W. Grosse-Kunstleve
45fab4087e Update version number to v3.0.2 (final) and set release date in changelog.md to February 16, 2026 (#5985) v3.0.2 2026-02-16 20:31:46 -08:00
Ralf W. Grosse-Kunstleve
44a0cd241e [ci skip] docs/changelog.md updates for 3.0.2, to account for 5 additional PRs (#5984) 2026-02-16 09:33:30 -08:00
Xuehai Pan
3ae5a173c5 Add fallback implementation of PyCriticalSection_BeginMutex for Python 3.13t (#5981)
* Add failback implementation of `PyCriticalSection_BeginMutex` for Python 3.13t

* Add comment for Python version

* Use `_PyCriticalSection_BeginSlow`

* Add forward declaration

* Fix forward declaration

* Remove always true condition `defined(PY_VERSION_HEX)`

* Detect musllinux

* Add manylinux test

* Use direct mutex locking for Python 3.13t

`_PyCriticalSection_BeginSlow` is a private CPython function not exported
on Linux. For Python < 3.14.0rc1, use direct `mutex.lock()`/`mutex.unlock()`
instead of critical section APIs.

* Empty commit to trigger CI

* Empty commit to trigger CI

* Empty commit to trigger CI

* Run apt update before apt install

* Remove unnecessary prefix

* Add manylinux test with Python 3.13t

* Simplify pycritical_section with std::unique_lock fallback for Python < 3.14

* Fix potential deadlock in make_iterator_impl for Python 3.13t

Refactor pycritical_section into a unified class with internal version
checks instead of using a type alias fallback. Skip locking in
make_iterator_impl for Python < 3.14.0rc1 to avoid deadlock during
type registration, as pycritical_section cannot release the mutex
during Python callbacks without PyCriticalSection_BeginMutex.

* Add reference for xfail message
2026-02-09 21:14:07 -08:00
Daniel Simon
5f2c678916 Add helpers to array that return the size and strides as a std::span (#5974)
* Add helper functions to pybind11::array to return the shape and strides as a std::span. These functions are hidden with macros unless PYBIND11_CPP20 is defined and the <span> include has been found.

* style: pre-commit fixes

* tests: Add unit tests for shape_span() and strides_span()

Add comprehensive unit tests for the new std::span helper functions:
- Test 0D, 1D, 2D, and 3D arrays
- Verify spans match regular shape()/strides() methods
- Test that spans can be used to construct new arrays
- Tests are conditionally compiled only when PYBIND11_HAS_SPAN is defined

* Use __cpp_lib_span feature test macro instead of __has_include

Replace __has_include(<span>) check with __cpp_lib_span feature test macro
to resolve ambiguity where some pre-C++20 systems might have a global
header called <span> that isn't the C++20 std::span.

The check is moved after <version> is included, consistent with how
__cpp_lib_char8_t is handled.

Co-authored-by: Cursor <cursoragent@cursor.com>

* Fix: Use py::ssize_t instead of ssize_t in span tests

On Windows/MSVC, ssize_t is not available in the standard namespace
without proper includes. Use py::ssize_t (the pybind11 typedef) instead
to ensure cross-platform compatibility.

Fixes compilation errors on:
- Windows/MSVC 2022 (C++20)
- GCC 10 (C++20)

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-08 15:04:46 -08:00
pre-commit-ci[bot]
8f68ecd32c chore(deps): update pre-commit hooks (#5982)
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.10 → v0.14.14](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.10...v0.14.14)
- [github.com/Lucas-C/pre-commit-hooks: v1.5.5 → v1.5.6](https://github.com/Lucas-C/pre-commit-hooks/compare/v1.5.5...v1.5.6)
- [github.com/adhtruong/mirrors-typos: v1.41.0 → v1.42.3](https://github.com/adhtruong/mirrors-typos/compare/v1.41.0...v1.42.3)
- [github.com/python-jsonschema/check-jsonschema: 0.36.0 → 0.36.1](https://github.com/python-jsonschema/check-jsonschema/compare/0.36.0...0.36.1)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-02-02 17:16:53 -08:00
Sam Gross
4d7d02a8e5 Fix race condition with py::make_key_iterator in free threading (#5971)
* Fix race condition with py::make_key_iterator in free threading

The creation of the iterator class needs to be synchronized.

* style: pre-commit fixes

* Use PyCriticalSection_BeginMutex instead of recursive mutex

* style: pre-commit fixes

* Make pycritical_section non-copyable and non-movable

The pycritical_section class is a RAII wrapper that manages a Python
critical section lifecycle:
- Acquires the critical section in the constructor via
  PyCriticalSection_BeginMutex
- Releases it in the destructor via PyCriticalSection_End
- Holds a reference to a pymutex

Allowing copy or move operations would be dangerous:

1. Copy: Both the original and copied objects would call
   PyCriticalSection_End on the same PyCriticalSection object in their
   destructors, leading to double-unlock and undefined behavior.

2. Move: The moved-from object's destructor would still run and attempt
   to end the critical section, while the moved-to object would also try
   to end it, again causing double-unlock.

This follows the same pattern used by other RAII lock guards in the
codebase, such as gil_scoped_acquire and gil_scoped_release, which also
explicitly delete copy/move operations to prevent similar issues.

By explicitly deleting these operations, we prevent accidental misuse
and ensure the critical section is properly managed by a single RAII
object throughout its lifetime.

* Drop Python 3.13t support from CI

Python 3.13t was experimental, while Python 3.14t is not. This PR
uses PyCriticalSection_BeginMutex which is only available in Python
3.14+, making Python 3.13t incompatible with the changes.

Removed all Python 3.13t CI jobs:
- ubuntu-latest, 3.13t (standard-large matrix)
- macos-15-intel, 3.13t (standard-large matrix)
- windows-latest, 3.13t (standard-large matrix)
- manylinux job testing 3.13t

This aligns with the decision to drop Python 3.13t support as
discussed in PR #5971.

* Add Python 3.13 (default) replacement jobs for removed 3.13t jobs

After removing Python 3.13t support (incompatible with PyCriticalSection_BeginMutex
which requires Python 3.14+), we're adding replacement jobs using Python 3.13
(default) to maintain test coverage in key dimensions:

1. ubuntu-latest, Python 3.13: C++20 + DISABLE_HANDLE_TYPE_NAME_DEFAULT_IMPLEMENTATION
   - Replaces: ubuntu-latest, 3.13t with same config
   - Maintains coverage for this specific configuration combination

2. macos-15-intel, Python 3.13: C++11
   - Replaces: macos-15-intel, 3.13t with same config
   - Maintains macOS coverage for Python 3.13

3. manylinux (musllinux), Python 3.13: GIL testing
   - Replaces: manylinux, 3.13t job
   - Maintains manylinux/musllinux container testing coverage

These additions are proposed to get feedback on which jobs should be kept
to maintain appropriate test coverage without the experimental 3.13t builds.

* ci: run in free-threading mode a bit more on 3.14

* Revert "ci: run in free-threading mode a bit more on 3.14"

This reverts commit 91189c9242.

Reason: https://github.com/pybind/pybind11/pull/5971#issuecomment-3831321903

* Reapply "ci: run in free-threading mode a bit more on 3.14"

This reverts commit f3197de975.

After #5972 is/was merged, tests should pass (already tested under #5980).

See also https://github.com/pybind/pybind11/pull/5972#discussion_r2752674989

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
2026-02-01 22:02:50 -08:00
Xuehai Pan
e7754de037 Revert internals destruction and add test for internals recreation (#5972)
* Bump internals version

* Prevent internals destruction before all pybind11 types are destroyed

* Use Py_XINCREF and Py_XDECREF

* Hold GIL before decref

* Use weakrefs

* Remove unused code

* Move code location

* Move code location

* Move code location

* Try add tests

* Fix PYTHONPATH

* Fix PYTHONPATH

* Skip tests for subprocess

* Revert to leak internals

* Revert to leak internals

* Revert "Revert to leak internals"

This reverts commit c5ec1cf886.
This reverts commit 72c2e0aa9b.

* Revert internals version bump

* Reapply to leak internals

This reverts commit 8f25a254e8.

* Add re-entrancy detection for internals creation

Prevent re-creation of internals after destruction during interpreter
shutdown. If pybind11 code runs after internals have been destroyed,
fail early with a clear error message instead of silently creating
new empty internals that would cause type lookup failures.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Fix C++11/C++14 support

* Add lock under multiple interpreters

* Try fix tests

* Try fix tests

* Try fix tests

* Update comments and assertion messages

* Update comments and assertion messages

* Update comments

* Update lock scope

* Use original pointer type for Windows

* Change hard error to warning

* Update lock scope

* Update lock scope to resolve deadlock

* Remove scope release of GIL

* Update comments

* Lock pp on reset

* Mark content created after assignment

* Update comments

* Simplify implementation

* Update lock scope when delete unique_ptr

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 21:54:05 -08:00
dependabot[bot]
0080cae388 chore(deps): bump jwlawson/actions-setup-cmake in the actions group (#5978)
Bumps the actions group with 1 update: [jwlawson/actions-setup-cmake](https://github.com/jwlawson/actions-setup-cmake).


Updates `jwlawson/actions-setup-cmake` from 2.0 to 2.1
- [Release notes](https://github.com/jwlawson/actions-setup-cmake/releases)
- [Commits](https://github.com/jwlawson/actions-setup-cmake/compare/v2.0...v2.1)

---
updated-dependencies:
- dependency-name: jwlawson/actions-setup-cmake
  dependency-version: '2.1'
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-31 22:10:57 -08:00
Ralf W. Grosse-Kunstleve
95d4af7ff9 Fix CI issue: numpy requirement for Python 3.14 on ARM64 Windows (#5977)
* Fix pip install conflicts in tests/requirements.txt

- Fix numpy version conflict for Python 3.14 on ARM64 by excluding
  Python 3.14+ from the ARM64-specific numpy>=2.3.0 requirement
- Add scipy requirement for Python 3.13 on Windows

Co-authored-by: Cursor <cursoragent@cursor.com>

* Revert "Fix pip install conflicts in tests/requirements.txt"

This reverts commit 1d63c73be4.

* Fix numpy requirement for Python 3.14 on ARM64 Windows

Change numpy==2.4.0 to numpy>=2.4.0 for Python 3.14+ to allow
pip to install numpy 2.4.1 or later versions, which are available
for Python 3.14 on ARM64 Windows (MSYS2) where numpy 2.4.0 is not.

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-31 22:09:06 -08:00
Sam Gross
4f81a12507 Fix deadlock in test with free threading (#5973)
Importing "widget_module" re-enables the GIL. In current versions of
CPython, this requires pausing all threads attached to all interpreters.
The spinning on sync/num without a py::gil_scoped_release causes
occasional deadlocks.
2026-01-28 23:02:08 -08:00
Ralf W. Grosse-Kunstleve
6c836071ad Changelog updates (#5967)
* docs: seed 3.0.2 changelog from needs-changelog PRs

Collect suggested entries early to streamline release prep.

* Misc trivial manual fixes.

* Shorten changelog entry for PR 5862

* Remove mention of a minor doc formatting fix.

* Cursor-generated "all past-tense" style

* Restore the meaning of the 5958 entry using the "... now ..." trick, and restore a couple other entries that also use the "now" trick.

* Replace ... now ... style with ... updated to ... style

* [skip ci] docs: group 3.0.2 entries under Internal heading

Align changelog categories with recent releases for review.

* Update changelog with CMake policy compatibility fix

Fix compatibility with CMake policy CMP0190 for cross-compiling.

* Add changelog entries for 5965 and 5968

* docs: make CMP0190 changelog entry past tense

Align 3.0.2 bug-fix entry with changelog style.

* [skip ci] docs: add missing 3.0.2 changelog entries

Capture remaining needs-changelog PRs across categories. (These slipped through the cracks somehow.)

---------

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2026-01-20 22:51:18 -08:00
Franz Pöschel
5a6edc9998 Exclude MSVC up to 19.16 from using std::launder (#5968)
* Exclude further MSVC versions from std::launder

Versions 19.4, 19.5 and 19.6 now also excluded. Error seen with 19.6, error triggered by this commit:
57b9a0af81

_deps\fetchedpybind11-src\include\pybind11\pybind11.h(3008): fatal error C1001: An internal error has occurred in the compiler. [C:\projects\openpmd-api\build\openPMD.py.vcxproj]
  (compiler file 'd:\agent\_work\8\s\src\vctools\compiler\utc\src\p2\main.c', line 187)
   To work around this problem, try simplifying or changing the program near the locations listed above.
  Please choose the Technical Support command on the Visual C++
   Help menu, or open the Technical Support help file for more information

* Add minimal comment // See PR #5968

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2026-01-20 21:49:59 -08:00
Malcolm Smith
53ccc32f84 fix: cross-compilation updates (#5829)
* Android updates

* Revert removal of Interpreter
2026-01-20 17:35:16 -05:00
b-pass
a8e223d0cd Directly check if/which interpreter is active before doing CLEAR in destructor (#5965)
* Directly check if/which interpreter is active before doing CLEAR in destructors.

Py_IsFinalizing only applies to the main interpreter.

* Backward compatibility fixes

* Make clang-tidy happy

* Add nullptr checks to istate as Cursor suggested
2026-01-20 06:51:25 -05:00
b-pass
da6e071084 Destruct internals during interpreter finalization (#5958)
* Add a shutdown method to internals.

shutdown can safely DECREF Python objects owned by the internals.

* Actually free internals during interpreter shutdown (instead of after)

* Make sure python is alive before DECREFing

If something triggers internals to be created during finalization, it might end up being destroyed after finalization and we don't want to do the DECREF at that point, we need the leaky behavior.

* make clang-tidy happy

* Check IsFinalizing and use Py_CLEAR, make capsule creation safe if the capsule already exists.

* oops, put TLS destructor back how it was.

* Oops, proper spelling of unstable _Py_IsFinalizing

* Add cleanup step to CI workflow

Added a step to clean out unused files to save space during CI.

* Accept suggested comment

* Avoid recreating internals during type deallocation at shutdown.

---------

Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
2026-01-18 13:24:34 -05:00
Henry Schreiner
e44aae2268 chore: bump CMake max policy to 4.2 (#5944)
* chore: bupm CMake to 4.1

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

* ci: use newest version of CMake in a few places

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

* chore: bupm CMake to 4.2

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>

---------

Signed-off-by: Henry Schreiner <henryfs@princeton.edu>
2026-01-15 12:38:45 -05:00
dependabot[bot]
ca1d996461 chore(deps): bump urllib3 from 2.6.0 to 2.6.3 in /docs (#5954)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.0 to 2.6.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.6.0...2.6.3)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-version: 2.6.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-10 11:44:19 -08:00
Xuehai Pan
cc551ada33 Appease MSVC Warning C4866: compiler may not enforce left-to-right evaluation order (#5955) 2026-01-10 10:46:01 -08:00
Xuehai Pan
d36f5dd5a4 Appease MSVC Warning C4866: compiler may not enforce left-to-right evaluation order (#5953)
* Appease MSVC Warning C4866: compiler may not enforce left-to-right evaluation order

* Remove const qualifier

* Reword comment to be self-explanatory without PR context

The previous comment referenced the MSVC warning but didn't explain
why the code is structured as two statements. The revised comment
clarifies the intent: fetching the value first ensures well-defined
evaluation order.

* chore(deps): switch typos to mirror repo

Switch from crate-ci/typos to adhtruong/mirrors-typos because
pre-commit autoupdate confuses tags in the upstream repo, selecting
the mutable `v1` tag instead of pinned versions like `v1.41.0`.

See https://github.com/crate-ci/typos/issues/390

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2026-01-07 16:28:38 -08:00
T.Yamada
745e5688e7 Fix longdouble complex docstring to clongdouble (#5952) 2026-01-06 23:04:17 -08:00
b-pass
10f8708979 Change function calls to use vectorcall (#5948)
* Make argument_vector re-usable for other types.

* Attempt to collect args into array for vectorcall

* Revert "Attempt to collect args into array for vectorcall"

This reverts commit 418a034195.

* Implement vectorcall args collector

* pre-commit fixes

* Checkpoint in moving to METH_FASTCALL

* pre-commit fixes

* Use the names tuple directly, cleaner code and less reference counting

* Fix unit test, the code now holds more references

It cannot re-use the incoming tuple as before, because it is no longer a tuple at all.  So a new tuple must be created, which then holds references for each member.

* Make clangtidy happy

* Oops, _v is C++14

* style: pre-commit fixes

* Minor code cleanup

* Fix signed conversions

* Fix args expansion

This would be easier with `if constexpr`

* style: pre-commit fixes

* Code cleanup

* fix(tests): Install multiple-interpreter test modules into wheel

The `mod_per_interpreter_gil`, `mod_shared_interpreter_gil`, and
`mod_per_interpreter_gil_with_singleton` modules were being built
but not installed into the wheel when using scikit-build-core
(SKBUILD=true). This caused iOS (and potentially Android) CIBW
tests to fail with ModuleNotFoundError.

Root cause analysis:
- The main test targets have install() commands (line 531)
- The PYBIND11_MULTIPLE_INTERPRETERS_TEST_MODULES were missing
  equivalent install() commands
- For regular CMake builds, this wasn't a problem because
  LIBRARY_OUTPUT_DIRECTORY places the modules next to pybind11_tests
- For wheel builds, only targets with explicit install() commands
  are included in the wheel

This issue was latent until commit fee2527d changed the test imports
from `pytest.importorskip()` (graceful skip) to direct `import`
statements (hard failure), which exposed the missing modules.

Failing tests:
- test_multiple_interpreters.py::test_independent_subinterpreters
- test_multiple_interpreters.py::test_dependent_subinterpreters

Error: ModuleNotFoundError: No module named 'mod_per_interpreter_gil'

* tests: Pin numpy 2.4.0 for Python 3.14 CI tests

Add numpy==2.4.0 requirement for Python 3.14 (both default and
free-threaded builds). NumPy 2.4.0 is the first version to provide
official PyPI wheels for Python 3.14:

- numpy-2.4.0-cp314-cp314-manylinux_2_27_x86_64...whl (default)
- numpy-2.4.0-cp314-cp314t-manylinux_2_27_x86_64...whl (free-threaded)

Previously, CI was skipping all numpy-dependent tests for Python 3.14
because PIP_ONLY_BINARY was set and no wheels were available:

  SKIPPED [...] test_numpy_array.py:8: could not import 'numpy':
  No module named 'numpy'

With this change, the full numpy test suite will run on Python 3.14,
providing better test coverage for the newest Python version.

Note: Using exact pin (==2.4.0) rather than compatible release (~=2.4.0)
to ensure reproducible CI results with the first known-working version.

* tests: Add verbose flag to CIBW pytest command

Add `-v` to the pytest command in tests/pyproject.toml to help
diagnose hanging tests in CIBW jobs (particularly iOS).

This will show each test name as it runs, making it easier to
identify which specific test is hanging.

* tests: Skip subinterpreter tests on iOS, add pytest timeout

- Add `IOS` platform constant to `tests/env.py` for consistency with
  existing `ANDROID`, `LINUX`, `MACOS`, `WIN`, `FREEBSD` constants.

- Skip `test_multiple_interpreters.py` module on iOS. Subinterpreters
  are not supported in the iOS simulator environment. These tests were
  previously skipped implicitly because the modules weren't installed
  in the wheel; now that they are (commit 6ed6d5a8), we need an
  explicit skip.

- Change pytest timeout from 0 (disabled) to 120 seconds. This provides
  a safety net to catch hanging tests before the CI job times out after
  hours. Normal test runs complete in 33-55 seconds total (~1100 tests),
  so 120 seconds per test is very generous.

- Add `-v` flag for verbose output to help diagnose any future issues.

* More cleanups in argument vector, per comments.

* Per Cursor, move all versions to Vectorcall since it has been supported since 3.8.

This means getting rid of simple_collector, we can do the same with a constexpr if in the unpacking_collector.

* Switch to a bool vec for the used_kwargs flag...

This makes more sense and saves a sort, and the small_vector implementation means it will actually take less space than a vector of size_t elements.

The most common case is that all kwargs are used.

* Fix signedness for clang

* Another signedness issue

* tests: Disable pytest-timeout for Pyodide (no signal.setitimer)

Pyodide runs in a WebAssembly sandbox without POSIX signals, so
`signal.setitimer` is not available. This causes pytest-timeout to
crash with `AttributeError: module 'signal' has no attribute 'setitimer'`
when timeout > 0.

Override the test-command for Pyodide to keep timeout=0 (disabled).

* Combine temp storage and args into one vector

It's a good bit faster at the cost of this one scary reinterpret_cast.

* Phrasing

* Delete incorrect comment

At 6, the struct is 144 bytes (not 128 bytes as the comment said).

* Fix push_back

* Update push_back in argument_vector.h

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>

* style: pre-commit fixes

* Use real types for these instead of object

They can be null if you "steal" a null handle.

* refactor: Replace small_vector<object> with ref_small_vector for explicit ownership

Introduce `ref_small_vector` to manage PyObject* references in `unpacking_collector`,
replacing the previous `small_vector<object>` approach.

Primary goals:

1. **Maintainability**: The previous implementation relied on
   `sizeof(object) == sizeof(PyObject*)` and used a reinterpret_cast to
   pass the object array to vectorcall. This coupling to py::object's
   internal layout could break if someone adds a debug field or other
   member to py::handle/py::object in the future.

2. **Readability**: The new `push_back_steal()` vs `push_back_borrow()`
   API makes reference counting intent explicit at each call site,
   rather than relying on implicit py::object semantics.

3. **Intuitive code**: Storing `PyObject*` directly and passing it to
   `_PyObject_Vectorcall` without casts is straightforward and matches
   what the C API expects. No "scary" reinterpret_cast needed.

Additional benefits:
- `PyObject*` is trivially copyable, simplifying vector operations
- Batch decref in destructor (tight loop vs N individual object destructors)
- Self-documenting ownership semantics

Design consideration: We considered folding the ref-counting functionality
directly into `small_vector` via template specialization for `PyObject*`.
We decided against this because:
- It would give `small_vector<PyObject*, N>` a different interface than the
  generic `small_vector<T, N>` (steal/borrow vs push_back)
- Someone might want a non-ref-counting `small_vector<PyObject*, N>`
- The specialization behavior could surprise users expecting uniform semantics

A separate `ref_small_vector` type makes the ref-counting behavior explicit
and self-documenting, while keeping `small_vector` generic and predictable.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
2026-01-06 16:32:57 -05:00
pre-commit-ci[bot]
c761608a22 chore(deps): update pre-commit hooks (#5951)
* chore(deps): update pre-commit hooks

updates:
- [github.com/pre-commit/mirrors-clang-format: v21.1.6 → v21.1.8](https://github.com/pre-commit/mirrors-clang-format/compare/v21.1.6...v21.1.8)
- [github.com/astral-sh/ruff-pre-commit: v0.14.7 → v0.14.10](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.7...v0.14.10)
- [github.com/pre-commit/mirrors-mypy: v1.19.0 → v1.19.1](https://github.com/pre-commit/mirrors-mypy/compare/v1.19.0...v1.19.1)
- [github.com/crate-ci/typos: v1.40.0 → v1](https://github.com/crate-ci/typos/compare/v1.40.0...v1)
- [github.com/python-jsonschema/check-jsonschema: 0.35.0 → 0.36.0](https://github.com/python-jsonschema/check-jsonschema/compare/0.35.0...0.36.0)

* fix(tests): allow numpy>=2.3.0 for ARM64 to fix Windows ARM CI

MSYS2 upgraded from numpy 2.3.x to 2.4.0, which no longer satisfies
the ~=2.3.0 constraint. Relaxing to >=2.3.0 allows the MSYS2-provided
numpy 2.4.0 to be used.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2026-01-05 23:02:23 -08:00
dependabot[bot]
2c9191e9dc chore(deps): bump egor-tensin/setup-clang in the actions group (#5950)
Bumps the actions group with 1 update: [egor-tensin/setup-clang](https://github.com/egor-tensin/setup-clang).


Updates `egor-tensin/setup-clang` from 1 to 2
- [Release notes](https://github.com/egor-tensin/setup-clang/releases)
- [Commits](https://github.com/egor-tensin/setup-clang/compare/v1...v2)

---
updated-dependencies:
- dependency-name: egor-tensin/setup-clang
  dependency-version: '2'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-12-31 21:17:09 -08:00
Ralf W. Grosse-Kunstleve
b93c0f7ed8 Fix ambiguous str(handle) constructor for object-derived types (#5949)
* Fix ambiguous `str(handle)` constructor for object-derived types

Templatize `str(handle h)` with SFINAE to exclude types derived from
`object`, resolving ambiguity with `str(const object&)` when calling
`py::str()` on types like `kwargs`, `dict`, etc.

The template now only accepts `handle` or `PyObject*`, while all
`object`-derived types use the `str(const object&)` overload.

* fix(tests): CIBW test fixes from b-pass→vectorcall branch

- Install multiple-interpreter test modules into wheel (CMakeLists.txt)
  The mod_per_interpreter_gil, mod_shared_interpreter_gil, and
  mod_per_interpreter_gil_with_singleton modules were being built
  but not installed into the wheel when using scikit-build-core.

- Pin numpy 2.4.0 for Python 3.14 CI tests (requirements.txt)
  NumPy 2.4.0 is the first version with official Python 3.14 wheels.

- Add IOS platform constant to tests/env.py

- Skip subinterpreter tests on iOS (test_multiple_interpreters.py)
  Subinterpreters are not supported in the iOS simulator environment.

- Enable pytest timeout of 120s for CIBW tests (pyproject.toml)
  Provides a safety net to catch hanging tests before CI job timeout.

- Disable pytest-timeout for Pyodide (no signal.setitimer)
  Pyodide runs in WebAssembly without POSIX signals.

- Add -v flag for verbose pytest output in CIBW tests
2025-12-30 04:54:44 -08:00
Xuehai Pan
fee2527dfa Fix concurrency consistency for internals_pp_manager under multiple-interpreters (#5947)
* Add per-interpreter storage for `gil_safe_call_once_and_store`

* Disable thread local cache for `internals_pp_manager`

* Disable thread local cache for `internals_pp_manager` for multi-interpreter only

* Use anonymous namespace to separate these type_ids from other tests with the same class names.

* style: pre-commit fixes

* Revert internals_pp_manager changes

* This is the crux of fix for the subinterpreter_before_main failure.

The pre_init needs to check if it is in a subinterpreter or not. But in 3.13+ this static initializer runs in the main interpreter.  So we need to check this later, during the exec phase.

* Continue to do the ensure in both places, there might be a reason it was where it was...

Should not hurt anything to do it extra times here.

* Change get_num_interpreters_seen to a boolean flag instead.

The count was not used, it was just checked for > 1, we now accomplish this by setting the flag.

* Spelling typo

* Work around older python versions, only need this check for newish versions

* Add more comments for test case

* Add more comments for test case

* Stop traceback propagation

* Re-enable subinterpreter support on ubuntu 3.14 builds

Was disabled in e4873e8

* As suggested, don't use an anonymous namespace.

* Typo in test assert format string

* Use a more appropriate function name

* Fix mod_per_interpreter_gil* output directory on Windows/MSVC

On Windows with MSVC (multi-configuration generators), CMake uses
config-specific properties like LIBRARY_OUTPUT_DIRECTORY_DEBUG when
set, otherwise falls back to LIBRARY_OUTPUT_DIRECTORY/<Config>/.

The main test modules (pybind11_tests, etc.) correctly set both
LIBRARY_OUTPUT_DIRECTORY and the config-specific variants (lines
517-528), so they output directly to tests/.

However, the mod_per_interpreter_gil* modules only copied the base
LIBRARY_OUTPUT_DIRECTORY property, causing them to be placed in
tests/Debug/ instead of tests/.

This mismatch caused test_import_in_subinterpreter_concurrently and
related tests to fail with ModuleNotFoundError on Windows Python 3.14,
because the test code sets sys.path based on pybind11_tests.__file__
(which is in tests/) but tries to import mod_per_interpreter_gil_with_singleton
(which ended up in tests/Debug/).

This bug was previously masked by @pytest.mark.xfail decorators on
these tests. Now that the underlying "Duplicate C++ type registration"
issue is fixed and the xfails are removed, this path issue surfaced.

The fix mirrors the same pattern used for main test targets: also set
LIBRARY_OUTPUT_DIRECTORY_<CONFIG> for each configuration type.

* Remove unneeded `pytest.importorskip`

* Remove comment

---------

Co-authored-by: b-pass <b-pass@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
2025-12-26 13:59:11 -05:00
Xuehai Pan
0057e4945d Add per-interpreter storage for gil_safe_call_once_and_store (#5933)
* Add new argument to `gil_safe_call_once_and_store::call_once_and_store_result`

* Add per-interpreter storage for `gil_safe_call_once_and_store`

* Make `~gil_safe_call_once_and_store` a no-op

* Fix C++11 compatibility

* Improve thread-safety and add default finalizer

* Try fix thread-safety

* Try fix thread-safety

* Add a warning comment

* Simplify `PYBIND11_INTERNALS_VERSION >= 12`

* Try fix thread-safety

* Try fix thread-safety

* Revert get_pp()

* Update comments

* Move call-once storage out of internals

* Revert internal version bump

* Cleanup outdated comments

* Move atomic_bool alias into pybind11::detail namespace

The `using atomic_bool = ...` declaration was at global scope,
polluting the global namespace. Move it into pybind11::detail
to avoid potential conflicts with user code.

* Add explicit #include <unordered_map> for subinterpreter support

The subinterpreter branch uses std::unordered_map but relied on
transitive includes. Add an explicit include for robustness.

* Remove extraneous semicolon after destructor definition

Style fix: remove trailing semicolon after ~call_once_storage()
destructor body.

* Add comment explaining unused finalize parameter

Clarify why the finalize callback parameter is intentionally ignored
when subinterpreter support is disabled: the storage is process-global
and leaked to avoid destructor calls after interpreter finalization.

* Add comment explaining error_scope usage

Clarify why error_scope is used: to preserve any existing Python
error state that might be cleared or modified by dict_getitemstringref.

* Improve exception safety in get_or_create_call_once_storage_map()

Use std::unique_ptr to hold the newly allocated storage map until
the capsule is successfully created. This prevents a memory leak
if capsule creation throws an exception.

* Add timeout-minutes: 3 to cpptest workflow steps

Add a 3-minute timeout to all C++ test (cpptest) steps across all
platforms to detect hangs early. This uses GitHub Actions' built-in
timeout-minutes property which works on Linux, macOS, and Windows.

* Add progress reporter for test_with_catch Catch2 runner

Add a custom Catch2 streaming reporter that prints one line per test
case as it starts and ends, with immediate flushing to keep CI logs
current. This makes it easy to see where the embedded/interpreter
tests are spending time and to pinpoint which test case is stuck
when builds hang (e.g., free-threading issues).

The reporter:
- Prints "[ RUN      ]" when each test starts
- Prints "[       OK ]" or "[  FAILED  ]" when each test ends
- Prints the Python version once at the start via Py_GetVersion()
- Uses StreamingReporterBase for immediate output (not buffered)
- Is set as the default reporter via CATCH_CONFIG_DEFAULT_REPORTER

This approach gives visibility into all tests without changing their
behavior, turning otherwise opaque 90-minute CI timeouts into
locatable issues in the Catch output.

* clang-format auto-fix (overlooked before)

* Disable "Move Subinterpreter" test on free-threaded Python 3.14+

This test hangs in Py_EndInterpreter() when the subinterpreter is
destroyed from a different thread than it was created on.

The hang was observed:
- Intermittently on macOS with Python 3.14.0t
- Predictably on macOS, Ubuntu, and Windows with Python 3.14.1t and 3.14.2t

Root cause analysis points to an interaction between pybind11's
subinterpreter creation code and CPython's free-threaded runtime,
specifically around PyThreadState_Swap() after PyThreadState_DeleteCurrent().

See detailed analysis: https://github.com/pybind/pybind11/pull/5933

* style: pre-commit fixes

* Add test for gil_safe_call_once_and_store per-interpreter isolation

This test verifies that gil_safe_call_once_and_store provides separate
storage for each interpreter when subinterpreter support is enabled.

The test caches the interpreter ID in the main interpreter, then creates
a subinterpreter and verifies it gets its own cached value (not the main
interpreter's). Without per-interpreter storage, the subinterpreter would
incorrectly see the main interpreter's cached object.

* Add STARTING/DONE timestamps to test_with_catch output

Print UTC timestamps at the beginning and end of the test run to make
it immediately clear when tests started and whether they ran to
completion. The DONE message includes the Catch session result value.

Example output:
  [ STARTING ] 2025-12-21 03:23:20.497Z
  [ PYTHON   ] 3.14.2 ...
  [ RUN      ] Threads
  [       OK ] Threads
  [ DONE     ] 2025-12-21 03:23:20.512Z (result 0)

* Disable stdout buffering in test_with_catch

Ensure test output appears immediately in CI logs by disabling stdout
buffering. Without this, output may be lost if the process is killed
by a timeout, making it difficult to diagnose which test was hanging.

* EXPERIMENT: Re-enable hanging test to verify CI log buffering fix

This is a temporary commit to verify that the unbuffered stdout fix
makes the hanging test visible in CI logs. REVERT THIS COMMIT after
confirming the output appears.

* Revert "Disable stdout buffering in test_with_catch"

This reverts commit 0f8f32a92a.

* Use USES_TERMINAL for cpptest to show output immediately

Ninja buffers subprocess output until completion. When a test hangs,
the output is never shown, making it impossible to diagnose which test
is hanging. USES_TERMINAL gives the command direct terminal access,
bypassing ninja's buffering.

This explains why Windows CI showed test progress but Linux/macOS did
not - Windows uses MSBuild which doesn't buffer the same way.

* Fix clang-tidy performance-avoid-endl warning

Use '\n' instead of std::endl since USES_TERMINAL now handles
output buffering at the CMake level.

* Add SIGTERM handler to show when test is killed by timeout

When a test hangs and is killed by `timeout`, Catch2 marks it as failed
but the process exits before printing [ DONE ]. This made it unclear
whether the test failed normally or was terminated.

The signal handler prints a clear message when SIGTERM is received,
making timeout-related failures obvious in CI logs.

* Fix typo: atleast -> at_least

* Fix GCC warn_unused_result error for write() in signal handler

Assign the return value to a variable to satisfy GCC's warn_unused_result
attribute, then cast to void to suppress unused variable warning.

* Add USES_TERMINAL to other C++ test targets

Apply the same ninja output buffering fix to test_cross_module_rtti
and test_pure_cpp targets. Also add explanatory comments to all
USES_TERMINAL usages.

* Revert "EXPERIMENT: Re-enable hanging test to verify CI log buffering fix"

This reverts commit a3abdeea89.

* Update comment to reference PR #5940 for Move Subinterpreter fix

* Add alias `interpid_t = std::int64_t`

* Add isolation and gc test for `gil_safe_call_once_and_store`

* Add thread local cache for gil_safe_call_once_and_store

* Revert "Add thread local cache for gil_safe_call_once_and_store"

This reverts commit 5d6681956d2d326fe74c7bf80e845c8e8ddb2a7c.

* Revert changes according to code review

* Relocate multiple-interpreters tests

* Add more tests for multiple interpreters

* Remove copy constructor

* Apply suggestions from code review

* Refactor to use per-storage capsule instead

* Update comments

* Update singleton tests

* Use interpreter id type for `get_num_interpreters_seen()`

* Suppress unused variable warning

* HACKING

* Revert "HACKING"

This reverts commit 534235ea55.

* Try fix concurrency

* Test even harder

* Reorg code to avoid duplicates

* Fix unique_ptr::reset -> unique_ptr::release

* Extract reusable functions

* Fix indentation

* Appease warnings for MSVC

* Appease warnings for MSVC

* Appease warnings for MSVC

* Try fix concurrency by not using `get_num_interpreters_seen() > 1`

* Try fix tests

* Make Python path handling more robust

* Update comments and assertion messages

* Revert changes according to code review

* Disable flaky tests

* Use `@pytest.mark.xfail` rather than `pytest.skip`

* Retrigger CI

* Retrigger CI

* Revert file moves

* Refactor atomic_get_or_create_in_state_dict: improve API and fix on_fetch_ bug

Three improvements to atomic_get_or_create_in_state_dict:

1. Return std::pair<Payload*, bool> instead of just Payload*
   - The bool indicates whether storage was newly created (true) or
     already existed (false), following std::map::insert convention.
   - This fixes a bug where on_fetch_ was called even for newly created
     internals, when it should only run for fetched (existing) ones.
     (Identified by @b-pass in code review)

2. Change LeakOnInterpreterShutdown from template param to runtime arg
   - Renamed to `clear_destructor` to describe what it does locally,
     rather than embedding assumptions about why it's used.
   - Reduces template instantiations (header-only library benefits).
   - The check is in the slow path (create) anyway, so negligible cost.

3. Remove unnecessary braces around the fast-path lookup
   - The braces created a nested scope but declared no local variables
     that would benefit from scoping.

* Remove unused PYBIND11_MULTIPLE_INTERPRETERS_TEST_FILES variable

This variable was defined but never used.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-12-24 23:33:02 -08:00