fix: strdup "self" arg in def_property_static, partially revert #6010 (gh-5976) (#6015)

* fix: strdup args added after initialize_generic in def_property_static (gh-5976)

`def_property_static` calls `process_attributes::init` on already-initialized
function records (after `initialize_generic`'s strdup loop has run).
Args added at this stage (e.g. "self" via `append_self_arg_if_needed`) remain
as string literals, so `destruct()` would call `free()` on them.

Fix by strdup'ing name/descr of any args appended by the late
`process_attributes::init` call. Root cause introduced by gh-5486.

Made-with: Cursor

* Partially revert gh-6010: remove py_is_finalizing() workarounds

Now that the root cause (free of string literals in def_property_static,
gh-5976) is fixed in the previous commit, the py_is_finalizing() guards
introduced in gh-6010 are no longer needed:

- tp_dealloc_impl: remove early return during finalization (was leaking
  all function records instead of properly destroying them)
- destruct(): remove guard around arg.value.dec_ref()
- common.h: remove py_is_finalizing() helper (no remaining callers)

The genuine fix from gh-6010 (PyObject_Free + Py_DECREF ordering in
tp_dealloc_impl) is retained.

Made-with: Cursor

* test: add embedding test for py::enum_ across interpreter restart (gh-5976)

py::enum_ is the primary trigger for gh-5976 because its constructor
creates properties via def_property_static / def_property_readonly_static,
which call process_attributes::init on already-initialized function records.
Yet none of the existing embedding tests used py::enum_ at all.

Add an PYBIND11_EMBEDDED_MODULE with py::enum_ and a test case that imports
it, finalize/reinitializes the interpreter, and re-imports it. This exercises
the def_property_static code path that was fixed in the preceding commit.

Note: on Python 3.14.2 (and likely 3.12+), tp_dealloc_impl is not called
during Py_FinalizeEx for function record PyObjects — they simply leak because
types are effectively immortalized. As a result, this test cannot trigger the
original free()-on-string-literal crash on this Python version. However, it
remains valuable as a regression guard: on Python builds where finalization
does clean up function records (or if CPython changes this behavior), the
test would catch the crash. It also verifies that py::enum_ survives
interpreter restart correctly, which was previously untested.

Made-with: Cursor

* test: skip enum restart test on Python 3.12 (pre-existing crash)

Made-with: Cursor

* Add test_standalone_enum_module.py, standalone_enum_module.cpp

* Make standalone_enum_module.cpp more similar to #5976 reproducer. Also fix clang-tidy error.

* This crashes when testing locally:

( cd /wrk/forked/pybind11/tests && PYTHONPATH=/wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/lib /wrk/bld/pybind11_gcc_v3.14.2_df793163d58_default/TestVenv/bin/python3 -m pytest test_standalone_enum_module.py )

============================= test session starts ==============================
platform linux -- Python 3.14.2, pytest-9.0.2, pluggy-1.6.0
installed packages of interest: build==1.4.2 numpy==2.4.3 scipy==1.17.1
C++ Info: 13.3.0 C++20 __pybind11_internals_v12_system_libstdcpp_gxx_abi_1xxx_use_cxx11_abi_1__ PYBIND11_SIMPLE_GIL_MANAGEMENT=False
rootdir: /wrk/forked/pybind11/tests
configfile: pytest.ini
plugins: timeout-2.4.0, xdist-3.8.0
collected 1 item

test_standalone_enum_module.py F                                         [100%]

=================================== FAILURES ===================================
________________________ test_enum_import_exit_no_crash ________________________

    def test_enum_import_exit_no_crash():
        # Modeled after reproducer under issue #5976
>       env.check_script_success_in_subprocess(
            f"""
            import sys
            sys.path.insert(0, {os.path.dirname(env.__file__)!r})
            import standalone_enum_module as m
            assert m.SomeEnum.__class__.__name__ == "pybind11_type"
            """,
            rerun=1,
        )

test_standalone_enum_module.py:10:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

code = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"'

    def check_script_success_in_subprocess(code: str, *, rerun: int = 8) -> None:
        """Runs the given code in a subprocess."""
        import os
        import subprocess
        import sys
        import textwrap

        if ANDROID or IOS or sys.platform.startswith("emscripten"):
            pytest.skip("Requires subprocess support")

        code = textwrap.dedent(code).strip()
        try:
            for _ in range(rerun):  # run flakily failing test multiple times
                subprocess.check_output(
                    [sys.executable, "-c", code],
                    cwd=os.getcwd(),
                    stderr=subprocess.STDOUT,
                    text=True,
                )
        except subprocess.CalledProcessError as ex:
>           raise RuntimeError(
                f"Subprocess failed with exit code {ex.returncode}.\n\n"
                f"Code:\n"
                f"```python\n"
                f"{code}\n"
                f"```\n\n"
                f"Output:\n"
                f"{ex.output}"
            ) from None
E           RuntimeError: Subprocess failed with exit code -6.
E
E           Code:
E           ```python
E           import sys
E           sys.path.insert(0, '/wrk/forked/pybind11/tests')
E           import standalone_enum_module as m
E           assert m.SomeEnum.__class__.__name__ == "pybind11_type"
E           ```
E
E           Output:
E           munmap_chunk(): invalid pointer

_          = 0
code       = 'import sys\nsys.path.insert(0, \'/wrk/forked/pybind11/tests\')\nimport standalone_enum_module as m\nassert m.SomeEnum.__class__.__name__ == "pybind11_type"'
os         = <module 'os' (frozen)>
rerun      = 1
subprocess = <module 'subprocess' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/subprocess.py'>
sys        = <module 'sys' (built-in)>
textwrap   = <module 'textwrap' from '/wrk/cpython_installs/v3.14.2_df793163d58_default/lib/python3.14/textwrap.py'>

env.py:68: RuntimeError
=========================== short test summary info ============================
FAILED test_standalone_enum_module.py::test_enum_import_exit_no_crash - Runti...
============================== 1 failed in 0.23s ===============================

ERROR: completed_process.returncode=1

* Add "Added in PR #6015" comments, for easy reference back to this PR

* test: use PYBIND11_CATCH2_SKIP_IF for Python 3.12 enum restart skip

Replace #if/#else/#endif preprocessor guard with runtime
PYBIND11_CATCH2_SKIP_IF so the test is always compiled and
shows [ SKIPPED ] in output on Python 3.12.

Made-with: Cursor

* fix: suppress MSVC C4127 in PYBIND11_CATCH2_SKIP_IF macro

The constant condition in PYBIND11_CATCH2_SKIP_IF triggers MSVC
warning C4127 (conditional expression is constant), which becomes
a build error under /WX.

Made-with: Cursor
This commit is contained in:
Ralf W. Grosse-Kunstleve
2026-03-30 10:17:22 +07:00
committed by GitHub
parent e8cead1626
commit 524d72b36d
7 changed files with 90 additions and 20 deletions

View File

@@ -606,15 +606,6 @@ enum class return_value_policy : uint8_t {
PYBIND11_NAMESPACE_BEGIN(detail)
// Py_IsFinalizing() is a public API since 3.13; before that use _Py_IsFinalizing().
inline bool py_is_finalizing() {
#if PY_VERSION_HEX >= 0x030D0000
return Py_IsFinalizing() != 0;
#else
return _Py_IsFinalizing() != 0;
#endif
}
static constexpr int log2(size_t n, int k = 0) { return (n <= 1) ? k : log2(n >> 1, k + 1); }
// Returns the size as a multiple of sizeof(void *), rounded up.

View File

@@ -936,11 +936,8 @@ protected:
std::free(const_cast<char *>(arg.descr));
}
}
// During finalization, default arg values may already be freed by GC.
if (!detail::py_is_finalizing()) {
for (auto &arg : rec->args) {
arg.value.dec_ref();
}
for (auto &arg : rec->args) {
arg.value.dec_ref();
}
if (rec->def) {
std::free(const_cast<char *>(rec->def->ml_doc));
@@ -1435,12 +1432,6 @@ PYBIND11_NAMESPACE_BEGIN(function_record_PyTypeObject_methods)
// This implementation needs the definition of `class cpp_function`.
inline void tp_dealloc_impl(PyObject *self) {
// Skip dealloc during finalization — GC may have already freed objects
// reachable from the function record (e.g. default arg values), causing
// use-after-free in destruct().
if (detail::py_is_finalizing()) {
return;
}
// Save type before PyObject_Free invalidates self.
auto *type = Py_TYPE(self);
auto *py_func_rec = reinterpret_cast<function_record_PyObject *>(self);
@@ -2687,19 +2678,41 @@ public:
if (rec_fget) {
char *doc_prev = rec_fget->doc; /* 'extra' field may include a property-specific
documentation string */
auto args_before = rec_fget->args.size();
detail::process_attributes<Extra...>::init(extra..., rec_fget);
if (rec_fget->doc && rec_fget->doc != doc_prev) {
std::free(doc_prev);
rec_fget->doc = PYBIND11_COMPAT_STRDUP(rec_fget->doc);
}
// Args added by process_attributes (e.g. "self" via is_method + pos_only/kw_only)
// need their strings strdup'd: initialize_generic's strdup loop already ran during
// cpp_function construction, so it won't process these late additions. Without this,
// destruct() would call free() on string literals. See gh-5976.
for (auto i = args_before; i < rec_fget->args.size(); ++i) {
if (rec_fget->args[i].name) {
rec_fget->args[i].name = PYBIND11_COMPAT_STRDUP(rec_fget->args[i].name);
}
if (rec_fget->args[i].descr) {
rec_fget->args[i].descr = PYBIND11_COMPAT_STRDUP(rec_fget->args[i].descr);
}
}
}
if (rec_fset) {
char *doc_prev = rec_fset->doc;
auto args_before = rec_fset->args.size();
detail::process_attributes<Extra...>::init(extra..., rec_fset);
if (rec_fset->doc && rec_fset->doc != doc_prev) {
std::free(doc_prev);
rec_fset->doc = PYBIND11_COMPAT_STRDUP(rec_fset->doc);
}
for (auto i = args_before; i < rec_fset->args.size(); ++i) {
if (rec_fset->args[i].name) {
rec_fset->args[i].name = PYBIND11_COMPAT_STRDUP(rec_fset->args[i].name);
}
if (rec_fset->args[i].descr) {
rec_fset->args[i].descr = PYBIND11_COMPAT_STRDUP(rec_fset->args[i].descr);
}
}
if (!rec_active) {
rec_active = rec_fset;
}

View File

@@ -172,6 +172,7 @@ set(PYBIND11_TEST_FILES
test_scoped_critical_section
test_sequences_and_iterators
test_smart_ptr
test_standalone_enum_module.py
test_stl
test_stl_binders
test_tagbased_polymorphic
@@ -249,6 +250,7 @@ tests_extra_targets("test_exceptions.py" "cross_module_interleaved_error_already
tests_extra_targets("test_gil_scoped.py" "cross_module_gil_utils")
tests_extra_targets("test_cpp_conduit.py"
"exo_planet_pybind11;exo_planet_c_api;home_planet_very_lonely_traveler")
tests_extra_targets("test_standalone_enum_module.py" "standalone_enum_module")
set(PYBIND11_EIGEN_REPO
"https://gitlab.com/libeigen/eigen.git"

View File

@@ -0,0 +1,13 @@
// Copyright (c) 2026 The pybind Community.
#include <pybind11/pybind11.h>
namespace standalone_enum_module_ns {
enum SomeEnum {};
} // namespace standalone_enum_module_ns
using namespace standalone_enum_module_ns;
PYBIND11_MODULE(standalone_enum_module, m) { // Added in PR #6015
pybind11::enum_<SomeEnum> some_enum_wrapper(m, "SomeEnum");
}

View File

@@ -0,0 +1,18 @@
from __future__ import annotations
import os
import env
def test_enum_import_exit_no_crash():
# Added in PR #6015. Modeled after reproducer under issue #5976
env.check_script_success_in_subprocess(
f"""
import sys
sys.path.insert(0, {os.path.dirname(env.__file__)!r})
import standalone_enum_module as m
assert m.SomeEnum.__class__.__name__ == "pybind11_type"
""",
rerun=1,
)

View File

@@ -4,13 +4,18 @@
#pragma once
#include <pybind11/detail/pybind11_namespace_macros.h>
#include <catch.hpp>
#define PYBIND11_CATCH2_SKIP_IF(condition, reason) \
do { \
PYBIND11_WARNING_PUSH \
PYBIND11_WARNING_DISABLE_MSVC(4127) \
if (condition) { \
Catch::cout() << "[ SKIPPED ] " << (reason) << '\n'; \
Catch::cout().flush(); \
return; \
} \
PYBIND11_WARNING_POP \
} while (0)

View File

@@ -5,6 +5,8 @@
// catch 2.0.1; this should be fixed in the next catch release after 2.0.1).
PYBIND11_WARNING_DISABLE_MSVC(4996)
#include "catch_skip.h"
#include <catch.hpp>
#include <cstdlib>
#include <fstream>
@@ -84,6 +86,14 @@ PYBIND11_EMBEDDED_MODULE(trampoline_module, m) {
.def("func", &test_override_cache_helper::func);
}
enum class SomeEnum { value1, value2 }; // Added in PR #6015
PYBIND11_EMBEDDED_MODULE(enum_module, m, py::multiple_interpreters::per_interpreter_gil()) {
py::enum_<SomeEnum>(m, "SomeEnum")
.value("value1", SomeEnum::value1)
.value("value2", SomeEnum::value2);
}
PYBIND11_EMBEDDED_MODULE(throw_exception, ) { throw std::runtime_error("C++ Error"); }
PYBIND11_EMBEDDED_MODULE(throw_error_already_set, ) {
@@ -343,6 +353,24 @@ TEST_CASE("Restart the interpreter") {
REQUIRE(py_widget.attr("the_message").cast<std::string>() == "Hello after restart");
}
TEST_CASE("Enum module survives restart") { // Added in PR #6015
// Regression test for gh-5976: py::enum_ uses def_property_static, which
// calls process_attributes::init after initialize_generic's strdup loop,
// leaving arg names as string literals. Without the fix, destruct() would
// call free() on those literals during interpreter finalization.
PYBIND11_CATCH2_SKIP_IF(PY_MAJOR_VERSION == 3 && PY_MINOR_VERSION == 12,
"Pre-existing crash in enum cleanup during finalize on Python 3.12");
auto enum_mod = py::module_::import("enum_module");
REQUIRE(enum_mod.attr("SomeEnum").attr("value1").attr("name").cast<std::string>() == "value1");
py::finalize_interpreter();
py::initialize_interpreter();
enum_mod = py::module_::import("enum_module");
REQUIRE(enum_mod.attr("SomeEnum").attr("value2").attr("name").cast<std::string>() == "value2");
}
TEST_CASE("Execution frame") {
// When the interpreter is embedded, there is no execution frame, but `py::exec`
// should still function by using reasonable globals: `__main__.__dict__`.