squash-merge smart_holder branch into master (#5542)

* Pure `git merge --squash smart_holder` (no manual interventions). * Remove ubench/ directory. * Remove include/pybind11/smart_holder.h * [ci skip] smart_ptrs.rst updates [WIP/unfinished] * [ci skip] smart_ptrs.rst updates continued; also updating classes.rst, advanced/classes.rst * Remove README_smart_holder.rst * Restore original README.rst from master * [ci skip] Minimal change to README.rst, to leave a hint that this is pybind11v3 * [ci skip] Work in ChatGPT suggestions. * Change macro name to PYBIND11_RUN_TESTING_WITH_SMART_HOLDER_AS_DEFAULT_BUT_NEVER_USE_IN_PRODUCTION_PLEASE * Add a note pointing to the holder reinterpret_cast. * Incorporate suggestion by @virtuald: https://github.com/pybind/pybind11/pull/5542#discussion_r1967000989 * Systematically change most py::class_ to py::classh under docs/ * Remove references to README_smart_holder.rst This should have been part of commit eb550d03d3. * [ci skip] Fix minor oversight (``class_`` -> ``py::class_``) noticed by chance. * [ci skip] Resolve suggestion by @virtuald https://github.com/pybind/pybind11/pull/5542#discussion_r1969940605 * [ci skip] Apply suggestions by @timohl (thanks!) * https://github.com/pybind/pybind11/pull/5542#discussion_r1970714551 * https://github.com/pybind/pybind11/pull/5542#discussion_r1971315329 * https://github.com/pybind/pybind11/pull/5542#discussion_r1971322821 * Replace `classh : class_` inhertance with `using`, as suggested by @henryiii https://github.com/pybind/pybind11/pull/5542#issuecomment-2689034104 * Revert "Systematically change most py::class_ to py::classh under docs/" This reverts commit ac9d31e13f. * docs: focus on py::smart_holder instead of py::classh Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> * Restore minor general fixes that got lost when ac9d31e13f was reverted. * Remove `- smart_holder` from list of branches in all .github/workflows * Extend classh note to explain whitespace noise motivation. * Suggest `py::smart_holder` for "most situations for safety" * Add back PYBIND11_HAS_INTERNALS_WITH_SMART_HOLDER_SUPPORT This define was * introduced with https://github.com/pybind/pybind11/pull/5286 * removed with https://github.com/pybind/pybind11/pull/5531 It is has been in use here: * f02a2b7653/pybind11_protobuf/native_proto_caster.h (L89-L101) Currently pybind11 unit tests for the two holder caster backwards compatibility traits * `copyable_holder_caster_shared_ptr_with_smart_holder_support_enabled` * `move_only_holder_caster_unique_ptr_with_smart_holder_support_enabled` are missing. * Add py::trampoline_self_life_support to all trampoline examples under docs/. Address suggestion by @timohl: * https://github.com/pybind/pybind11/pull/5542#issuecomment-2686452062 Add to the "please think twice" note: the overhead for safety is likely in the noise. Also fix a two-fold inconsistency introduced by revert-commit 1e646c91b4: 1. py::trampoline_self_life_support is mentioned in a note, but is missing in the example right before. 2. The section starting with To enable safely passing a ``std::unique_ptr`` to a trampoline object between is obsolete. * Fix whitespace accident (indentation) introduced with 1e646c91b4 Apparently the mis-indentation was introduced when resolving merge conflicts for what became 1e646c91b4 * WHITESPACE CHANGES ONLY in README.rst (list of people that made significant contributions) * Add Ethan Steinberg to list of people that made significant contributions (for completeness, unrelated to smart_holder work). * [ci skip] Add to list of people that made significant contributions: major and/or influential contributors to smart_holder branch * #2904 by @rhaschke was merged on Mar 16, 2021 * #3012 by @rhaschke was merged on May 28, 2021 * #3039 by @jakobandersen was merged on Jun 29, 2021 * #3048 by @Skylion007 was merged on Jun 18, 2021 * #3588 by @virtuald was merged on Jan 3, 2022 * #3633 by @wangxf123456 was merged on Jan 25, 2022 * #3635 by @virtuald was merged on Jan 26, 2022 * #3645 by @wangxf123456 was merged on Jan 25, 2022 * #3796 by @wangxf123456 was merged on Mar 10, 2022 * #3807 by @wangxf123456 was merged on Mar 18, 2022 * #3838 by @wangxf123456 was merged on Apr 15, 2022 * #3929 by @tomba was merged on May 7, 2022 * #4031 by @wangxf123456 was merged on Jun 27, 2022 * #4343 by @wangxf123456 was merged on Nov 18, 2022 * #4381 by @wangxf123456 was merged on Dec 5, 2022 * #4539 by @wangxf123456 was merged on Feb 28, 2023 * #4609 by @wangxf123456 was merged on Apr 6, 2023 * #4775 by @wangxf123456 was merged on Aug 3, 2023 * #4921 by @iwanders was merged on Nov 7, 2023 * #4924 by @iwanders was merged on Nov 6, 2023 * #5401 by @msimacek was merged on Oct 8, 2024 Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com> Co-authored-by: Ivor Wanders <iwanders@users.noreply.github.com> Co-authored-by: Jakob Lykke Andersen <Jakob@caput.dk> Co-authored-by: Michael Šimáček <michael.simacek@oracle.com> Co-authored-by: Robert Haschke <rhaschke@users.noreply.github.com> Co-authored-by: Tomi Valkeinen <tomi.valkeinen@iki.fi> Co-authored-by: Xiaofei Wang <6218006+wangxf123456@users.noreply.github.com> --------- Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com> Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com> Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com> Co-authored-by: Ivor Wanders <iwanders@users.noreply.github.com> Co-authored-by: Jakob Lykke Andersen <Jakob@caput.dk> Co-authored-by: Michael Šimáček <michael.simacek@oracle.com> Co-authored-by: Robert Haschke <rhaschke@users.noreply.github.com> Co-authored-by: Tomi Valkeinen <tomi.valkeinen@iki.fi> Co-authored-by: Xiaofei Wang <6218006+wangxf123456@users.noreply.github.com>
2026-04-19 22:39:09 +00:00 · 2025-03-05 12:40:53 -08:00
parent d8565ac731
commit 2943a27a14
66 changed files with 5533 additions and 169 deletions
--- a/docs/advanced/cast/stl.rst
+++ b/docs/advanced/cast/stl.rst
@@ -169,8 +169,8 @@ macro must be specified at the top level (and outside of any namespaces), since
 it adds a template instantiation of ``type_caster``. If your binding code consists of
 multiple compilation units, it must be present in every file (typically via a
 common header) preceding any usage of ``std::vector<int>``. Opaque types must
-also have a corresponding ``class_`` declaration to associate them with a name
-in Python, and to define a set of available operations, e.g.:
+also have a corresponding ``py::class_`` declaration to associate them with a
+name in Python, and to define a set of available operations, e.g.:

 .. code-block:: cpp

--- a/docs/advanced/classes.rst
+++ b/docs/advanced/classes.rst
@@ -64,7 +64,7 @@ helper class that is defined as follows:

 .. code-block:: cpp

-    class PyAnimal : public Animal {
+    class PyAnimal : public Animal, py::trampoline_self_life_support {
    public:
        /* Inherit the constructors */
        using Animal::Animal;
@@ -80,6 +80,18 @@ helper class that is defined as follows:
        }
    };

+The ``py::trampoline_self_life_support`` base class is needed to ensure
+that a ``std::unique_ptr`` can safely be passed between Python and C++. To
+steer clear of notorious pitfalls (e.g. inheritance slicing), it is best
+practice to always use the base class, in combination with
+``py::smart_holder``.
+
+.. note::
+    For completeness, the base class has no effect if a holder other than
+    ``py::smart_holder`` used, including the default ``std::unique_ptr<T>``.
+    Please think twice, though, the pitfalls are very real, and the overhead
+    for using the safer ``py::smart_holder`` is very likely to be in the noise.
+
 The macro :c:macro:`PYBIND11_OVERRIDE_PURE` should be used for pure virtual
 functions, and :c:macro:`PYBIND11_OVERRIDE` should be used for functions which have
 a default implementation.  There are also two alternate macros
@@ -95,18 +107,18 @@ The binding code also needs a few minor adaptations (highlighted):
    :emphasize-lines: 2,3

    PYBIND11_MODULE(example, m) {
-        py::class_<Animal, PyAnimal /* <--- trampoline*/>(m, "Animal")
+        py::class_<Animal, PyAnimal /* <--- trampoline */, py::smart_holder>(m, "Animal")
            .def(py::init<>())
            .def("go", &Animal::go);

-        py::class_<Dog, Animal>(m, "Dog")
+        py::class_<Dog, Animal, py::smart_holder>(m, "Dog")
            .def(py::init<>());

        m.def("call_go", &call_go);
    }

 Importantly, pybind11 is made aware of the trampoline helper class by
-specifying it as an extra template argument to :class:`class_`. (This can also
+specifying it as an extra template argument to ``py::class_``. (This can also
 be combined with other template arguments such as a custom holder type; the
 order of template types does not matter).  Following this, we are able to
 define a constructor as usual.
@@ -116,9 +128,9 @@ Bindings should be made against the actual class, not the trampoline helper clas
 .. code-block:: cpp
    :emphasize-lines: 3

-    py::class_<Animal, PyAnimal /* <--- trampoline*/>(m, "Animal");
+    py::class_<Animal, PyAnimal /* <--- trampoline */, py::smart_holder>(m, "Animal");
        .def(py::init<>())
-        .def("go", &PyAnimal::go); /* <--- THIS IS WRONG, use &Animal::go */
+        .def("go", &Animal::go); /* <--- DO NOT USE &PyAnimal::go HERE */

 Note, however, that the above is sufficient for allowing python classes to
 extend ``Animal``, but not ``Dog``: see :ref:`virtual_and_inheritance` for the
@@ -244,13 +256,13 @@ override the ``name()`` method):

 .. code-block:: cpp

-    class PyAnimal : public Animal {
+    class PyAnimal : public Animal, py::trampoline_self_life_support {
    public:
        using Animal::Animal; // Inherit constructors
        std::string go(int n_times) override { PYBIND11_OVERRIDE_PURE(std::string, Animal, go, n_times); }
        std::string name() override { PYBIND11_OVERRIDE(std::string, Animal, name, ); }
    };
-    class PyDog : public Dog {
+    class PyDog : public Dog, py::trampoline_self_life_support {
    public:
        using Dog::Dog; // Inherit constructors
        std::string go(int n_times) override { PYBIND11_OVERRIDE(std::string, Dog, go, n_times); }
@@ -272,7 +284,7 @@ declare or override any virtual methods itself:
 .. code-block:: cpp

    class Husky : public Dog {};
-    class PyHusky : public Husky {
+    class PyHusky : public Husky, py::trampoline_self_life_support {
    public:
        using Husky::Husky; // Inherit constructors
        std::string go(int n_times) override { PYBIND11_OVERRIDE_PURE(std::string, Husky, go, n_times); }
@@ -287,13 +299,15 @@ follows:

 .. code-block:: cpp

-    template <class AnimalBase = Animal> class PyAnimal : public AnimalBase {
+    template <class AnimalBase = Animal>
+    class PyAnimal : public AnimalBase, py::trampoline_self_life_support {
    public:
        using AnimalBase::AnimalBase; // Inherit constructors
        std::string go(int n_times) override { PYBIND11_OVERRIDE_PURE(std::string, AnimalBase, go, n_times); }
        std::string name() override { PYBIND11_OVERRIDE(std::string, AnimalBase, name, ); }
    };
-    template <class DogBase = Dog> class PyDog : public PyAnimal<DogBase> {
+    template <class DogBase = Dog>
+    class PyDog : public PyAnimal<DogBase>, py::trampoline_self_life_support {
    public:
        using PyAnimal<DogBase>::PyAnimal; // Inherit constructors
        // Override PyAnimal's pure virtual go() with a non-pure one:
@@ -311,9 +325,9 @@ The classes are then registered with pybind11 using:

 .. code-block:: cpp

-    py::class_<Animal, PyAnimal<>> animal(m, "Animal");
-    py::class_<Dog, Animal, PyDog<>> dog(m, "Dog");
-    py::class_<Husky, Dog, PyDog<Husky>> husky(m, "Husky");
+    py::class_<Animal, PyAnimal<>, py::smart_holder> animal(m, "Animal");
+    py::class_<Dog, Animal, PyDog<>, py::smart_holder> dog(m, "Dog");
+    py::class_<Husky, Dog, PyDog<Husky>, py::smart_holder> husky(m, "Husky");
    // ... add animal, dog, husky definitions

 Note that ``Husky`` did not require a dedicated trampoline template class at
@@ -499,12 +513,12 @@ an alias:
        // ...
        virtual ~Example() = default;
    };
-    class PyExample : public Example {
+    class PyExample : public Example, py::trampoline_self_life_support {
    public:
        using Example::Example;
        PyExample(Example &&base) : Example(std::move(base)) {}
    };
-    py::class_<Example, PyExample>(m, "Example")
+    py::class_<Example, PyExample, py::smart_holder>(m, "Example")
        // Returns an Example pointer.  If a PyExample is needed, the Example
        // instance will be moved via the extra constructor in PyExample, above.
        .def(py::init([]() { return new Example(); }))
@@ -550,9 +564,10 @@ pybind11. The underlying issue is that the ``std::unique_ptr`` holder type that
 is responsible for managing the lifetime of instances will reference the
 destructor even if no deallocations ever take place. In order to expose classes
 with private or protected destructors, it is possible to override the holder
-type via a holder type argument to ``class_``. Pybind11 provides a helper class
-``py::nodelete`` that disables any destructor invocations. In this case, it is
-crucial that instances are deallocated on the C++ side to avoid memory leaks.
+type via a holder type argument to ``py::class_``. Pybind11 provides a helper
+class ``py::nodelete`` that disables any destructor invocations. In this case,
+it is crucial that instances are deallocated on the C++ side to avoid memory
+leaks.

 .. code-block:: cpp

@@ -871,7 +886,7 @@ Multiple Inheritance

 pybind11 can create bindings for types that derive from multiple base types
 (aka. *multiple inheritance*). To do so, specify all bases in the template
-arguments of the ``class_`` declaration:
+arguments of the ``py::class_`` declaration:

 .. code-block:: cpp

@@ -946,11 +961,11 @@ because of conflicting definitions on the external type:
    // dogs.cpp

    // Binding for external library class:
-    py::class<pets::Pet>(m, "Pet")
+    py::class_<pets::Pet>(m, "Pet")
        .def("name", &pets::Pet::name);

    // Binding for local extension class:
-    py::class<Dog, pets::Pet>(m, "Dog")
+    py::class_<Dog, pets::Pet>(m, "Dog")
        .def(py::init<std::string>());

 .. code-block:: cpp
@@ -958,11 +973,11 @@ because of conflicting definitions on the external type:
    // cats.cpp, in a completely separate project from the above dogs.cpp.

    // Binding for external library class:
-    py::class<pets::Pet>(m, "Pet")
+    py::class_<pets::Pet>(m, "Pet")
        .def("get_name", &pets::Pet::name);

    // Binding for local extending class:
-    py::class<Cat, pets::Pet>(m, "Cat")
+    py::class_<Cat, pets::Pet>(m, "Cat")
        .def(py::init<std::string>());

 .. code-block:: pycon
@@ -980,13 +995,13 @@ the ``py::class_`` constructor:
 .. code-block:: cpp

    // Pet binding in dogs.cpp:
-    py::class<pets::Pet>(m, "Pet", py::module_local())
+    py::class_<pets::Pet>(m, "Pet", py::module_local())
        .def("name", &pets::Pet::name);

 .. code-block:: cpp

    // Pet binding in cats.cpp:
-    py::class<pets::Pet>(m, "Pet", py::module_local())
+    py::class_<pets::Pet>(m, "Pet", py::module_local())
        .def("get_name", &pets::Pet::name);

 This makes the Python-side ``dogs.Pet`` and ``cats.Pet`` into distinct classes,
@@ -1104,7 +1119,7 @@ described trampoline:
        virtual int foo() const { return 42; }
    };

-    class Trampoline : public A {
+    class Trampoline : public A, py::trampoline_self_life_support {
    public:
        int foo() const override { PYBIND11_OVERRIDE(int, A, foo, ); }
    };
@@ -1114,7 +1129,7 @@ described trampoline:
        using A::foo;
    };

-    py::class_<A, Trampoline>(m, "A") // <-- `Trampoline` here
+    py::class_<A, Trampoline, py::smart_holder>(m, "A") // <-- `Trampoline` here
        .def("foo", &Publicist::foo); // <-- `Publicist` here, not `Trampoline`!

 Binding final classes
@@ -1195,7 +1210,7 @@ but once again each instantiation must be explicitly specified:
        T fn(V v);
    };

-    py::class<MyClass<int>>(m, "MyClassT")
+    py::class_<MyClass<int>>(m, "MyClassT")
        .def("fn", &MyClass<int>::fn<std::string>);

 Custom automatic downcasters
--- a/docs/advanced/misc.rst
+++ b/docs/advanced/misc.rst
@@ -80,7 +80,7 @@ could be realized as follows (important changes highlighted):
 .. code-block:: cpp
    :emphasize-lines: 8,30,31

-    class PyAnimal : public Animal {
+    class PyAnimal : public Animal, py::trampoline_self_life_support {
    public:
        /* Inherit the constructors */
        using Animal::Animal;
@@ -98,12 +98,12 @@ could be realized as follows (important changes highlighted):
    };

    PYBIND11_MODULE(example, m) {
-        py::class_<Animal, PyAnimal> animal(m, "Animal");
+        py::class_<Animal, PyAnimal, py::smart_holder> animal(m, "Animal");
        animal
            .def(py::init<>())
            .def("go", &Animal::go);

-        py::class_<Dog>(m, "Dog", animal)
+        py::class_<Dog, py::smart_holder>(m, "Dog", animal)
            .def(py::init<>());

        m.def("call_go", [](Animal *animal) -> std::string {
@@ -188,7 +188,7 @@ from Section :ref:`inheritance`.
 Suppose now that ``Pet`` bindings are defined in a module named ``basic``,
 whereas the ``Dog`` bindings are defined somewhere else. The challenge is of
 course that the variable ``pet`` is not available anymore though it is needed
-to indicate the inheritance relationship to the constructor of ``class_<Dog>``.
+to indicate the inheritance relationship to the constructor of ``py::class_<Dog>``.
 However, it can be acquired as follows:

 .. code-block:: cpp
@@ -200,7 +200,7 @@ However, it can be acquired as follows:
        .def("bark", &Dog::bark);

 Alternatively, you can specify the base class as a template parameter option to
-``class_``, which performs an automated lookup of the corresponding Python
+``py::class_``, which performs an automated lookup of the corresponding Python
 type. Like the above code, however, this also requires invoking the ``import``
 function once to ensure that the pybind11 binding code of the module ``basic``
 has been executed:
--- a/docs/advanced/smart_ptrs.rst
+++ b/docs/advanced/smart_ptrs.rst
@@ -1,11 +1,70 @@
-Smart pointers
-##############
+Smart pointers & ``py::class_``
+###############################

-std::unique_ptr
-===============
+The binding generator for classes, ``py::class_``, can be passed a template
+type that denotes a special *holder* type that is used to manage references to
+the object. If no such holder type template argument is given, the default for
+a type ``T`` is ``std::unique_ptr<T>``.

-Given a class ``Example`` with Python bindings, it's possible to return
-instances wrapped in C++11 unique pointers, like so
+.. note::
+
+    A ``py::class_`` for a given C++ type ``T`` — and all its derived types —
+    can only use a single holder type.
+
+
+.. _smart_holder:
+
+``py::smart_holder``
+====================
+
+Starting with pybind11v3, ``py::smart_holder`` is built into pybind11. It is
+the recommended ``py::class_`` holder for most situations. However, for
+backward compatibility it is **not** the default holder, and there are no
+plans to make it the default holder in the future.
+
+It is extremely easy to use the safer and more versatile ``py::smart_holder``:
+simply add ``py::smart_holder`` to ``py::class_``:
+
+* ``py::class_<T>`` to
+
+* ``py::class_<T, py::smart_holder>``.
+
+.. note::
+
+    A shorthand, ``py::classh<T>``, is provided for
+    ``py::class_<T, py::smart_holder>``. The ``h`` in ``py::classh`` stands
+    for **smart_holder** but is shortened for brevity, ensuring it has the
+    same number of characters as ``py::class_``. This design choice facilitates
+    easy experimentation with ``py::smart_holder`` without introducing
+    distracting whitespace noise in diffs.
+
+The ``py::smart_holder`` functionality includes the following:
+
+* Support for **two-way** Python/C++ conversions for both
+  ``std::unique_ptr<T>`` and ``std::shared_ptr<T>`` **simultaneously**.
+
+* Passing a Python object back to C++ via ``std::unique_ptr<T>``, safely
+  **disowning** the Python object.
+
+* Safely passing "trampoline" objects (objects with C++ virtual function
+  overrides implemented in Python, see :ref:`overriding_virtuals`) via
+  ``std::unique_ptr<T>`` or ``std::shared_ptr<T>`` back to C++:
+  associated Python objects are automatically kept alive for the lifetime
+  of the smart-pointer.
+
+* Full support for ``std::enable_shared_from_this`` (`cppreference
+  <http://en.cppreference.com/w/cpp/memory/enable_shared_from_this>`_).
+
+
+``std::unique_ptr``
+===================
+
+This is the default ``py::class_`` holder and works as expected in
+most situations. However, handling base-and-derived classes involves a
+``reinterpret_cast``, which is, strictly speaking, undefined behavior.
+Also note that the ``std::unique_ptr`` holder only supports passing a
+``std::unique_ptr`` from C++ to Python, but not the other way around.
+For example, the following code works as expected with ``py::class_<Example>``:

 .. code-block:: cpp

@@ -15,112 +74,50 @@ instances wrapped in C++11 unique pointers, like so

    m.def("create_example", &create_example);

-In other words, there is nothing special that needs to be done. While returning
-unique pointers in this way is allowed, it is *illegal* to use them as function
-arguments. For instance, the following function signature cannot be processed
-by pybind11.
+However, this will fail with ``py::class_<Example>`` (but works with
+``py::class_<Example, py::smart_holder>``):

 .. code-block:: cpp

    void do_something_with_example(std::unique_ptr<Example> ex) { ... }

-The above signature would imply that Python needs to give up ownership of an
-object that is passed to this function, which is generally not possible (for
-instance, the object might be referenced elsewhere).
+.. note::

-std::shared_ptr
-===============
+    The ``reinterpret_cast`` mentioned above is `here
+    <https://github.com/pybind/pybind11/blob/30eb39ed79d1e2eeff15219ac00773034300a5e6/include/pybind11/cast.h#L235>`_.
+    For completeness: The same cast is also applied to ``py::smart_holder``,
+    but that is safe, because ``py::smart_holder`` is not templated.

-The binding generator for classes, :class:`class_`, can be passed a template
-type that denotes a special *holder* type that is used to manage references to
-the object.  If no such holder type template argument is given, the default for
-a type named ``Type`` is ``std::unique_ptr<Type>``, which means that the object
-is deallocated when Python's reference count goes to zero.

-It is possible to switch to other types of reference counting wrappers or smart
-pointers, which is useful in codebases that rely on them. For instance, the
-following snippet causes ``std::shared_ptr`` to be used instead.
+``std::shared_ptr``
+===================
+
+It is possible to use ``std::shared_ptr`` as the holder, for example:

 .. code-block:: cpp

-    py::class_<Example, std::shared_ptr<Example> /* <- holder type */> obj(m, "Example");
+    py::class_<Example, std::shared_ptr<Example> /* <- holder type */>(m, "Example");

-Note that any particular class can only be associated with a single holder type.
+Compared to using ``py::class_<Example, py::smart_holder>``, there are two noteworthy disadvantages:

-One potential stumbling block when using holder types is that they need to be
-applied consistently. Can you guess what's broken about the following binding
-code?
+* Because a ``py::class_`` for a given C++ type ``T`` can only use a
+  single holder type, ``std::unique_ptr<T>`` cannot even be passed from C++
+  to Python. This will become apparent only at runtime, often through a
+  segmentation fault.

-.. code-block:: cpp
+* Similar to the ``std::unique_ptr`` holder, the handling of base-and-derived
+  classes involves a ``reinterpret_cast`` that has strictly speaking undefined
+  behavior, although it works as expected in most situations.

-    class Child { };
-
-    class Parent {
-    public:
-       Parent() : child(std::make_shared<Child>()) { }
-       Child *get_child() { return child.get(); }  /* Hint: ** DON'T DO THIS ** */
-    private:
-        std::shared_ptr<Child> child;
-    };
-
-    PYBIND11_MODULE(example, m) {
-        py::class_<Child, std::shared_ptr<Child>>(m, "Child");
-
-        py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
-           .def(py::init<>())
-           .def("get_child", &Parent::get_child);
-    }
-
-The following Python code will cause undefined behavior (and likely a
-segmentation fault).
-
-.. code-block:: python
-
-   from example import Parent
-
-   print(Parent().get_child())
-
-The problem is that ``Parent::get_child()`` returns a pointer to an instance of
-``Child``, but the fact that this instance is already managed by
-``std::shared_ptr<...>`` is lost when passing raw pointers. In this case,
-pybind11 will create a second independent ``std::shared_ptr<...>`` that also
-claims ownership of the pointer. In the end, the object will be freed **twice**
-since these shared pointers have no way of knowing about each other.
-
-There are two ways to resolve this issue:
-
-1. For types that are managed by a smart pointer class, never use raw pointers
-   in function arguments or return values. In other words: always consistently
-   wrap pointers into their designated holder types (such as
-   ``std::shared_ptr<...>``). In this case, the signature of ``get_child()``
-   should be modified as follows:
-
-.. code-block:: cpp
-
-    std::shared_ptr<Child> get_child() { return child; }
-
-2. Adjust the definition of ``Child`` by specifying
-   ``std::enable_shared_from_this<T>`` (see cppreference_ for details) as a
-   base class. This adds a small bit of information to ``Child`` that allows
-   pybind11 to realize that there is already an existing
-   ``std::shared_ptr<...>`` and communicate with it. In this case, the
-   declaration of ``Child`` should look as follows:
-
-.. _cppreference: http://en.cppreference.com/w/cpp/memory/enable_shared_from_this
-
-.. code-block:: cpp
-
-    class Child : public std::enable_shared_from_this<Child> { };

 .. _smart_pointers:

 Custom smart pointers
 =====================

-pybind11 supports ``std::unique_ptr`` and ``std::shared_ptr`` right out of the
-box. For any other custom smart pointer, transparent conversions can be enabled
-using a macro invocation similar to the following. It must be declared at the
-top namespace level before any binding code:
+For custom smart pointers (e.g. ``c10::intrusive_ptr`` in pytorch), transparent
+conversions can be enabled using a macro invocation similar to the following.
+It must be declared at the top namespace level before any binding code:

 .. code-block:: cpp

@@ -167,8 +164,70 @@ specialized:
 The above specialization informs pybind11 that the custom ``SmartPtr`` class
 provides ``.get()`` functionality via ``.getPointer()``.

+.. note::
+
+    The two noteworthy disadvantages mentioned under the ``std::shared_ptr``
+    section apply similarly to custom smart pointer holders, but there is no
+    established safe alternative in this case.
+
 .. seealso::

    The file :file:`tests/test_smart_ptr.cpp` contains a complete example
    that demonstrates how to work with custom reference-counting holder types
    in more detail.
+
+
+Be careful not to accidentally undermine automatic lifetime management
+======================================================================
+
+``py::class_``-wrapped objects automatically manage the lifetime of the
+wrapped C++ object, in collaboration with the chosen holder type.
+When wrapping C++ functions involving raw pointers, care needs to be taken
+to not inadvertently transfer ownership, resulting in multiple Python
+objects acting as owners, causing heap-use-after-free or double-free errors.
+For example:
+
+.. code-block:: cpp
+
+    class Child { };
+
+    class Parent {
+    public:
+       Parent() : child(std::make_shared<Child>()) { }
+       Child *get_child() { return child.get(); }  /* DANGER */
+    private:
+        std::shared_ptr<Child> child;
+    };
+
+    PYBIND11_MODULE(example, m) {
+        py::class_<Child, std::shared_ptr<Child>>(m, "Child");
+
+        py::class_<Parent, std::shared_ptr<Parent>>(m, "Parent")
+           .def(py::init<>())
+           .def("get_child", &Parent::get_child);  /* PROBLEM */
+    }
+
+The following Python code leads to undefined behavior, likely resulting in
+a segmentation fault.
+
+.. code-block:: python
+
+   from example import Parent
+
+   print(Parent().get_child())
+
+Part of the ``/* PROBLEM */`` here is that pybind11 falls back to using
+``return_value_policy::take_ownership`` as the default (see
+:ref:`return_value_policies`). The fact that the ``Child`` instance is
+already managed by ``std::shared_ptr<Child>`` is lost. Therefore pybind11
+will create a second independent ``std::shared_ptr<Child>`` that also
+claims ownership of the pointer, eventually leading to heap-use-after-free
+or double-free errors.
+
+There are various ways to resolve this issue, either by changing
+the ``Child`` or ``Parent`` C++ implementations (e.g. using
+``std::enable_shared_from_this<Child>`` as a base class for
+``Child``, or adding a member function to ``Parent`` that returns
+``std::shared_ptr<Child>``), or if that is not feasible, by using
+``return_value_policy::reference_internal``. What is the best approach
+depends on the exact situation.
--- a/docs/classes.rst
+++ b/docs/classes.rst
@@ -34,11 +34,18 @@ The binding code for ``Pet`` looks as follows:
            .def("getName", &Pet::getName);
    }

-:class:`class_` creates bindings for a C++ *class* or *struct*-style data
+``py::class_`` creates bindings for a C++ *class* or *struct*-style data
 structure. :func:`init` is a convenience function that takes the types of a
 constructor's parameters as template arguments and wraps the corresponding
-constructor (see the :ref:`custom_constructors` section for details). An
-interactive Python session demonstrating this example is shown below:
+constructor (see the :ref:`custom_constructors` section for details).
+
+.. note::
+
+    Starting with pybind11v3, it is recommended to include `py::smart_holder`
+    in most situations for safety, especially if you plan to support conversions
+    to C++ smart pointers. See :ref:`smart_holder` for more information.
+
+An interactive Python session demonstrating this example is shown below:

 .. code-block:: pycon

@@ -258,7 +265,7 @@ inheritance relationship:

 There are two different ways of indicating a hierarchical relationship to
 pybind11: the first specifies the C++ base class as an extra template
-parameter of the :class:`class_`:
+parameter of the ``py::class_``:

 .. code-block:: cpp

@@ -272,7 +279,7 @@ parameter of the :class:`class_`:
        .def("bark", &Dog::bark);

 Alternatively, we can also assign a name to the previously bound ``Pet``
-:class:`class_` object and reference it when binding the ``Dog`` class:
+``py::class_`` object and reference it when binding the ``Dog`` class:

 .. code-block:: cpp

@@ -498,7 +505,7 @@ The binding code for this example looks as follows:


 To ensure that the nested types ``Kind`` and ``Attributes`` are created within the scope of ``Pet``, the
-``pet`` :class:`class_` instance must be supplied to the :class:`enum_` and :class:`class_`
+``pet`` ``py::class_`` instance must be supplied to the :class:`enum_` and ``py::class_``
 constructor. The :func:`enum_::export_values` function exports the enum entries
 into the parent scope, which should be skipped for newer C++11-style strongly
 typed enums.
--- a/docs/reference.rst
+++ b/docs/reference.rst
@@ -68,8 +68,8 @@ Convenience functions converting to Python types

 .. _extras:

-Passing extra arguments to ``def`` or ``class_``
-================================================
+Passing extra arguments to ``def`` or ``py::class_``
+====================================================

 .. doxygengroup:: annotations
    :members: