squash-merge smart_holder branch into master (#5542)

* Pure `git merge --squash smart_holder` (no manual interventions).

* Remove ubench/ directory.

* Remove include/pybind11/smart_holder.h

* [ci skip] smart_ptrs.rst updates [WIP/unfinished]

* [ci skip] smart_ptrs.rst updates continued; also updating classes.rst, advanced/classes.rst

* Remove README_smart_holder.rst

* Restore original README.rst from master

* [ci skip] Minimal change to README.rst, to leave a hint that this is pybind11v3

* [ci skip] Work in ChatGPT suggestions.

* Change macro name to PYBIND11_RUN_TESTING_WITH_SMART_HOLDER_AS_DEFAULT_BUT_NEVER_USE_IN_PRODUCTION_PLEASE

* Add a note pointing to the holder reinterpret_cast.

* Incorporate suggestion by @virtuald: https://github.com/pybind/pybind11/pull/5542#discussion_r1967000989

* Systematically change most py::class_ to py::classh under docs/

* Remove references to README_smart_holder.rst

This should have been part of commit eb550d03d3.

* [ci skip] Fix minor oversight (``class_`` -> ``py::class_``) noticed by chance.

* [ci skip] Resolve suggestion by @virtuald

https://github.com/pybind/pybind11/pull/5542#discussion_r1969940605

* [ci skip] Apply suggestions by @timohl (thanks!)

* https://github.com/pybind/pybind11/pull/5542#discussion_r1970714551
* https://github.com/pybind/pybind11/pull/5542#discussion_r1971315329
* https://github.com/pybind/pybind11/pull/5542#discussion_r1971322821

* Replace `classh : class_` inhertance with `using`, as suggested by @henryiii

https://github.com/pybind/pybind11/pull/5542#issuecomment-2689034104

* Revert "Systematically change most py::class_ to py::classh under docs/"

This reverts commit ac9d31e13f.

* docs: focus on py::smart_holder instead of py::classh

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Restore minor general fixes that got lost when ac9d31e13f was reverted.

* Remove `- smart_holder` from list of branches in all .github/workflows

* Extend classh note to explain whitespace noise motivation.

* Suggest `py::smart_holder` for "most situations for safety"

* Add back PYBIND11_HAS_INTERNALS_WITH_SMART_HOLDER_SUPPORT

This define was
* introduced with https://github.com/pybind/pybind11/pull/5286
* removed with https://github.com/pybind/pybind11/pull/5531

It is has been in use here:
* f02a2b7653/pybind11_protobuf/native_proto_caster.h (L89-L101)

Currently pybind11 unit tests for the two holder caster backwards compatibility traits

* `copyable_holder_caster_shared_ptr_with_smart_holder_support_enabled`
* `move_only_holder_caster_unique_ptr_with_smart_holder_support_enabled`

are missing.

* Add py::trampoline_self_life_support to all trampoline examples under docs/.

Address suggestion by @timohl:

* https://github.com/pybind/pybind11/pull/5542#issuecomment-2686452062

Add to the "please think twice" note: the overhead for safety is likely in the noise.

Also fix a two-fold inconsistency introduced by revert-commit 1e646c91b4:

1.

py::trampoline_self_life_support is mentioned in a note, but is missing in the example right before.

2.

The section starting with

    To enable safely passing a ``std::unique_ptr`` to a trampoline object between

is obsolete.

* Fix whitespace accident (indentation) introduced with 1e646c91b4

Apparently the mis-indentation was introduced when resolving merge conflicts for what became 1e646c91b4

* WHITESPACE CHANGES ONLY in README.rst (list of people that made significant contributions)

* Add Ethan Steinberg to list of people that made significant contributions (for completeness, unrelated to smart_holder work).

* [ci skip] Add to list of people that made significant contributions: major and/or influential contributors to smart_holder branch

* #2904 by @rhaschke was merged on Mar 16, 2021
* #3012 by @rhaschke was merged on May 28, 2021
* #3039 by @jakobandersen was merged on Jun 29, 2021
* #3048 by @Skylion007 was merged on Jun 18, 2021
* #3588 by @virtuald was merged on Jan 3, 2022
* #3633 by @wangxf123456 was merged on Jan 25, 2022
* #3635 by @virtuald was merged on Jan 26, 2022
* #3645 by @wangxf123456 was merged on Jan 25, 2022
* #3796 by @wangxf123456 was merged on Mar 10, 2022
* #3807 by @wangxf123456 was merged on Mar 18, 2022
* #3838 by @wangxf123456 was merged on Apr 15, 2022
* #3929 by @tomba was merged on May 7, 2022
* #4031 by @wangxf123456 was merged on Jun 27, 2022
* #4343 by @wangxf123456 was merged on Nov 18, 2022
* #4381 by @wangxf123456 was merged on Dec 5, 2022
* #4539 by @wangxf123456 was merged on Feb 28, 2023
* #4609 by @wangxf123456 was merged on Apr 6, 2023
* #4775 by @wangxf123456 was merged on Aug 3, 2023
* #4921 by @iwanders was merged on Nov 7, 2023
* #4924 by @iwanders was merged on Nov 6, 2023
* #5401 by @msimacek was merged on Oct 8, 2024

Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: Ivor Wanders <iwanders@users.noreply.github.com>
Co-authored-by: Jakob Lykke Andersen <Jakob@caput.dk>
Co-authored-by: Michael Šimáček <michael.simacek@oracle.com>
Co-authored-by: Robert Haschke <rhaschke@users.noreply.github.com>
Co-authored-by: Tomi Valkeinen <tomi.valkeinen@iki.fi>
Co-authored-by: Xiaofei Wang <6218006+wangxf123456@users.noreply.github.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: Ivor Wanders <iwanders@users.noreply.github.com>
Co-authored-by: Jakob Lykke Andersen <Jakob@caput.dk>
Co-authored-by: Michael Šimáček <michael.simacek@oracle.com>
Co-authored-by: Robert Haschke <rhaschke@users.noreply.github.com>
Co-authored-by: Tomi Valkeinen <tomi.valkeinen@iki.fi>
Co-authored-by: Xiaofei Wang <6218006+wangxf123456@users.noreply.github.com>

tests/pure_cpp/smart_holder_poc.h

@@ -0,0 +1,51 @@
+// Copyright (c) 2020-2024 The Pybind Development Team.
+// All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+#pragma once
+
+#include "pybind11/detail/struct_smart_holder.h"
+
+namespace pybindit {
+namespace memory {
+namespace smart_holder_poc { // Proof-of-Concept implementations.
+
+template <typename T>
+T &as_lvalue_ref(const smart_holder &hld) {
+    static const char *context = "as_lvalue_ref";
+    hld.ensure_is_populated(context);
+    hld.ensure_has_pointee(context);
+    return *hld.as_raw_ptr_unowned<T>();
+}
+
+template <typename T>
+T &&as_rvalue_ref(const smart_holder &hld) {
+    static const char *context = "as_rvalue_ref";
+    hld.ensure_is_populated(context);
+    hld.ensure_has_pointee(context);
+    return std::move(*hld.as_raw_ptr_unowned<T>());
+}
+
+template <typename T>
+T *as_raw_ptr_release_ownership(smart_holder &hld,
+                                const char *context = "as_raw_ptr_release_ownership") {
+    hld.ensure_can_release_ownership(context);
+    T *raw_ptr = hld.as_raw_ptr_unowned<T>();
+    hld.release_ownership();
+    return raw_ptr;
+}
+
+template <typename T, typename D = std::default_delete<T>>
+std::unique_ptr<T, D> as_unique_ptr(smart_holder &hld) {
+    static const char *context = "as_unique_ptr";
+    hld.ensure_compatible_rtti_uqp_del<T, D>(context);
+    hld.ensure_use_count_1(context);
+    T *raw_ptr = hld.as_raw_ptr_unowned<T>();
+    hld.release_ownership();
+    // KNOWN DEFECT (see PR #4850): Does not copy the deleter.
+    return std::unique_ptr<T, D>(raw_ptr);
+}
+
+} // namespace smart_holder_poc
+} // namespace memory
+} // namespace pybindit