pybind/pybind11
1
0
Fork 0
mirror of https://github.com/pybind/pybind11.git synced 2026-04-20 06:49:25 +00:00
Code Issues Packages Projects Releases Wiki Activity

Option for arg/return type hints and correct typing for std::filesystem::path (#5450)

Browse Source 
* Added arg/return type handling.

* Added support for nested arg/return type in py::typing::List

* Added support for arg/return type in stl/filesystem

* Added tests for arg/return type in stl/filesystem and py::typing::List

* Added arg/return name to more py::typing classes

* Added arg/return type to Callable[...]

* Added tests for typing container classes (also nested)

* Changed typing classes to avoid using C++14 auto return type deduction.

* Fixed clang-tidy errors.

* Changed Enable to SFINAE

* Added test for Tuple[T, ...]

* Added RealNumber with custom caster for testing typing classes.

* Added tests for Set, Iterable, Iterator, Union, and Optional

* Added tests for Callable

* Fixed Callable with ellipsis test

* Changed TypeGuard/TypeIs to use return type (being the narrower type) + Tests

* Added test for use of fallback type name with stl vector

* Updated documentation.

* Fixed unnecessary constructor call in test.

* Fixed reference counting in example type caster.

* Fixed clang-tidy issues.

* Fix for clang-tidy

* Updated cast method to use pybind11 API rather than Python C API in custom caster example

* Updated load to use pybind11 API rather than Python C API in custom caster example

* Changed test of arg/return name to use pybind11 API instead of Python C API

* Updated code in adcanced/cast example and improved documentation text

* Fixed references in custom type caster docs

* Fixed wrong logical and operator in test

* Fixed wrong logical operator in doc example

* Added comment to test about `float` vs `float | int`

* Updated std::filesystem::path docs in cast/overview section

* Remove one stray dot.

---------

Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
This commit is contained in:
Tim Ohliger
2024-12-08 20:30:49 +01:00
committed by GitHub
parent a6d1ff2460
commit 1d09fc8300
13 changed files with 619 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

158
docs/advanced/cast/custom.rst
View File

@@ -1,35 +1,53 @@
Custom type casters
===================
In very rare cases, applications may require custom type casters that cannot be
expressed using the abstractions provided by pybind11, thus requiring raw
Python C API calls. This is fairly advanced usage and should only be pursued by
experts who are familiar with the intricacies of Python reference counting.
Some applications may prefer custom type casters that convert between existing
Python types and C++ types, similar to the ``list`` ↔ ``std::vector``
and ``dict`` ↔ ``std::map`` conversions which are built into pybind11.
Implementing custom type casters is fairly advanced usage.
While it is recommended to use the pybind11 API as much as possible, more complex examples may
require familiarity with the intricacies of the Python C API.
You can refer to the `Python/C API Reference Manual <https://docs.python.org/3/c-api/index.html>`_
for more information.
The following snippets demonstrate how this works for a very simple ``inty``
type that that should be convertible from Python types that provide a
``__int__(self)`` method.
The following snippets demonstrate how this works for a very simple ``Point2D`` type.
We want this type to be convertible to C++ from Python types implementing the
``Sequence`` protocol and having two elements of type ``float``.
When returned from C++ to Python, it should be converted to a Python ``tuple[float, float]``.
For this type we could provide Python bindings for different arithmetic functions implemented
in C++ (here demonstrated by a simple ``negate`` function).
..
PLEASE KEEP THE CODE BLOCKS IN SYNC WITH
tests/test_docs_advanced_cast_custom.cpp
tests/test_docs_advanced_cast_custom.py
Ideally, change the test, run pre-commit (incl. clang-format),
then copy the changed code back here.
Also use TEST_SUBMODULE in tests, but PYBIND11_MODULE in docs.
.. code-block:: cpp
struct inty { long long_value; };
namespace user_space {
void print(inty s) {
std::cout << s.long_value << std::endl;
}
struct Point2D {
double x;
double y;
};
The following Python snippet demonstrates the intended usage from the Python side:
Point2D negate(const Point2D &point) { return Point2D{-point.x, -point.y}; }
} // namespace user_space
The following Python snippet demonstrates the intended usage of ``negate`` from the Python side:
.. code-block:: python
class A:
def __int__(self):
return 123
from my_math_module import docs_advanced_cast_custom as m
from example import print
print(A())
point1 = [1.0, -1.0]
point2 = m.negate(point1)
assert point2 == (-1.0, 1.0)
To register the necessary conversion routines, it is necessary to add an
instantiation of the ``pybind11::detail::type_caster<T>`` template.
@@ -38,47 +56,59 @@ type is explicitly allowed.
.. code-block:: cpp
namespace PYBIND11_NAMESPACE { namespace detail {
template <> struct type_caster<inty> {
public:
/**
* This macro establishes the name 'inty' in
* function signatures and declares a local variable
* 'value' of type inty
*/
PYBIND11_TYPE_CASTER(inty, const_name("inty"));
namespace pybind11 {
namespace detail {
/**
* Conversion part 1 (Python->C++): convert a PyObject into a inty
* instance or return false upon failure. The second argument
* indicates whether implicit conversions should be applied.
*/
bool load(handle src, bool) {
/* Extract PyObject from handle */
PyObject *source = src.ptr();
/* Try converting into a Python integer value */
PyObject *tmp = PyNumber_Long(source);
if (!tmp)
template <>
struct type_caster<user_space::Point2D> {
// This macro inserts a lot of boilerplate code and sets the default type hint to `tuple`
PYBIND11_TYPE_CASTER(user_space::Point2D, const_name("tuple"));
// `arg_name` and `return_name` may optionally be used to specify type hints separately for
// arguments and return values.
// The signature of our negate function would then look like:
// `negate(Sequence[float]) -> tuple[float, float]`
static constexpr auto arg_name = const_name("Sequence[float]");
static constexpr auto return_name = const_name("tuple[float, float]");
// C++ -> Python: convert `Point2D` to `tuple[float, float]`. The second and third arguments
// are used to indicate the return value policy and parent object (for
// return_value_policy::reference_internal) and are often ignored by custom casters.
// The return value should reflect the type hint specified by `return_name`.
static handle
cast(const user_space::Point2D &number, return_value_policy /*policy*/, handle /*parent*/) {
return py::make_tuple(number.x, number.y).release();
}
// Python -> C++: convert a `PyObject` into a `Point2D` and return false upon failure. The
// second argument indicates whether implicit conversions should be allowed.
// The accepted types should reflect the type hint specified by `arg_name`.
bool load(handle src, bool /*convert*/) {
// Check if handle is a Sequence
if (!py::isinstance<py::sequence>(src)) {
return false;
}
auto seq = py::reinterpret_borrow<py::sequence>(src);
// Check if exactly two values are in the Sequence
if (seq.size() != 2) {
return false;
}
// Check if each element is either a float or an int
for (auto item : seq) {
if (!py::isinstance<py::float_>(item) && !py::isinstance<py::int_>(item)) {
return false;
/* Now try to convert into a C++ int */
value.long_value = PyLong_AsLong(tmp);
Py_DECREF(tmp);
/* Ensure return code was OK (to avoid out-of-range errors etc) */
return !(value.long_value == -1 && !PyErr_Occurred());
}
}
value.x = seq[0].cast<double>();
value.y = seq[1].cast<double>();
return true;
}
};
/**
* Conversion part 2 (C++ -> Python): convert an inty instance into
* a Python object. The second and third arguments are used to
* indicate the return value policy and parent object (for
* ``return_value_policy::reference_internal``) and are generally
* ignored by implicit casters.
*/
static handle cast(inty src, return_value_policy /* policy */, handle /* parent */) {
return PyLong_FromLong(src.long_value);
}
};
}} // namespace PYBIND11_NAMESPACE::detail
} // namespace detail
} // namespace pybind11
// Bind the negate function
PYBIND11_MODULE(docs_advanced_cast_custom, m) { m.def("negate", user_space::negate); }
.. note::
@@ -86,8 +116,22 @@ type is explicitly allowed.
that ``T`` is default-constructible (``value`` is first default constructed
and then ``load()`` assigns to it).
.. note::
For further information on the ``return_value_policy`` argument of ``cast`` refer to :ref:`return_value_policies`.
To learn about the ``convert`` argument of ``load`` see :ref:`nonconverting_arguments`.
.. warning::
When using custom type casters, it's important to declare them consistently
in every compilation unit of the Python extension module. Otherwise,
in every compilation unit of the Python extension module to satisfy the C++ One Definition Rule
(`ODR <https://en.cppreference.com/w/cpp/language/definition>`_). Otherwise,
undefined behavior can ensue.
.. note::
Using the type hint ``Sequence[float]`` signals to static type checkers, that not only tuples may be
passed, but any type implementing the Sequence protocol, e.g., ``list[float]``.
Unfortunately, that loses the length information ``tuple[float, float]`` provides.
One way of still providing some length information in type hints is using ``typing.Annotated``, e.g.,
``Annotated[Sequence[float], 2]``, or further add libraries like
`annotated-types <https://github.com/annotated-types/annotated-types>`_.

4
docs/advanced/cast/overview.rst
View File

@@ -151,7 +151,7 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
+------------------------------------+---------------------------+-----------------------------------+
| ``std::variant<...>`` | Type-safe union (C++17) | :file:`pybind11/stl.h` |
+------------------------------------+---------------------------+-----------------------------------+
| ``std::filesystem::path<T>`` | STL path (C++17) [#]_ | :file:`pybind11/stl/filesystem.h` |
| ``std::filesystem::path`` | STL path (C++17) [#]_ | :file:`pybind11/stl/filesystem.h` |
+------------------------------------+---------------------------+-----------------------------------+
| ``std::function<...>`` | STL polymorphic function | :file:`pybind11/functional.h` |
+------------------------------------+---------------------------+-----------------------------------+
@@ -167,4 +167,4 @@ as arguments and return values, refer to the section on binding :ref:`classes`.
+------------------------------------+---------------------------+-----------------------------------+
.. [#] ``std::filesystem::path`` is converted to ``pathlib.Path`` and
``os.PathLike`` is converted to ``std::filesystem::path``.
can be loaded from ``os.PathLike``, ``str``, and ``bytes``.