Add format_descriptor<> & npy_format_descriptor<> PyObject * specializations. (#4674)

* Add `npy_format_descriptor<PyObject *>` to enable `py::array_t<PyObject *>` to/from-python conversions.

* resolve clang-tidy warning

* Use existing constructor instead of adding a static method. Thanks @Skylion007 for pointing out.

* Add `format_descriptor<PyObject *>`

Trivial addition, but still in search for a meaningful test.

* Add test_format_descriptor_format

* Ensure the Eigen `type_caster`s do not segfault when loading arrays with dtype=object

* Use `static_assert()` `!std::is_pointer<>` to replace runtime guards.

* Add comments to explain how to check for ref-count bugs. (NO code changes.)

* Make the "Pointer types ... are not supported" message Eigen-specific, as suggested by @Lalaland. Move to new pybind11/eigen/common.h header.

* Change "format_descriptor_format" implementation as suggested by @Lalaland. Additional tests meant to ensure consistency between py::format_descriptor<>, np.array, np.format_parser turn out to be useful only to highlight long-standing inconsistencies.

* resolve clang-tidy warning

* Account for np.float128, np.complex256 not being available on Windows, in a future-proof way.

* Fully address i|q|l ambiguity (hopefully).

* Remove the new `np.format_parser()`-based test, it's much more distracting than useful.

* Use bi.itemsize to disambiguate "l" or "L"

* Use `py::detail::compare_buffer_info<T>::compare()` to validate the `format_descriptor<T>::format()` strings.

* Add `buffer_info::compare<T>` to make `detail::compare_buffer_info<T>::compare` more visible & accessible.

* silence clang-tidy warning

* pytest-compatible access to np.float128, np.complex256

* Revert "pytest-compatible access to np.float128, np.complex256"

This reverts commit e9a289c50f.

* Use `sizeof(long double) == sizeof(double)` instead of `std::is_same<>`

* Report skipped `long double` tests.

* Change the name of the new `buffer_info` member function to `item_type_is_equivalent_to`. Add comment defining "equivalent" by example.

* Change `item_type_is_equivalent_to<>()` from `static` function to member function, as suggested by @Lalaland
This commit is contained in:
Ralf W. Grosse-Kunstleve
2023-05-23 10:49:32 -07:00
committed by GitHub
parent 6e6bcca5b2
commit 8e1f9d5c40
12 changed files with 255 additions and 7 deletions

View File

@@ -595,3 +595,74 @@ def test_round_trip_float():
arr = np.zeros((), np.float64)
arr[()] = 37.2
assert m.round_trip_float(arr) == 37.2
# HINT: An easy and robust way (although only manual unfortunately) to check for
# ref-count leaks in the test_.*pyobject_ptr.* functions below is to
# * temporarily insert `while True:` (one-by-one),
# * run this test, and
# * run the Linux `top` command in another shell to visually monitor
# `RES` for a minute or two.
# If there is a leak, it is usually evident in seconds because the `RES`
# value increases without bounds. (Don't forget to Ctrl-C the test!)
# For use as a temporary user-defined object, to maximize sensitivity of the tests below:
# * Ref-count leaks will be immediately evident.
# * Sanitizers are much more likely to detect heap-use-after-free due to
# other ref-count bugs.
class PyValueHolder:
def __init__(self, value):
self.value = value
def WrapWithPyValueHolder(*values):
return [PyValueHolder(v) for v in values]
def UnwrapPyValueHolder(vhs):
return [vh.value for vh in vhs]
def test_pass_array_pyobject_ptr_return_sum_str_values_ndarray():
# Intentionally all temporaries, do not change.
assert (
m.pass_array_pyobject_ptr_return_sum_str_values(
np.array(WrapWithPyValueHolder(-3, "four", 5.0), dtype=object)
)
== "-3four5.0"
)
def test_pass_array_pyobject_ptr_return_sum_str_values_list():
# Intentionally all temporaries, do not change.
assert (
m.pass_array_pyobject_ptr_return_sum_str_values(
WrapWithPyValueHolder(2, "three", -4.0)
)
== "2three-4.0"
)
def test_pass_array_pyobject_ptr_return_as_list():
# Intentionally all temporaries, do not change.
assert UnwrapPyValueHolder(
m.pass_array_pyobject_ptr_return_as_list(
np.array(WrapWithPyValueHolder(-1, "two", 3.0), dtype=object)
)
) == [-1, "two", 3.0]
@pytest.mark.parametrize(
("return_array_pyobject_ptr", "unwrap"),
[
(m.return_array_pyobject_ptr_cpp_loop, list),
(m.return_array_pyobject_ptr_from_list, UnwrapPyValueHolder),
],
)
def test_return_array_pyobject_ptr_cpp_loop(return_array_pyobject_ptr, unwrap):
# Intentionally all temporaries, do not change.
arr_from_list = return_array_pyobject_ptr(WrapWithPyValueHolder(6, "seven", -8.0))
assert isinstance(arr_from_list, np.ndarray)
assert arr_from_list.dtype == np.dtype("O")
assert unwrap(arr_from_list) == [6, "seven", -8.0]