pybind/pybind11
1
0
Fork 0
mirror of https://github.com/pybind/pybind11.git synced 2026-03-14 20:27:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: reapply fixed version of #3271 (#3293)

Browse Source 
* Add make_value_iterator (#3271)

* Add make_value_iterator

This is the counterpart to make_key_iterator, and will allow
implementing a `value` method in `bind_map` (although doing so is left
for a subsequent PR).

I made a few design changes to reduce copy-and-paste boilerplate.
Previously detail::iterator_state had a boolean template parameter to
indicate whether it was being used for make_iterator or
make_key_iterator. I replaced the boolean with a class that determines
how to dereference the iterator. This allows for a generic
implementation of `__next__`.

I also added the ValueType and Extra... parameters to the iterator_state
template args, because I think it was a bug that they were missing: if
make_iterator is called twice with different values of these, only the
first set has effect (because the state class is only registered once).
There is still a potential issue in that the *values* of the extra
arguments are latched on the first call, but since most policies are
empty classes this should be even less common.

* Add some remove_cv_t to appease clang-tidy

* Make iterator_access and friends take reference

For some reason I'd accidentally made it take a const value, which
caused some issues with third-party packages.

* Another attempt to remove remove_cv_t from iterators

Some of the return types were const (non-reference) types because of the
pecularities of decltype: `decltype((*it).first)` is the *declared* type
of the member of the pair, rather than the type of the expression. So if
the reference type of the iterator is `pair<const int, int> &`, then the
decltype is `const int`. Wrapping an extra set of parentheses to form
`decltype(((*it).first))` would instead give `const int &`.

This means that the existing make_key_iterator actually returns by value
from `__next__`, rather than by reference. Since for mapping types, keys
are always const, this probably hasn't been noticed, but it will affect
make_value_iterator if the Python code tries to mutate the returned
objects. I've changed things to use double parentheses so that
make_iterator, make_key_iterator and make_value_iterator should now all
return the reference type of the iterator. I'll still need to add a test
for that; for now I'm just checking whether I can keep Clang-Tidy happy.

* Add back some NOLINTNEXTLINE to appease Clang-Tidy

This is favoured over using remove_cv_t because in some cases a const
value return type is deliberate (particularly for Eigen).

* Add a unit test for iterator referencing

Ensure that make_iterator, make_key_iterator and make_value_iterator
return references to the container elements, rather than copies. The
test for make_key_iterator fails to compile on master, which gives me
confidence that this branch has fixed it.

* Make the iterator_access etc operator() const

I'm actually a little surprised it compiled at all given that the
operator() is called on a temporary, but I don't claim to fully
understand all the different value types in C++11.

* Attempt to work around compiler bugs

https://godbolt.org/ shows an example where ICC gets the wrong result
for a decltype used as the default for a template argument, and CI also
showed problems with PGI. This is a shot in the dark to see if it fixes
things.

* Make a test constructor explicit (Clang-Tidy)

* Fix unit test on GCC 4.8.5

It seems to require the arguments to the std::pair constructor to be
implicitly convertible to the types in the pair, rather than just
requiring is_constructible.

* Remove DOXYGEN_SHOULD_SKIP_THIS guards

Now that a complex decltype expression has been replaced by a simpler
nested type, I'm hoping Doxygen will be able to build it without issues.

* Add comment to explain iterator_state template params

* fix: regression in #3271

Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com>
This commit is contained in:
Henry Schreiner
2021-09-23 15:06:07 -04:00
committed by GitHub
parent 2a78abffd8
commit 21282e645a
4 changed files with 195 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
tests/test_sequences_and_iterators.cpp
View File

@@ -15,6 +15,7 @@
#include <algorithm>
#include <utility>
#include <vector>
#ifdef PYBIND11_HAS_OPTIONAL
#include <optional>
@@ -37,6 +38,29 @@ bool operator==(const NonZeroIterator<std::pair<A, B>>& it, const NonZeroSentine
return !(*it).first || !(*it).second;
}
class NonCopyableInt {
public:
explicit NonCopyableInt(int value) : value_(value) {}
NonCopyableInt(const NonCopyableInt &) = delete;
NonCopyableInt(NonCopyableInt &&other) noexcept : value_(other.value_) {
other.value_ = -1; // detect when an unwanted move occurs
}
NonCopyableInt &operator=(const NonCopyableInt &) = delete;
NonCopyableInt &operator=(NonCopyableInt &&other) noexcept {
value_ = other.value_;
other.value_ = -1; // detect when an unwanted move occurs
return *this;
}
int get() const { return value_; }
void set(int value) { value_ = value; }
~NonCopyableInt() = default;
private:
int value_;
};
using NonCopyableIntPair = std::pair<NonCopyableInt, NonCopyableInt>;
PYBIND11_MAKE_OPAQUE(std::vector<NonCopyableInt>);
PYBIND11_MAKE_OPAQUE(std::vector<NonCopyableIntPair>);
template <typename PythonType>
py::list test_random_access_iterator(PythonType x) {
if (x.size() < 5)
@@ -288,6 +312,10 @@ TEST_SUBMODULE(sequences_and_iterators, m) {
.def(
"items",
[](const StringMap &map) { return py::make_iterator(map.begin(), map.end()); },
py::keep_alive<0, 1>())
.def(
"values",
[](const StringMap &map) { return py::make_value_iterator(map.begin(), map.end()); },
py::keep_alive<0, 1>());
// test_generalized_iterators
@@ -308,19 +336,62 @@ TEST_SUBMODULE(sequences_and_iterators, m) {
.def("nonzero_keys", [](const IntPairs& s) {
return py::make_key_iterator(NonZeroIterator<std::pair<int, int>>(s.begin()), NonZeroSentinel());
}, py::keep_alive<0, 1>())
.def("nonzero_values", [](const IntPairs& s) {
return py::make_value_iterator(NonZeroIterator<std::pair<int, int>>(s.begin()), NonZeroSentinel());
}, py::keep_alive<0, 1>())
// test single-argument make_iterator
.def("simple_iterator", [](IntPairs& self) {
return py::make_iterator(self);
}, py::keep_alive<0, 1>())
.def("simple_keys", [](IntPairs& self) {
return py::make_key_iterator(self);
}, py::keep_alive<0, 1>())
.def("simple_values", [](IntPairs& self) {
return py::make_value_iterator(self);
}, py::keep_alive<0, 1>())
// test iterator with keep_alive (doesn't work so not used at runtime, but tests compile)
.def("make_iterator_keep_alive", [](IntPairs& self) {
return py::make_iterator(self, py::keep_alive<0, 1>());
// Test iterator with an Extra (doesn't do anything useful, so not used
// at runtime, but tests need to be able to compile with the correct
// overload. See PR #3293.
.def("_make_iterator_extras", [](IntPairs& self) {
return py::make_iterator(self, py::call_guard<int>());
}, py::keep_alive<0, 1>())
.def("_make_key_extras", [](IntPairs& self) {
return py::make_key_iterator(self, py::call_guard<int>());
}, py::keep_alive<0, 1>())
.def("_make_value_extras", [](IntPairs& self) {
return py::make_value_iterator(self, py::call_guard<int>());
}, py::keep_alive<0, 1>())
;
// test_iterater_referencing
py::class_<NonCopyableInt>(m, "NonCopyableInt")
.def(py::init<int>())
.def("set", &NonCopyableInt::set)
.def("__int__", &NonCopyableInt::get)
;
py::class_<std::vector<NonCopyableInt>>(m, "VectorNonCopyableInt")
.def(py::init<>())
.def("append", [](std::vector<NonCopyableInt> &vec, int value) {
vec.emplace_back(value);
})
.def("__iter__", [](std::vector<NonCopyableInt> &vec) {
return py::make_iterator(vec.begin(), vec.end());
})
;
py::class_<std::vector<NonCopyableIntPair>>(m, "VectorNonCopyableIntPair")
.def(py::init<>())
.def("append", [](std::vector<NonCopyableIntPair> &vec, const std::pair<int, int> &value) {
vec.emplace_back(NonCopyableInt(value.first), NonCopyableInt(value.second));
})
.def("keys", [](std::vector<NonCopyableIntPair> &vec) {
return py::make_key_iterator(vec.begin(), vec.end());
})
.def("values", [](std::vector<NonCopyableIntPair> &vec) {
return py::make_value_iterator(vec.begin(), vec.end());
})
;
#if 0
// Obsolete: special data structure for exposing custom iterator types to python