Fix potential crash when calling an overloaded function (#1327)

* Fix potential crash when calling an overloaded function

The crash would occur if:
- dispatcher() uses two-pass logic (because the target is overloaded and some arguments support conversions)
- the first pass (with conversions disabled) doesn't find any matching overload
- the second pass does find a matching overload, but its return value can't be converted to Python

The code for formatting the error message assumed `it` still pointed to the selected overload,
but during the second-pass loop `it` was nullptr. Fix by setting `it` correctly if a second-pass
call returns a nullptr `handle`. Add a new test that segfaults without this fix.

* Make overload iteration const-correct so we don't have to iterate again on second-pass error

* Change test_error_after_conversions dependencies to local classes/variables
This commit is contained in:
oremanj
2018-09-25 14:55:18 -07:00
committed by Wenzel Jakob
parent 9343e68b46
commit e7761e3383
5 changed files with 33 additions and 8 deletions

View File

@@ -341,6 +341,19 @@ TEST_SUBMODULE(class_, m) {
"a"_a, "b"_a, "c"_a);
base.def("g", [](NestBase &, Nested &) {});
base.def("h", []() { return NestBase(); });
// test_error_after_conversion
// The second-pass path through dispatcher() previously didn't
// remember which overload was used, and would crash trying to
// generate a useful error message
struct NotRegistered {};
struct StringWrapper { std::string str; };
m.def("test_error_after_conversions", [](int) {});
m.def("test_error_after_conversions",
[](StringWrapper) -> NotRegistered { return {}; });
py::class_<StringWrapper>(m, "StringWrapper").def(py::init<std::string>());
py::implicitly_convertible<std::string, StringWrapper>();
}
template <int N> class BreaksBase { public: virtual ~BreaksBase() = default; };