Skip to content

Commit 86d825f

Browse files
wjakobwjakob
committed
Redesigned virtual call mechanism and user-facing syntax (breaking change!)
Sergey Lyskov pointed out that the trampoline mechanism used to override virtual methods from within Python caused unnecessary overheads when instantiating the original (i.e. non-extended) class. This commit removes this inefficiency, but some syntax changes were needed to achieve this. Projects using this features will need to make a few changes: In particular, the example below shows the old syntax to instantiate a class with a trampoline: class_<TrampolineClass>("MyClass") .alias<MyClass>() .... This is what should be used now: class_<MyClass, std::unique_ptr<MyClass, TrampolineClass>("MyClass") .... Importantly, the trampoline class is now specified as the *third* argument to the class_ template, and the alias<..>() call is gone. The second argument with the unique pointer is simply the default holder type used by pybind11.
1 parent 60abf29 commit 86d825f

8 files changed

Lines changed: 113 additions & 33 deletions

File tree

‎docs/advanced.rst‎

Lines changed: 12 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -283,9 +283,8 @@ The binding code also needs a few minor adaptations (highlighted):
283283
PYBIND11_PLUGIN(example) {
284284
py::module m("example", "pybind11 example plugin");
285285
286-
py::class_<PyAnimal> animal(m, "Animal");
286+
py::class_<Animal, std::unique_ptr<Animal>, PyAnimal /* <--- trampoline*/> animal(m, "Animal");
287287
animal
288-
.alias<Animal>()
289288
.def(py::init<>())
290289
.def("go", &Animal::go);
291290
@@ -297,10 +296,10 @@ The binding code also needs a few minor adaptations (highlighted):
297296
return m.ptr();
298297
}
299298
300-
Importantly, the trampoline helper class is used as the template argument to
301-
:class:`class_`, and a call to :func:`class_::alias` informs the binding
302-
generator that this is merely an alias for the underlying type ``Animal``.
303-
Following this, we are able to define a constructor as usual.
299+
Importantly, pybind11 is made aware of the trampoline trampoline helper class
300+
by specifying it as the *third* template argument to :class:`class_`. The
301+
second argument with the unique pointer is simply the default holder type used
302+
by pybind11. Following this, we are able to define a constructor as usual.
304303

305304
The Python session below shows how to override ``Animal::go`` and invoke it via
306305
a virtual method call.
@@ -321,12 +320,12 @@ a virtual method call.
321320
322321
.. warning::
323322

324-
Both :func:`PYBIND11_OVERLOAD` and :func:`PYBIND11_OVERLOAD_PURE` are
325-
macros, which means that they can get confused by commas in a template
326-
argument such as ``PYBIND11_OVERLOAD(MyReturnValue<T1, T2>, myFunc)``. In
327-
this case, the preprocessor assumes that the comma indicates the beginnning
328-
of the next parameter. Use a ``typedef`` to bind the template to another
329-
name and use it in the macro to avoid this problem.
323+
The :func:`PYBIND11_OVERLOAD_*` calls are all just macros, which means that
324+
they can get confused by commas in a template argument such as
325+
``PYBIND11_OVERLOAD(MyReturnValue<T1, T2>, myFunc)``. In this case, the
326+
preprocessor assumes that the comma indicates the beginnning of the next
327+
parameter. Use a ``typedef`` to bind the template to another name and use
328+
it in the macro to avoid this problem.
330329

331330
.. seealso::
332331

@@ -369,9 +368,8 @@ be realized as follows (important changes highlighted):
369368
PYBIND11_PLUGIN(example) {
370369
py::module m("example", "pybind11 example plugin");
371370
372-
py::class_<PyAnimal> animal(m, "Animal");
371+
py::class_<Animal, std::unique_ptr<Animal>, PyAnimal> animal(m, "Animal");
373372
animal
374-
.alias<Animal>()
375373
.def(py::init<>())
376374
.def("go", &Animal::go);
377375

‎docs/changelog.rst‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,11 @@ Changelog
55

66
1.8 (Not yet released)
77
----------------------
8+
* Redesigned virtual call mechanism and user-facing syntax (breaking change!)
89
* Prevent implicit conversion of floating point values to integral types in
910
function arguments
1011
* Transparent conversion of sparse and dense Eigen data types
12+
* ``std::vector<>`` type bindings analogous to Boost.Python's ``indexing_suite``
1113
* Fixed incorrect default return value policy for functions returning a shared
1214
pointer
1315
* Don't allow casting a ``None`` value into a C++ lvalue reference
@@ -16,10 +18,19 @@ Changelog
1618
* Extended ``str`` type to also work with ``bytes`` instances
1719
* Added ``[[noreturn]]`` attribute to ``pybind11_fail()`` to quench some
1820
compiler warnings
21+
* List function arguments in exception text when the dispatch code cannot find
22+
a matching overload
1923
* Various minor ``iterator`` and ``make_iterator()`` improvements
24+
* Transparently support ``__bool__`` on Python 2.x and Python 3.x
25+
* Fixed issue with destructor of unpickled object not being called
2026
* Minor CMake build system improvements on Windows
2127
* Many ``mkdoc.py`` improvements (enumerations, template arguments, ``DOC()``
2228
macro accepts more arguments)
29+
* New ``pybind11::args`` and ``pybind11::kwargs`` types to create functions which
30+
take an arbitrary number of arguments and keyword arguments
31+
* New syntax to call a Python function from C++ using ``*args`` and ``*kwargs``
32+
* Added an ``ExtraFlags`` template argument to the NumPy ``array_t<>`` wrapper. This
33+
can be used to disable an enforced cast that may lose precision
2334
* Documentation improvements (pickling support, ``keep_alive``)
2435

2536
1.7 (April 30, 2016)

‎example/example12.cpp‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -82,15 +82,11 @@ void runExample12Virtual(Example12 *ex) {
8282
}
8383

8484
void init_ex12(py::module &m) {
85-
/* Important: use the wrapper type as a template
86-
argument to class_<>, but use the original name
87-
to denote the type */
88-
py::class_<PyExample12>(m, "Example12")
89-
/* Declare that 'PyExample12' is really an alias for the original type 'Example12' */
90-
.alias<Example12>()
85+
/* Important: indicate the trampoline class PyExample12 using the third
86+
argument to py::class_. The second argument with the unique pointer
87+
is simply the default holder type used by pybind11. */
88+
py::class_<Example12, std::unique_ptr<Example12>, PyExample12>(m, "Example12")
9189
.def(py::init<int>())
92-
/* Copy constructor (not needed in this case, but should generally be declared in this way) */
93-
.def(py::init<const PyExample12 &>())
9490
/* Reference original class in function definitions */
9591
.def("run", &Example12::run)
9692
.def("run_bool", &Example12::run_bool)

‎example/issues.cpp‎

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,7 @@ void init_issues(py::module &m) {
4242
}
4343
};
4444

45-
py::class_<DispatchIssue> base(m2, "DispatchIssue");
46-
base.alias<Base>()
45+
py::class_<Base, std::unique_ptr<Base>, DispatchIssue>(m2, "DispatchIssue")
4746
.def(py::init<>())
4847
.def("dispatch", &Base::dispatch);
4948

@@ -108,4 +107,28 @@ void init_issues(py::module &m) {
108107
// (no id): don't cast doubles to ints
109108
m2.def("expect_float", [](float f) { return f; });
110109
m2.def("expect_int", [](int i) { return i; });
110+
111+
// (no id): don't invoke Python dispatch code when instantiating C++
112+
// classes that were not extended on the Python side
113+
struct A {
114+
virtual ~A() {}
115+
virtual void f() { std::cout << "A.f()" << std::endl; }
116+
};
117+
118+
struct PyA : A {
119+
PyA() { std::cout << "PyA.PyA()" << std::endl; }
120+
121+
void f() override {
122+
std::cout << "PyA.f()" << std::endl;
123+
PYBIND11_OVERLOAD(void, A, f);
124+
}
125+
};
126+
127+
auto call_f = [](A *a) { a->f(); };
128+
129+
pybind11::class_<A, std::unique_ptr<A>, PyA>(m2, "A")
130+
.def(py::init<>())
131+
.def("f", &A::f);
132+
133+
m2.def("call_f", call_f);
111134
}

‎example/issues.py‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
from example.issues import iterator_passthrough
1010
from example.issues import ElementList, ElementA, print_element
1111
from example.issues import expect_float, expect_int
12+
from example.issues import A, call_f
1213
import gc
1314

1415
print_cchar("const char *")
@@ -55,3 +56,19 @@ def dispatch(self):
5556
print("Failed as expected: " + str(e))
5657

5758
print(expect_float(12))
59+
60+
class B(A):
61+
def __init__(self):
62+
super(B, self).__init__()
63+
64+
def f(self):
65+
print("In python f()")
66+
67+
print("C++ version")
68+
a = A()
69+
call_f(a)
70+
71+
print("Python version")
72+
b = B()
73+
call_f(b)
74+

‎example/issues.ref‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,9 @@ Failed as expected: Incompatible function arguments. The following argument type
1212
1. (int) -> int
1313
Invoked with: 5.2
1414
12.0
15+
C++ version
16+
A.f()
17+
Python version
18+
PyA.PyA()
19+
PyA.f()
20+
In python f()

‎include/pybind11/attr.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ enum op_type : int;
6565
struct undefined_t;
6666
template <op_id id, op_type ot, typename L = undefined_t, typename R = undefined_t> struct op_;
6767
template <typename... Args> struct init;
68+
template <typename... Args> struct init_alias;
6869
inline void keep_alive_impl(int Nurse, int Patient, handle args, handle ret);
6970

7071
/// Internal data structure which holds metadata about a keyword argument

‎include/pybind11/pybind11.h‎

Lines changed: 37 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -503,7 +503,7 @@ class module : public object {
503503
NAMESPACE_BEGIN(detail)
504504
/// Generic support for creating new Python heap types
505505
class generic_type : public object {
506-
template <typename type, typename holder_type> friend class class_;
506+
template <typename type, typename holder_type, typename type_alias> friend class class_;
507507
public:
508508
PYBIND11_OBJECT_DEFAULT(generic_type, object, PyType_Check)
509509
protected:
@@ -721,7 +721,7 @@ class generic_type : public object {
721721
};
722722
NAMESPACE_END(detail)
723723

724-
template <typename type, typename holder_type = std::unique_ptr<type>>
724+
template <typename type, typename holder_type = std::unique_ptr<type>, typename type_alias = type>
725725
class class_ : public detail::generic_type {
726726
public:
727727
typedef detail::instance<type, holder_type> instance_type;
@@ -743,6 +743,11 @@ class class_ : public detail::generic_type {
743743
detail::process_attributes<Extra...>::init(extra..., &record);
744744

745745
detail::generic_type::initialize(&record);
746+
747+
if (!std::is_same<type, type_alias>::value) {
748+
auto &instances = pybind11::detail::get_internals().registered_types_cpp;
749+
instances[std::type_index(typeid(type_alias))] = instances[std::type_index(typeid(type))];
750+
}
746751
}
747752

748753
template <typename Func, typename... Extra>
@@ -780,6 +785,12 @@ class class_ : public detail::generic_type {
780785
return *this;
781786
}
782787

788+
template <typename... Args, typename... Extra>
789+
class_ &def(const detail::init_alias<Args...> &init, const Extra&... extra) {
790+
init.template execute<type>(*this, extra...);
791+
return *this;
792+
}
793+
783794
template <typename Func> class_& def_buffer(Func &&func) {
784795
struct capture { Func func; };
785796
capture *ptr = new capture { std::forward<Func>(func) };
@@ -856,11 +867,6 @@ class class_ : public detail::generic_type {
856867
return *this;
857868
}
858869

859-
template <typename target> class_ alias() {
860-
auto &instances = pybind11::detail::get_internals().registered_types_cpp;
861-
instances[std::type_index(typeid(target))] = instances[std::type_index(typeid(type))];
862-
return *this;
863-
}
864870
private:
865871
/// Initialize holder object, variant 1: object derives from enable_shared_from_this
866872
template <typename T>
@@ -959,9 +965,31 @@ template <typename Type> class enum_ : public class_<Type> {
959965

960966
NAMESPACE_BEGIN(detail)
961967
template <typename... Args> struct init {
962-
template <typename Base, typename Holder, typename... Extra> void execute(pybind11::class_<Base, Holder> &class_, const Extra&... extra) const {
968+
template <typename Base, typename Holder, typename Alias, typename... Extra,
969+
typename std::enable_if<std::is_same<Base, Alias>::value, int>::type = 0>
970+
void execute(pybind11::class_<Base, Holder, Alias> &class_, const Extra&... extra) const {
963971
/// Function which calls a specific C++ in-place constructor
964-
class_.def("__init__", [](Base *instance, Args... args) { new (instance) Base(args...); }, extra...);
972+
class_.def("__init__", [](Base *self, Args... args) { new (self) Base(args...); }, extra...);
973+
}
974+
975+
template <typename Base, typename Holder, typename Alias, typename... Extra,
976+
typename std::enable_if<!std::is_same<Base, Alias>::value &&
977+
std::is_constructible<Base, Args...>::value, int>::type = 0>
978+
void execute(pybind11::class_<Base, Holder, Alias> &class_, const Extra&... extra) const {
979+
handle cl_type = class_;
980+
class_.def("__init__", [cl_type](handle self, Args... args) {
981+
if (self.get_type() == cl_type)
982+
new (self.cast<Base *>()) Base(args...);
983+
else
984+
new (self.cast<Alias *>()) Alias(args...);
985+
}, extra...);
986+
}
987+
988+
template <typename Base, typename Holder, typename Alias, typename... Extra,
989+
typename std::enable_if<!std::is_same<Base, Alias>::value &&
990+
!std::is_constructible<Base, Args...>::value, int>::type = 0>
991+
void execute(pybind11::class_<Base, Holder, Alias> &class_, const Extra&... extra) const {
992+
class_.def("__init__", [](Alias *self, Args... args) { new (self) Alias(args...); }, extra...);
965993
}
966994
};
967995

0 commit comments

Comments
 (0)