bpl_mods.txt

来自「Boost provides free peer-reviewed portab」· 文本 代码 · 共 913 行 · 第 1/3 页

    {
        // Store a pointer to the Python object
        BaseWrap(PyObject* self_) : self(self_) {}
        PyObject* self;

        // Default implementation, for when f is not overridden
        int f_default(std::string x) { return this->Base::f(x); }
        // Dispatch implementation
        int f(std::string x) { return call_method<int>(self, "f", x); }
    };

    ...
        def("calls_f", calls_f);
        class_<Base, BaseWrap>("Base")
            .def("f", &Base::f, &BaseWrap::f_default)
            ;

Now here's some Python code which demonstrates:

>>> class Derived(Base):
...     def f(self, s):
...          return len(s)
...
>>> calls_f(Base(), 'foo')
42
>>> calls_f(Derived(), 'forty-two')
9

Things to notice about the dispatcher class:

* The key element which allows overriding in Python is the
  ``call_method`` invocation, which uses the same global type
  conversion registry as the C++ function wrapping does to convert its
  arguments from C++ to Python and its return type from Python to C++.

* Any constructor signatures you wish to wrap must be replicated with
  an initial ``PyObject*`` argument

* The dispatcher must store this argument so that it can be used to
  invoke ``call_method``

* The ``f_default`` member function is needed when the function being
  exposed is not pure virtual; there's no other way ``Base::f`` can be
  called on an object of type ``BaseWrap``, since it overrides ``f``.

Admittedly, this formula is tedious to repeat, especially on a project
with many polymorphic classes; that it is necessary reflects
limitations in C++'s compile-time reflection capabilities.  Several
efforts are underway to write front-ends for Boost.Python which can
generate these dispatchers (and other wrapping code) automatically.
If these are successful it will mark a move away from wrapping
everything directly in pure C++ for many of our users.

---------------
 Serialization
---------------

*Serialization* is the process of converting objects in memory to a
form that can be stored on disk or sent over a network connection. The
serialized object (most often a plain string) can be retrieved and
converted back to the original object. A good serialization system will
automatically convert entire object hierarchies. Python's standard
``pickle`` module is such a system.  It leverages the language's strong
runtime introspection facilities for serializing practically arbitrary
user-defined objects. With a few simple and unintrusive provisions this
powerful machinery can be extended to also work for wrapped C++ objects.
Here is an example::

    #include <string>

    struct World
    {
        World(std::string a_msg) : msg(a_msg) {}
        std::string greet() const { return msg; }
        std::string msg;
    };

    #include <boost/python.hpp>
    using namespace boost::python;

    struct World_picklers : pickle_suite
    {
      static tuple
      getinitargs(World const& w) { return make_tuple(w.greet()); }
    };

    BOOST_PYTHON_MODULE(hello)
    {
        class_<World>("World", init<std::string>())
            .def("greet", &World::greet)
            .def_pickle(World_picklers())
        ;
    }

Now let's create a ``World`` object and put it to rest on disk::

    >>> import hello
    >>> import pickle
    >>> a_world = hello.World("howdy")
    >>> pickle.dump(a_world, open("my_world", "w"))

In a potentially *different script* on a potentially *different
computer* with a potentially *different operating system*::

    >>> import pickle
    >>> resurrected_world = pickle.load(open("my_world", "r"))
    >>> resurrected_world.greet()
    'howdy'

Of course the ``cPickle`` module can also be used for faster
processing.

Boost.Python's ``pickle_suite`` fully supports the ``pickle`` protocol
defined in the standard Python documentation. There is a one-to-one
correspondence between the standard pickling methods (``__getinitargs__``,
``__getstate__``, ``__setstate__``) and the functions defined by the
user in the class derived from ``pickle_suite`` (``getinitargs``,
``getstate``, ``setstate``). The ``class_::def_pickle()`` member function
is used to establish the Python bindings for all user-defined functions
simultaneously. Correct signatures for these functions are enforced at
compile time. Non-sensical combinations of the three pickle functions
are also rejected at compile time. These measures are designed to
help the user in avoiding obvious errors.

Enabling serialization of more complex C++ objects requires a little
more work than is shown in the example above. Fortunately the
``object`` interface (see next section) greatly helps in keeping the
code manageable.

------------------
 Object interface
------------------

Experienced extension module authors will be familiar with the 'C' view
of Python objects, the ubiquitous ``PyObject*``. Most if not all Python
'C' API functions involve ``PyObject*`` as arguments or return type.  A
major complication is the raw reference counting interface presented to
the 'C' programmer. E.g. some API functions return *new references* and
others return *borrowed references*. It is up to the extension module
writer to properly increment and decrement reference counts.  This
quickly becomes cumbersome and error prone, especially if there are
multiple execution paths.

Boost.Python provides a type ``object`` which is essentially a high
level wrapper around ``PyObject*``. ``object`` automates reference
counting as much as possible. It also provides the facilities for
converting arbitrary C++ types to Python objects and vice versa.
This significantly reduces the learning effort for prospective
extension module writers.

Creating an ``object`` from any other type is extremely simple::

    object o(3);

``object`` has templated interactions with all other types, with
automatic to-python conversions. It happens so naturally that it's
easily overlooked.

The ``extract<T>`` class template can be used to convert Python objects
to C++ types::

    double x = extract<double>(o);

All registered user-defined conversions are automatically accessible
through the ``object`` interface. With reference to the ``World`` class
defined in previous examples::

    object as_python_object(World("howdy"));
    World back_as_c_plus_plus_object = extract<World>(as_python_object);

If a C++ type cannot be converted to a Python object an appropriate
exception is thrown at runtime.  Similarly, an appropriate exception is
thrown if a C++ type cannot be extracted from a Python object.
``extract<T>`` provides facilities for avoiding exceptions if this is
desired.

The ``object::attr()`` member function is available for accessing
and manipulating attributes of Python objects. For example::

    object planet(World());
    planet.attr("set")("howdy");

``planet.attr("set")`` returns a callable ``object``.  ``"howdy"`` is
converted to a Python string object which is then passed as an argument
to the ``set`` method.

The ``object`` type is accompanied by a set of derived types
that mirror the Python built-in types such as ``list``, ``dict``,
``tuple``, etc. as much as possible. This enables convenient
manipulation of these high-level types from C++::

    dict d;
    d["some"] = "thing";
    d["lucky_number"] = 13;
    list l = d.keys();

This almost looks and works like regular Python code, but it is pure C++.

=================
 Thinking hybrid
=================

For many applications runtime performance considerations are very
important. This is particularly true for most scientific applications.
Often the performance considerations dictate the use of a compiled
language for the core algorithms. Traditionally the decision to use a
particular programming language is an exclusive one. Because of the
practical and mental difficulties of combining different languages many
systems are written in just one language. This is quite unfortunate
because the price payed for runtime performance is typically a
significant overhead due to static typing. For example, our experience
shows that developing maintainable C++ code is typically much more
time-consuming and requires much more hard-earned working experience
than developing useful Python code. A related observation is that many
compiled packages are augmented by some type of rudimentary scripting
layer. These ad hoc solutions clearly show that many times a compiled
language alone does not get the job done. On the other hand it is also
clear that a pure Python implementation is too slow for numerically
intensive production code.

Boost.Python enables us to *think hybrid* when developing new
applications. Python can be used for rapidly prototyping a
new application. Python's ease of use and the large pool of standard
libraries give us a head start on the way to a first working system. If
necessary, the working procedure can be used to discover the
rate-limiting algorithms. To maximize performance these can be
reimplemented in C++, together with the Boost.Python bindings needed to
tie them back into the existing higher-level procedure.

Of course, this *top-down* approach is less attractive if it is clear
from the start that many algorithms will eventually have to be
implemented in a compiled language. Fortunately Boost.Python also
enables us to pursue a *bottom-up* approach. We have used this approach
very successfully in the development of a toolbox for scientific
applications (scitbx) that we will describe elsewhere. The toolbox
started out mainly as a library of C++ classes with Boost.Python
bindings, and for a while the growth was mainly concentrated on the C++
parts. However, as the toolbox is becoming more complete, more and more
newly added functionality can be implemented in Python. We expect this
trend to continue, as illustrated qualitatively in this figure:

.. image:: python_cpp_mix.png

This figure shows the ratio of newly added C++ and Python code over
time as new algorithms are implemented. We expect this ratio to level
out near 70% Python. The increasing ability to solve new problems
mostly with the easy-to-use Python language rather than a necessarily
more arcane statically typed language is the return on the investment
of learning how to use Boost.Python. The ability to solve some problems
entirely using only Python will enable a larger group of people to
participate in the rapid development of new applications.

=============
 Conclusions
=============

The examples in this paper illustrate that Boost.Python enables
seamless interoperability between C++ and Python. Importantly, this is
achieved without introducing a third syntax: the Python/C++ interface
definitions are written in pure C++. This avoids any problems with
parsing the C++ code to be interfaced to Python, yet the interface
definitions are concise and maintainable. Freed from most of the
development-time penalties of crossing a language boundary, software
designers can take full advantage of two rich and complimentary
language environments. In practice it turns out that some things are
very difficult to do with pure Python/C (e.g. an efficient array
library with an intuitive interface in the compiled language) and
others are very difficult to do with pure C++ (e.g. serialization).
If one has the luxury of being able to design a software system as a
hybrid system from the ground up there are many new ways of avoiding
road blocks in one language or the other.

.. I'm not ready to give up on all of this quite yet

.. Perhaps one day we'll have a language with the simplicity and
   expressive power of Python and the compile-time muscle of C++.  Being
   able to take advantage of all of these facilities without paying the
   mental and development-time penalties of crossing a language barrier
   would bring enormous benefits.  Until then, interoperability tools
   like Boost.Python can help lower the barrier and make the benefits of
   both languages more accessible to both communities.

===========
 Footnotes
===========

.. [#mi] For hard-core new-style class/extension module writers it is
   worth noting that the normal requirement that all extension classes
   with data form a layout-compatible single-inheritance chain is
   lifted for Boost.Python extension classes.  Clearly, either
   ``Base1`` or ``Base2`` has to occupy a different offset in the
   ``Derived`` class instance.  This is possible because the wrapped
   part of BPL extension class instances is never assumed to have a
   fixed offset within the wrapper.

===========
 Citations
===========

.. [VELD1995] T. Veldhuizen, "Expression Templates," C++ Report,
   Vol. 7 No. 5 June 1995, pp. 26-31.
   http://osl.iu.edu/~tveldhui/papers/Expression-Templates/exprtmpl.html