Nicolas Lehuen's Weblog

After a few hours of struggle, I managed to build a Python binding of tst 0.97 using Boost.Python. The results are great, my (unscientific) performance benchmarks show a 25% performance improvement over the version using SWIG. This is not surprising since SWIG takes your C++ class, builds a flat-view of your class as a set of C functions, then build a Python wrapper around this set of functions.

A few remarks, though :

• Boost and Boost.Python is a whole new world to discover. If you want things to remain a bit under your control, you HAVE to forget about make, nmake or Visual Studio and learn a new project building tool, bjam. It does not go without its surprises (for example, as a hint, don’t name the top-level directory of your project « boost ». It will confuse bjam.). Also, I’m a bit worried about the « buildability » of pytst. Up until now, it could be built using python setup.py install, now you have to use a whole new toolset.
• What’s cool is that the .pyd extension does not need a .py module. What’s less cool is that it need a runtime DLL named boost_python.dll (but it is built along with your own extension).
• Boost.Python is quite clever, but it does not understand the char* string, int string_length idiom. SWIG does not, either, but at least you can teach it using SWIG templates. To work around this I had to use std::basic_string which is recognized by Boost and mapped to a Python str type. Anyway, I have to change tst so that it uses basic_strings instead of this idiom which is hackish and not supported by Delphi, either. Because, you know, the Yoono client is developed in Delphi (hint, hint).
• Boost.Python is very clean and cool, using hardcore C++ meta-programming constructs. The problem is that you don’t get to know what is the result of all this magic. There is no code generation, since everything is done through meta-programming. This is a bit disturbing because you have to have an absolute trust in Boost.Python and hope it always does The Right Thing™.
• Since it uses hardcore C++ meta-programming, I strongly advise you to refresh your knowledge on C++ templates. On the one hand, the mapping code is pretty simple and straightforward (see below), but on the other hand, as soon as you dwelve a bit further in the tricks and trades of overloading pure virtual functions in Python, things get quite hairy (see below, too).

To sum up : what’s cool with Boost.Python ? Performance, and code like that :

class_< TST >("TST")
         .def("put",&TST::put)
         .def("__setitem__",&TST::put)
         .def("get",&TST::get)
         .def("__getitem__",&TST::get)
         .def("get_or_build",&TST::get_or_build)
         .def("remove",&TST::remove)
         .def("write",&TST::write)
         .def("read",&TST::read)
         .def("walk",&TST::walk)
         .def("close_match",&TST::close_match)
         .def("prefix_match",&TST::prefix_match)
         .def("pack",&TST::pack);

What’s less cool is code like that :

template <class S, class T> class string_action_wrapper :public action<S,T>, public wrapper< action<S,T> > {
     public:
         void perform(S* string,int string_length,int remaining_distance,T data) {
             return call<void,std::basic_string<S>,int,T>(
                 this->get_override("perform").ptr(),
                 std::basic_string<S>(string,string_length),
                 remaining_distance,
                 data
             );
         }

         T result() {
             return call<T>(this->get_override("result").ptr());
         }
}; 

But who cares as long as we get some performance ! Mu-HAHAHAHA ! Err, sorry…