Boost.Python rocks !
By Nico on Wednesday, December 7 2005, 12:24 - pytst - Permalink
After a few hours of struggle, I managed to build a Python binding of tst 0.97 using Boost.Python. The results are great, my (unscientific) performance benchmarks show a 25% performance improvement over the version using SWIG. This is not surprising since SWIG takes your C++ class, builds a flat-view of your class as a set of C functions, then build a Python wrapper around this set of functions.
A few remarks, though :
- Boost and Boost.Python is a whole new world to discover. If you want things to remain a bit under your control, you HAVE to forget about make, nmake or Visual Studio and learn a new project building tool, bjam. It does not go without its surprises (for example, as a hint, don't name the top-level directory of your project "boost". It will confuse bjam.). Also, I'm a bit worried about the "buildability" of
pytst. Up until now, it could be built usingpython setup.py install, now you have to use a whole new toolset. - What's cool is that the
.pydextension does not need a.pymodule. What's less cool is that it need a runtime DLL namedboost_python.dll(but it is built along with your own extension). - Boost.Python is quite clever, but it does not understand the
char* string, int string_lengthidiom. SWIG does not, either, but at least you can teach it using SWIG templates. To work around this I had to usestd::basic_stringwhich is recognized by Boost and mapped to a Pythonstrtype. Anyway, I have to changetstso that it usesbasic_strings instead of this idiom which is hackish and not supported by Delphi, either. Because, you know, the Yoono client is developed in Delphi (hint, hint). - Boost.Python is very clean and cool, using hardcore C++ meta-programming constructs. The problem is that you don't get to know what is the result of all this magic. There is no code generation, since everything is done through meta-programming. This is a bit disturbing because you have to have an absolute trust in Boost.Python and hope it always does The Right Thing™.
- Since it uses hardcore C++ meta-programming, I strongly advise you to refresh your knowledge on C++ templates. On the one hand, the mapping code is pretty simple and straightforward (see below), but on the other hand, as soon as you dwelve a bit further in the tricks and trades of overloading pure virtual functions in Python, things get quite hairy (see below, too).
To sum up : what's cool with Boost.Python ? Performance, and code like that :
class_< TST >("TST")
.def("put",&TST::put)
.def("__setitem__",&TST::put)
.def("get",&TST::get)
.def("__getitem__",&TST::get)
.def("get_or_build",&TST::get_or_build)
.def("remove",&TST::remove)
.def("write",&TST::write)
.def("read",&TST::read)
.def("walk",&TST::walk)
.def("close_match",&TST::close_match)
.def("prefix_match",&TST::prefix_match)
.def("pack",&TST::pack)
;
What's less cool is code like that :
template <class S, class T> class string_action_wrapper :
public action<S,T>,
public wrapper< action<S,T> > {
public:
void perform(S* string,int string_length,int remaining_distance,T data) {
return call<void,std::basic_string<S>,int,T>(
this->get_override("perform").ptr(),
std::basic_string<S>(string,string_length),
remaining_distance,
data
);
}
T result() {
return call<T>(this->get_override("result").ptr());
}
};
But who cares as long as we get some performance ! Mu-HAHAHAHA ! Err, sorry...


Comments
Hello Nicolas,
What swig version were you using?, I ask this because
the latest version, in CVS, has much much better performance.
Also, did you now you can capture the 'char *str, int lenght'
idiom by just using
%include cstring.i
%cstring_input_binary(char *string,int string_length);
Also, now you can use
swig -O -c++ -python ...
where '-O' enables several new optimizations introduced after
version 1.3.27.
Let me now if you want and/or need some help for testing it.
I am looking for users/applications which require a lot of performance, so, I would like to know how your application
behaves with the new swig version.
Marcelo
I was using SWIG 1.3.27, so I guess I haven't used the latest optimizations that can be found in the CVS version. I'll give it a try as soon as I've managed to build it for Win32.
Thanks for the info !
Ok, try now the prerelease candidate:
A second release candidate for SWIG-1.3.28 is online at
www.dabeaz.com/swig/swig-...
www.dabeaz.com/swig/swigw...
....
Marcelo