[python] Function objects in place of member functions

View: New views
12 Messages — Rating Filter:   Alert me  

[python] Function objects in place of member functions

by Ravi-41 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello,
  If a free function 'func' has X* as its first argument, then, boost.python
allows it to be bound to a member function on the python side, i.e., the
following is legal:
  void func( X* x, arg1_t arg ) { ... }
  class_<X>( "X" ).def( "func", &func );

In order to use a function object in place of a free function, one must
specialize/overload
  boost::python::detail::get_signature
which, for some reason, does not account for function objects. Here's a very
simple example that works:

----------------------------------------------
#include <boost/mpl/vector.hpp>

struct X { int y; };

// Function object
struct Z { int operator()( X *x, int z ) { return z + x->y; } };

namespace boost { namespace python { namespace detail {
mpl::vector<int, X*, int> get_signature( Z&, X* )
  {return mpl::vector<int, X*, int>();}
}}}

#include <boost/python/class.hpp>
#include <boost/python/module.hpp>

BOOST_PYTHON_MODULE( mft ) {
  boost::python::class_<X>( "X" )
    .def( "z", Z() ).def_readwrite( "y", &X::y ); }
-------------------------------------------------

However, note that the overload of get_signature precedes the inclusion of the
boost.python headers, which is extremely inconvenient. However, if the headers
are moved to their proper location as in the following,

----------------------------------------------
#include <boost/mpl/vector.hpp>
#include <boost/python/class.hpp>
#include <boost/python/module.hpp>

struct X { int y; };

// Function object
struct Z { int operator()( X *x, int z ) { return z + x->y; } };

namespace boost { namespace python { namespace detail {
boost::mpl::vector<int, X*, int> get_signature( Z&, X* )
  {return boost::mpl::vector<int, X*, int>();}
}}}

BOOST_PYTHON_MODULE( mft ) {
  boost::python::class_<X>( "X" )
    .def( "z", Z() ).def_readwrite( "y", &X::y ); }
-------------------------------------------------

the compilation fails with the following error message (gcc 4.4.1 with boost
1.37 or 1.39):

$ g++ -Wall -shared -o mft.so memft.cc -lboost_python-mt -
I/usr/include/python2.6 -fPIC
In file included from memft.cc:1:
/usr/include/boost/python/class.hpp: In member function ‘void
boost::python::class_<T, X1, X2, X3>::def_impl(T*, const char*, Fn, const
Helper&, ...) [with T = X, Fn = Z, Helper =
boost::python::detail::def_helper<const char*,
boost::python::detail::not_specified, boost::python::detail::not_specified,
boost::python::detail::not_specified>, W = X, X1 =
boost::python::detail::not_specified, X2 =
boost::python::detail::not_specified, X3 =
boost::python::detail::not_specified]’:
/usr/include/boost/python/class.hpp:235:   instantiated from
‘boost::python::class_<T, X1, X2, X3>& boost::python::class_<T, X1, X2,
X3>::def(const char*, F) [with F = Z, W = X, X1 =
boost::python::detail::not_specified, X2 =
boost::python::detail::not_specified, X3 =
boost::python::detail::not_specified]’
memft.cc:16:   instantiated from here
/usr/include/boost/python/class.hpp:536: error: no matching function for call
to ‘get_signature(Z&, X*)’

Why is the overloaded get_signature not picked up when it is declared *after*
the inclusion of the headers?

The reason for using function objects is to change the argument types of a
member function of X as seen by python; for example, the functor would have an
argument type 'double' and would then internally convert the received 'double'
argument from the python side into some custom type prior to passing it to the
member function of X. In other words, the functor would act as a C++
equivalent of a python decorator.

Regards,
Ravi

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] Function objects in place of member functions

by troy d. straszheim :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ravi wrote:
[snip]
>
> In order to use a function object in place of a free function, one must
> specialize/overload
>   boost::python::detail::get_signature
> which, for some reason, does not account for function objects. Here's a very
> simple example that works:
>
[snip]
>
> However, note that the overload of get_signature precedes the inclusion of the
> boost.python headers, which is extremely inconvenient. However, if the headers
> are moved to their proper location as in the following,
>
[snip]
>
> Why is the overloaded get_signature not picked up when it is declared *after*
> the inclusion of the headers?
>

I'm not sure why it isn't picked up.  I've been working in this area,
replacing most of detail/caller.hpp and detail/invoke.hpp with
boost.fusion, seen here:

http://gitorious.org/~straszheim/boost/straszheim/blobs/python/boost/python/detail/caller.hpp

In the process, I overhauled get_signature to use boost::function_types,
and to be a metafunction, not a function:

http://gitorious.org/~straszheim/boost/straszheim/blobs/python/boost/python/signature.hpp

The overall effect is a lot less preprocessor stuff to look around.

I have function objects (given a specialization of get_signature), and
boost::function (only up to 3 arguments ATM) working,
here's the test (which passes):

/////////////////////////////////////

#include <boost/mpl/vector.hpp>
#include <boost/function.hpp>
#include <boost/python/class.hpp>
#include <boost/python/module.hpp>

namespace mpl = boost::mpl;

struct X { int y; };

struct FnObject
{
   typedef int result_type;

   int operator()(X *x, int z) const
   {
     return z + x->y;
   }
};

int f1(X* x) { return x->y ; }
int f2(X* x, int i) { return x->y * i; }
int f3(X* x, int i, int j) { return x->y * i + j; }

namespace boost {
   namespace python {
     namespace detail {

       template<>
       struct get_signature<FnObject, X>
       {
        typedef mpl::vector<int, X*, int> type;
       };

     }
   }
}

using namespace boost::python;

BOOST_PYTHON_MODULE( function_objects_ext )
{
   FnObject fobj;
   boost::function<int(X*, int)> bf0(fobj);

   boost::function<int(X*)> bf1(f1);
   boost::function<int(X*, int)> bf2(f2);
   boost::function<int(X*, int, double)> bf3(f3);

   boost::python::class_<X>( "X" )
     .def( "fobj", fobj)
     .def( "bf0",  bf0)
     .def( "bf1",  bf1)
     .def( "bf2",  bf2)
     .def( "bf3",  bf3)
     .def_readwrite( "y", &X::y )
     ;
}

/////////////////////////////

 >>> from function_objects_ext import *
 >>> x = X()
 >>> x.y = 13
 >>> x.fobj(12)
25
 >>> x.bf1()
13
 >>> x.bf2(2)
26
 >>> x.bf3(2, -27)
-1

/////////////////////////////

I'm fairly new to the internals of boost.python, and only just now got
this working...  Do you see problems with this, specifically the
conversion of get_signature from function to metafunction?

-t


_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] Function objects in place of member functions

by Ravi-41 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Sunday 11 October 2009 19:44:29 troy d. straszheim wrote:
> > Why is the overloaded get_signature not picked up when it is declared
> > after  the inclusion of the headers?
>
> I'm not sure why it isn't picked up.

Does that mean that you can reproduce the problem I pointed out?

> I've been working in this area,
> replacing most of detail/caller.hpp and detail/invoke.hpp with
> boost.fusion, seen here:
>
> http://gitorious.org/~straszheim/boost/straszheim/blobs/python/boost/python
> /detail/caller.hpp
>
> In the process, I overhauled get_signature to use boost::function_types,
> and to be a metafunction, not a function:
>
> http://gitorious.org/~straszheim/boost/straszheim/blobs/python/boost/python
> /signature.hpp

IMHO, this is the right way to do it. This avoids relying on the compiler to
optimize out all the ugly tag-dispatching. Of course, Dave A & Ralf WGK did
not have function types when they wrote this originally.

[snip]

>    boost::function<int(X*, int)> bf0(fobj);

Why do you need to use boost::function here? Shouldn't the type be deduced
automatically?

> I'm fairly new to the internals of boost.python, and only just now got
> this working...  Do you see problems with this, specifically the
> conversion of get_signature from function to metafunction?

I don't see any problems with the conversion of get_signature to a
metafunction. Do compile times get any longer?

Regards,
Ravi

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] Function objects in place of member functions

by troy d. straszheim :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ravi wrote:
> On Sunday 11 October 2009 19:44:29 troy d. straszheim wrote:
>>> Why is the overloaded get_signature not picked up when it is declared
>>> after  the inclusion of the headers?
>> I'm not sure why it isn't picked up.
>
> Does that mean that you can reproduce the problem I pointed out?
>

Yes I can.

>> I've been working in this area,
>> replacing most of detail/caller.hpp and detail/invoke.hpp with
>> boost.fusion, seen here:
>>
>> http://gitorious.org/~straszheim/boost/straszheim/blobs/python/boost/python
>> /detail/caller.hpp
>>
>> In the process, I overhauled get_signature to use boost::function_types,
>> and to be a metafunction, not a function:
>>
>> http://gitorious.org/~straszheim/boost/straszheim/blobs/python/boost/python
>> /signature.hpp
>
> IMHO, this is the right way to do it. This avoids relying on the compiler to
> optimize out all the ugly tag-dispatching. Of course, Dave A & Ralf WGK did
> not have function types when they wrote this originally.
>

Yes, the code was all dated ~2002.  There are more corners of the code
that could use modernization as well.

> [snip]
>
>>    boost::function<int(X*, int)> bf0(fobj);
>
> Why do you need to use boost::function here? Shouldn't the type be deduced
> automatically?
>

Note that the .def() of the various boost::function objects work without
requiring the user to specialize get_signature, (since this modified
get_signature metafunction knows about boost::function objects).  Here's
a different (working) example:

#include <boost/function.hpp>
#include <boost/lexical_cast.hpp>
#include <boost/python.hpp>
#include <boost/python/class.hpp>
#include <boost/python/module.hpp>

struct X { int y; };

struct add_to
{
   template <typename T>
   T operator()(X* x, T t) const
   {
     return t + x->y;
   }

   std::string operator()(X* x, std::string s)
   {
     return s + boost::lexical_cast<std::string>(x->y);
   }

};

using namespace boost::python;

BOOST_PYTHON_MODULE( function_objects_ext )
{
   add_to add_to_inst;
   boost::function<int(X*, int)> f_int(add_to_inst);
   boost::function<std::string(X*, std::string)> f_string(add_to_inst);

   boost::python::class_<X>( "X" )
     .def_readwrite( "y", &X::y )
     .def( "add_to", f_int)
     .def( "add_to", f_string)
     ;
}

 >>> from function_objects_ext import *
 >>> x = X()
 >>> x.y = 13
 >>> x.add_to(1)
14
 >>> x.add_to("foo")
'foo13'

Some musings:

I think interoperability with boost::function is a Good Thing, but in
the example above (and the previous), all it does is give def() a way to
tell what the signature of the function object is at compile time, and
it does so at a runtime cost.   A specialization of get_signature works,
but strikes me as kinda ugly (it should at least be moved out of
'detail' if it is a user customization point).  How about something like

   class_<T>("T")
     .def("f1", f1, sig<void(T*, double)>)
     .def<void(T*, double)>("fi", f1)

Which looks pretty straightforward when dealing with function objects,
but when dealing with overloads, things get tricky:

   void f2(T*, int);
   void f2(T*, double);

   class_<T>("T")
     .def("f2", f2)                     //  #1 error
     .def("f2", (void(*)(T*, int))f2)   //  #2 ok but ugly
     .def<void(T*, int)>("f2", f2)      //  #3 works but tricky
     ;

#1 is an error, because f2's type is undetermined.

#2 is how you can fix #1 in current practice (or with an alias).

#3 could work (I've tried, it works) if def, when passed a function
type, only accepts pointers to functions of that type, so that the
overload is resolved.  But how to use this same form to also specify the
types of arbitrary function objects... haven't thought about that yet.

>> I'm fairly new to the internals of boost.python, and only just now got
>> this working...  Do you see problems with this, specifically the
>> conversion of get_signature from function to metafunction?
>
> I don't see any problems with the conversion of get_signature to a
> metafunction. Do compile times get any longer?

I haven't checked this yet.  I have other concerns re typechecking and
automatic conversions...  I'll go back and look more closely.

-t


_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] Function objects in place of member functions

by Ravi-41 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Monday 12 October 2009 08:47:22 troy d. straszheim wrote:

> >>    boost::function<int(X*, int)> bf0(fobj);
> >
> >
> > Why do you need to use boost::function here? Shouldn't the type be
> > deduced  automatically?
> >
>
> Note that the .def() of the various boost::function objects work without
> requiring the user to specialize get_signature, (since this modified
> get_signature metafunction knows about boost::function objects).

[snip]

> I think interoperability with boost::function is a Good Thing, but in
> the example above (and the previous), all it does is give def() a way to
> tell what the signature of the function object is at compile time, and
> it does so at a runtime cost.

At least for my case, that extra virtual function call is a killer.

> A specialization of get_signature works,
> but strikes me as kinda ugly (it should at least be moved out of
> 'detail' if it is a user customization point).  How about something like
>
>    class_<T>("T")
>      .def("f1", f1, sig<void(T*, double)>)
>      .def<void(T*, double)>("fi", f1)

I'd rather have something along the lines of
      .def< mpl::vector<void,T*,double> >("f1",f1)
by using function_types to compose the currect function signature.
Alternatively, one could have the sig<> template above be a def_visitor and
take the appropriate template arguments without the need to modify class_::def
at all.

> >> I'm fairly new to the internals of boost.python, and only just now got
> >> this working...  Do you see problems with this, specifically the
> >> conversion of get_signature from function to metafunction?
> >
> >
> > I don't see any problems with the conversion of get_signature to a
> > metafunction. Do compile times get any longer?
>
> I haven't checked this yet.  I have other concerns re typechecking and
> automatic conversions...

Care to elaborate?

Regards,
Ravi


_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] python + phoenix

by troy d. straszheim :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ravi wrote:
 >
 > I'd rather have something along the lines of
 >       .def< mpl::vector<void,T*,double> >("f1",f1)

I have something working.  There is still a bunch of stuff to iron out
yet.  I went with

   def("name", as<Signature>(callable));

where Signature is the signature with which callable will be called,
e.g.

   struct times_seven
   {
     template <typename T> struct result;

     template <typename Self, typename T>
     struct result<Self(T&)>  // annoying: reference
     {
       typedef T type;
     };

     template <typename T>
     T operator()(const T& t)
     {
       return t * 7;
     }
   };

   BOOST_PYTHON_MODULE( function_objects3_ext )
   {
     def("times_seven", as<int(int)>(times_seven()));
   }

the T& in the struct result is something I haven't yet sorted out, has
something to do with how I am constructing fusion sequences from the
converted python objects before calling fusion::invoke.  Anyhow, it
works:

   >>> times_seven(7)
   49

I thought that one might try to remove the necessity of having the
return type in the Signature and/or implementing the result_of
protocol, but maybe it is nice that they don't have to match:

   def("times_seven", as<double(int)>(times_seven()));

   >>> times_seven(7)
   49.0

boost::function still works, and doesn't require as<>:

   boost::function<int(int, int)> myplus = std::plus<int>();
   def("myplus", myplus);

and old-school function objects:

   def("otherplus", std::plus<int>())

I was surprised to find that it all plays nice with boost::phoenix:

   def("plus", as<int(int, int)>(arg1 + arg2));

   def("throw_if_gt_5",
       as<void(int)>(if_(arg1 > 5)
                    [ throw_(std::runtime_error("too big!!")) ]
                    .else_
                    [ std::cout << val("okay")                ]));

And boost::phoenix plays nice with shared_ptr, which plays nice with
boost::python:

   struct X { int x; };   typedef boost::shared_ptr<X> XPtr;

   class_<X, XPtr>("X")
     .def_readwrite("x", &X::x)
     .def("add7", as<void(X*)>  (arg1->*&X::x      += 7))
     .def("add8", as<void(XPtr)>(arg1->*&X::x      += 8))
     .def("add9", as<void(X&)>  (bind(&X::x, arg1) += 9))
   ;

the other little thing is overload resolution, for which I propose a
little helper function "unload" (underload? deload? resolve?  I dunno):

   void foo(int);
   void foo(double);

   def("foo", unload<void(int)>(foo));

Which is safer than

   def("foo", (void(*)(int)) foo);

and just as safe as, and less verbose than

   void(*fooint)(int) = foo;
   def("foo", fooint);

So that's where I'm at...  what do you make of that interface?

-t

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] python + phoenix

by Ravi-41 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Wednesday 14 October 2009 21:59:42 troy d. straszheim wrote:

> boost::function still works, and doesn't require as<>:
>
>    boost::function<int(int, int)> myplus = std::plus<int>();
>    def("myplus", myplus);
>
> and old-school function objects:
>
>    def("otherplus", std::plus<int>())
>
> I was surprised to find that it all plays nice with boost::phoenix:
>
>    def("plus", as<int(int, int)>(arg1 + arg2));
>
>    def("throw_if_gt_5",
>        as<void(int)>(if_(arg1 > 5)
>                     [ throw_(std::runtime_error("too big!!")) ]
>                     .else_
>                     [ std::cout << val("okay")                ]));
>
> And boost::phoenix plays nice with shared_ptr, which plays nice with
> boost::python:
>
>    struct X { int x; };   typedef boost::shared_ptr<X> XPtr;
>
>    class_<X, XPtr>("X")
>      .def_readwrite("x", &X::x)
>      .def("add7", as<void(X*)>  (arg1->*&X::x      += 7))
>      .def("add8", as<void(XPtr)>(arg1->*&X::x      += 8))
>      .def("add9", as<void(X&)>  (bind(&X::x, arg1) += 9))
>    ;
>
> the other little thing is overload resolution, for which I propose a
> little helper function "unload" (underload? deload? resolve?  I dunno):
>
>    void foo(int);
>    void foo(double);
>
>    def("foo", unload<void(int)>(foo));
>
> Which is safer than
>
>    def("foo", (void(*)(int)) foo);
>
> and just as safe as, and less verbose than
>
>    void(*fooint)(int) = foo;
>    def("foo", fooint);
>
> So that's where I'm at...  what do you make of that interface?

This is very cool and pretty intuitive. My main concern is that one needs to
build up the function type in generic code (as opposed to mpl vectors), but we
do have function_types to help us.

You are going to have to propose this as several small patched, before Ralf &
Dave agree to it.

Regards,
Ravi

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] python + phoenix

by troy d. straszheim :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ravi wrote:
> On Wednesday 14 October 2009 21:59:42 troy d. straszheim wrote:
>>
>>    def("plus", as<int(int, int)>(arg1 + arg2));
>>
>
> This is very cool and pretty intuitive. My main concern is that one needs to
> build up the function type in generic code (as opposed to mpl vectors), but we
> do have function_types to help us.

I take it that you have a use-case where it is difficult to specify

   as<R(A1,A2)>(thing)

and easy to specify

   as<mpl::vector<R, A1, A2> >(thing)

Could you elaborate?  I'm quite sure there is a way to provide (roughly)
both interfaces:

   def("foo", as<int(double, string)>(functor()));

   def("foo", as<mpl::vector<int, double, std::string> >(functor()));

but I'm not sure I understand the problem.

> You are going to have to propose this as several small patched,
 > before Ralf & Dave agree to it.

Shit.  I knew this would happen, I've written too much code again.
Dave?   Ralf?

-t



_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] python + phoenix

by Ravi-41 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Saturday 31 October 2009 01:24:16 troy d. straszheim wrote:
> I take it that you have a use-case where it is difficult to specify
>
>    as<R(A1,A2)>(thing)
>
> and easy to specify
>
>    as<mpl::vector<R, A1, A2> >(thing)
>
> Could you elaborate?

I have some code that takes member function pointers of the form
  R Obj::*( A1, A2, A3, ..., An )
and converts them to a function object Q with the following signature:
  Rnew Q( Obj&, B1, B2, ..., Bn )
where
  Rnew = result_conversion_metafunction<R>::type
  Bi = arg_conversion_metafunction<Ai>::type
and Q is exposed via boost.python as a member of Obj.

For example, Ai could be a fixed-point number, with Bi being a double so that
the python side does not know anything about fixed-point numbers. An instance
of Q would convert eh floating point numbers passed from the python side into
fixed-point numbers, call the member function, and convert the returned value
to a double for use on the python side.

For these "converters", metafunctions from boost.function_types are used to
obtain the mpl::vector specifying result type and argument types. Extending
the technique above to function objects which take Obj* as their first
argument, I have a protocol which relies on the presence of a typedef called
'components' in the function object so that I can use the converter when
exposing via boost.python:

struct my_functor {
  typedef mpl::vector<R, Obj*, A1, A2, A3, A4> components;
  R operator()( Obj*, A1, A2, A3, A4 ) { ... }
};

// my_converter uses the components typedef
typedef typename my_converter<my_functor>::type Q;

Q q;
class_<Obj,...>( "Obj" )
  .def( "my_func", q )
  ;

Regards,
Ravi

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: [python] python + phoenix

by troy d. straszheim :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ravi wrote:

> On Saturday 31 October 2009 01:24:16 troy d. straszheim wrote:
>> I take it that you have a use-case where it is difficult to specify
>>
>>    as<R(A1,A2)>(thing)
>>
>> and easy to specify
>>
>>    as<mpl::vector<R, A1, A2> >(thing)
>>
>> Could you elaborate?
>
> I have some code that takes member function pointers of the form
>   R Obj::*( A1, A2, A3, ..., An )
> and converts them to a function object Q with the following signature:
>   Rnew Q( Obj&, B1, B2, ..., Bn )
> where
>   Rnew = result_conversion_metafunction<R>::type
>   Bi = arg_conversion_metafunction<Ai>::type
> and Q is exposed via boost.python as a member of Obj.
>
> For example, Ai could be a fixed-point number, with Bi being a double so that
> the python side does not know anything about fixed-point numbers.

I gather that custom converters aren't preferred because:

1.  It is a hassle to talk directly to the underlying raw PyObject
pointers and manage storage in the converter methods

2.  The converted type still leaks out to python in docstrings

(Both effects are visible in the custom string example:
http://www.boost.org/doc/libs/1_40_0/libs/python/doc/v2/faq.html#custom_string)

Are there others?  Or am I completely off?

 > An instance

> of Q would convert eh floating point numbers passed from the python side into
> fixed-point numbers, call the member function, and convert the returned value
> to a double for use on the python side.
>
> For these "converters", metafunctions from boost.function_types are used to
> obtain the mpl::vector specifying result type and argument types. Extending
> the technique above to function objects which take Obj* as their first
> argument, I have a protocol which relies on the presence of a typedef called
> 'components' in the function object so that I can use the converter when
> exposing via boost.python:
>
> struct my_functor {
>   typedef mpl::vector<R, Obj*, A1, A2, A3, A4> components;
>   R operator()( Obj*, A1, A2, A3, A4 ) { ... }
> };

To be sure I'm following you, correct me:

* for each member function signature that you wrap, you have a matching
my_functor that calls it
* my_functor does nothing but call the member function it wraps
* my_converter<my_functor>::type does the conversion from Rnew (B1, B2,
... Bn) to R (A1, A2, ...An) and calls my_functor.

Have I got it?

It is all very interesting, and to a certain degree duplicates some
functionality in detail/caller.hpp.  How do you generate these
my_functors...   preprocessor?  fusion?

I'm wondering if there isn't motivation here to cleanly integrate a
general facility for additional c++-side type conversions.  The
following came to mind, which imitates a function type style that
boost::proto uses extensively:

   struct hidden_type;  // python shouldn't see this at all

   struct int_2_hidden  // converts python-side 'int' to hidden
   {
     typedef hidden_type result_type;

     hidden_type operator()(int) const;
   };

   // fnobj takes a hidden, doesn't know it is wrapped in python
   struct fnobj {
     typedef void result_type;
     void operator()(float, hidden_type);
   };

   def("myfunctor", as<void(float, int_to_hidden(int))>(fnobj()));

where int_to_hidden(int) is a function type (not a function call, but it
later becomes a function call), indicating that what python passes
should be converted to int, then the int converted to hidden_type via an
instance of int_to_hidden, then the hidden_type passed to the underlying
instance of fnobj.

I realize this doesn't involve using the mpl::vectors you've already
calculated, just throwing it out there.

> // my_converter uses the components typedef
> typedef typename my_converter<my_functor>::type Q;

so Q could have a nested typedef (note I say Rnew, not R):

  typedef mpl::vector<Rnew, Obj*, B1, B2, B3, B4> components;

or use that in combination with function_types to synthesize

  typedef Rnew(Obj*, B1, B2, B3, B4) signature;

> Q q;
> class_<Obj,...>( "Obj" )
>   .def( "my_func", q )
>   ;

-t


_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Parent Message unknown Re: [python] python + phoenix

by Ravi-41 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

>> I have some code that takes member function pointers of the form
>>   R Obj::*( A1, A2, A3, ..., An )
>> and converts them to a function object Q with the following signature:
>>   Rnew Q( Obj&, B1, B2, ..., Bn )
>> where
>>   Rnew = result_conversion_metafunction<R>::type
>>   Bi = arg_conversion_metafunction<Ai>::type
>> and Q is exposed via boost.python as a member of Obj.
>>
>> For example, Ai could be a fixed-point number, with Bi being a double so
>> that
>> the python side does not know anything about fixed-point numbers.
>
> I gather that custom converters aren't preferred because:
>
> 1.  It is a hassle to talk directly to the underlying raw PyObject
> pointers and manage storage in the converter methods
>
> 2.  The converted type still leaks out to python in docstrings
>
> Are there others?  Or am I completely off?

Neither of the above is a big issue. In fact, python-side knowledge of the
converted type would even be mildly useful. The main problem is the number
of types:
  my_fixed_point_type< int bit_width, // 1 to 63
                       int binary_pt_location, // -63 to 63
                       bool signed,
                       property_tag // policies for rounding, etc.
                     >
  std::complex< my_fixed_point_type<...> >
  boost::numeric::ublas::array< ... >
  boost::numeric::ublas::matrix< ... >
There are simply far too many types to create converters for and to
register explicitly. Therefore, I use inline functions/functors to do all
the conversions on the C++ side, and python knows only about real and
complex doubles.

>  > An instance
>> of Q would convert eh floating point numbers passed from the python side
>> into
>> fixed-point numbers, call the member function, and convert the returned
>> value
>> to a double for use on the python side.
>>
>> For these "converters", metafunctions from boost.function_types are used
>> to
>> obtain the mpl::vector specifying result type and argument types.
>> Extending
>> the technique above to function objects which take Obj* as their first
>> argument, I have a protocol which relies on the presence of a typedef
>> called
>> 'components' in the function object so that I can use the converter when
>> exposing via boost.python:
>>
>> struct my_functor {
>>   typedef mpl::vector<R, Obj*, A1, A2, A3, A4> components;
>>   R operator()( Obj*, A1, A2, A3, A4 ) { ... }
>> };
>
> To be sure I'm following you, correct me:
>
> * for each member function signature that you wrap, you have a matching
> my_functor that calls it
> * my_functor does nothing but call the member function it wraps
> * my_converter<my_functor>::type does the conversion from Rnew (B1, B2,
> ... Bn) to R (A1, A2, ...An) and calls my_functor.
>
> Have I got it?

Yes.

> It is all very interesting, and to a certain degree duplicates some
> functionality in detail/caller.hpp.  How do you generate these
> my_functors...   preprocessor?  fusion?

Mostly boost.preprocessor (with some template metaprogramming using mpl).
Fusion doesn't work very well for me since the effort needed to get my
functors into a form usable from fusion is greater than just coding my own
invocation functions via boost.preprocessor; pseudo-code:

template <typename Func> struct my_converter
{
Func f_;

typedef typename ft::components<F>::type components;
typedef typename arg_converter<components>::type arg_types;
typedef typename result_converter<components>::type result_type;

// Shown for non-void return with one argument; can be generalized
// using enable_if for void types and using boost.preprocessor for
// different numbers of arguments

result_type operator()( typename at_c<1,arg_types>::type a1 )
{
  return convert_result<result_type>( f_(
       convert_arg<typename at_c<1,components>::type>( a1 ) ) );
}
};

template <typename Func>
my_converter<Func> make_converted_func( Func f )
{
  return my_converter<Func>( f );
}


> I'm wondering if there isn't motivation here to cleanly integrate a
> general facility for additional c++-side type conversions.  The
> following came to mind, which imitates a function type style that
> boost::proto uses extensively:
>
>    struct hidden_type;  // python shouldn't see this at all
>
>    struct int_2_hidden  // converts python-side 'int' to hidden
>    {
>      typedef hidden_type result_type;
>
>      hidden_type operator()(int) const;
>    };
>
>    // fnobj takes a hidden, doesn't know it is wrapped in python
>    struct fnobj {
>      typedef void result_type;
>      void operator()(float, hidden_type);
>    };
>
>    def("myfunctor", as<void(float, int_to_hidden(int))>(fnobj()));
>
> where int_to_hidden(int) is a function type (not a function call, but it
> later becomes a function call), indicating that what python passes
> should be converted to int, then the int converted to hidden_type via an
> instance of int_to_hidden, then the hidden_type passed to the underlying
> instance of fnobj.
>
> I realize this doesn't involve using the mpl::vectors you've already
> calculated, just throwing it out there.

Such a general facility would probably be better than my code and would be
greatly useful for me, but are there enough people out there with this use
case? I don't care about using mpl::vector if there is a more general
solution.

>> // my_converter uses the components typedef
>> typedef typename my_converter<my_functor>::type Q;
>
> so Q could have a nested typedef (note I say Rnew, not R):
>
>   typedef mpl::vector<Rnew, Obj*, B1, B2, B3, B4> components;
>
> or use that in combination with function_types to synthesize
>
>   typedef Rnew(Obj*, B1, B2, B3, B4) signature;

Yes.

Regards,
Ravi



_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

function types as user-defined converters

by troy d. straszheim :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

>> I'm wondering if there isn't motivation here to cleanly integrate a
>> general facility for additional c++-side type conversions.


I got distracted with boost.cmake stuff for a while but just got some
things working.  Simple example:

//  type hidden from python
struct Hidden
{
   double value;
};

//
//  c++ interface that uses hidden type
//
std::string takes_hidden(Hidden h)
{
   return std::string("Got an Hidden containing ")
     + boost::lexical_cast<std::string>(h.value);
}

//
//  converter object
//
struct dbl_to_Hidden
{
   typedef Hidden result_type;

   result_type operator()(double v) const
   {
     Hidden h;
     h.value = v;
     return a;
   }
};

// the module
BOOST_PYTHON_MODULE(udconversions_ext)
{
   def("takes_hidden",
     as<std::string(dbl_to_Hidden(double))>(&takes_hidden));
}                  ^^^^^^^^^^^^^^^^^^^^^

And the python:

 >>> from udconversions_ext import *
 >>> takes_a(13)
'Got an A containing 13'


Here's another contrived example:


// Here we have a class that takes optional<int> arguments that
// python ought not know about:

struct S
{
   void set(optional<int> v)
   {
     i = v;
   }
   void add(boost::optional<int> oi)
   {
     if (i && oi)
       (*i) += (*oi);
   }
   std::string show()
   {
     if (i)
       return boost::lexical_cast<std::string>(*i);
     else
       return "<unset>";
   }
   optional<int> i;
};

// And a converter function object that converts a python::object
// to an optional<T> (holding a T) if the object is
// convertible to T, otherwise it returns an empty optional<T>:

template <typename T>
struct to_optional
{
   typedef optional<T> result_type;

   result_type operator()(const bp::object& o)
   {
     bp::extract<T> ex(o);
     if (ex.check())
       return optional<T>(ex());
     else
       return optional<T>();
   }
};


BOOST_PYTHON_MODULE(opt_ext)
{

   bp::class_<S>("S")
     .def("set", bp::as<void(S*, to_optional<int>(bp::object))>(&S::set))
     .def("add", bp::as<void(S*, to_optional<int>(bp::object))>(&S::add))
     .def("show", &S::show)      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     ;
};

And the result:

 >>> from opt_ext import *
 >>> s = S()
 >>> s.set(13)
 >>> s.show()
'13'
 >>> s.set(None)
 >>> s.show()
'<unset>'
 >>> s.set(None)
 >>> s.add(13)
 >>> s.show()
'<unset>'
 >>> s.set(13)
 >>> s.add(13)
 >>> s.show()
'26'

There are plenty of issues yet.

Currently, the converter type still leaks out to python in the signature:

     class S(Boost.Python.instance)
      |  add(...)
      |      add( (S)arg1, (object)arg2) -> None :
      |
      |          C++ signature :
      |              void add(S*,to_optional<int>
(*)(boost::python::api::object))


converters won't nest:

   def("foo", as<returnvalue(arg1, conv1(conv2(arg2)))>(&somefn)); // nah

I dont have a mechanism for hooking a user defined converter into the
*return* type.  I'm going to have to think about that one a bit more.
specifying a function type, whose return value is a pointer to a
function, is kinda nasty anyway:

   def("foo", as<(converter(*)(int))(bool, converter(float))>(&fn));

I'm just not sure it gets one anything other than terseness.  Maybe
a special return value policy is in order.

Also, there isn't a way to use an object's constructor as a converter,
That is, I'd like to support this syntax:

struct X {
   X(double);
};

void takes_X(X x);

def("takes_x", as<void(X(double))>(&takes_x));

But I think this involves adding some scaffolding so that
boost.python can differentiate between what in proto-ese is a Callable
Transform (what now works) and an Object Transform (convert
via constructor).

-t

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig