Python converters for raw pointers: request for help

[Michele De Stefano]

Hello to everyone.

I'm trying to write "from Python" converters for PETSc objects of the
petsc4py package.
Even if no-one of you uses petsc4py, I think you can help me, if I
explain some basics.
Please, be patient.

I want to make converters for the PETSc Mat type.

Mat is defined as below:

typedef struct _p_Mat*           Mat;

where _p_Mat is a structure whose details are not interesting for my problem.

Now, I managed to make a functioning "to Python converter".

My problem is the "from Python converter".

I made a test function

void print_Mat(Mat m) {
    ... // The implementation is not interesting
};

And I tried to make a "from Python converter" with the code below
(please, look at it ... it's not too long and it's very simple):


struct PETSc_Mat_from_Python {
       
        PETSc_Mat_from_Python() {

                static bool do_registration(true);
               
                if (do_registration) {
                       
                        boost::python::converter::registry::push_back(
                                        &convertible,&construct,boost::python::type_id<Mat>());
                       
                        do_registration = false;
                }
        }
       
        static void* convertible(PyObject* obj_ptr) {
               
                // In the line below, PyPetscMat_Get takes a PyObject*
as input and returns a Mat
                // (if it succedes) or a PETSC_NULL (if it fails).
This is a function of the petsc4py C API.
                // The ownership of the returned Mat is not
transferred (I think)

                if (PyPetscMat_Get(obj_ptr) == PETSC_NULL) return NULL;
               
                return obj_ptr;
        }
       
        static void construct(PyObject* obj_ptr,
                        boost::python::converter::rvalue_from_python_stage1_data* data) {

                void* storage = (
                                (boost::python::converter::
                                rvalue_from_python_storage<Mat>*)data)->storage.bytes;
               
                data->convertible = storage;
               
                // NOTE: The instruction below does not transfer
ownership, but this is not my problem, for now
                new (storage) Mat(PyPetscMat_Get(obj_ptr));
        }
};


Then, obviously ...

BOOST_PYTHON_MODULE(petsc_ext) {

        ... some not important lines ...

        PETSc_Mat_from_Python(); // Registers the converter
       
        def("print_Mat",print_Mat);
}


The code compiles without errors.
The problem is that when calling "print_Mat" from the Python shell,
the interpreter gives the error:

print_Mat(m)  # where m was previously and correctly built from Python
and it is a Python object containing a Mat

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
Boost.Python.ArgumentError: Python argument types in
    petsc_ext.print_Mat(petsc4py.PETSc.Mat)
did not match C++ signature:
    print_Mat(_p_Mat*)


I tried also to modify "void print_Mat(Mat m)" into "void
print_Mat(const Mat m)" or "void print_Mat(const Mat& m)", but the
result is the same.

I tried also to debug the code, but the debugger does not enter into
the "convertible" method, so I guess there is something wrong that I
can't see.

So may be I am missing something for the process of writing a "from
Python converter" for pointer arguments.

Can anyone help me, please ?


--
Michele De Stefano
http://www.linkedin.com/in/micdestefano
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Randolph Fritz-2]

Do you want to return an opaque type, would you like to expose some
of the mat's methods in Python, or would you perhaps like to convert a
numpy Matrix to a PETSc mat?  These all take different approaches.
Could you perhaps give an example of some of the Python code you would
like to write?
--
Randolph Fritz
  design machine group, architecture department, university of washington
rfritz@... -or- rfritz333@...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Randolph Fritz-2]

Additional thought.  Take a look at:
  <http://wiki.python.org/moin/boost.python/PointersAndSmartPointers#OSGexample>
& see if it helps clarify some of the problems.

--
Randolph Fritz
  design machine group, architecture department, university of washington
rfritz@... -or- rfritz333@...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Michele De Stefano]

Ok. Let me clarify (I looked at the link you suggested, but I've not
found the answers I was looking for).

There exist a "petsc4py" package, which can be extended using its C
API. This package, already exposes PETSc types (and also the Mat
type). petsc4py is not programmed using Boost Python.

Now, assume I already have a C++ function that takes a Mat parameter;
to make it simple, let's take

void print_Mat(Mat m) {
   ... // The implementation is not interesting
};

I'd like to expose this function to Python, with a seamless
integration mechanism, using Boost Python.

Unfortunately, if I use the petsc4py API, I have to expose a wrapper
for "print_Mat", in this way:

void print_Mat_Wrap(const object& mat_obj) {
    Mat m( PyPetscMat_Get(mat_obj.ptr()) );
    ... // The rest is not important
}

I don't know anything of the petsc4py implementation, but I know that
it already exposes PETSc Mat to Python. I know also that, using the
petsc4py C API, the only way to retrieve a Mat from a
PyObject* is through the PyPetscMat_Get function, whose signature is this one:

Mat (*PyPetscMat_Get)(PyObject *);

I know for sure that Mat objects are not allocated with the C++ new
operator (I don't know if this is a useful information for you).

What I'd really like to do, would be to write a from-Python-converter
from a PyObject* (containing a petsc4py exposed Mat) into a Mat, in
order to be able to directly expose my "void print_Mat(Mat m)" without
writing any wrapper ... but I didn't manage to properly make it ...

Is this possible ?


2009/10/26 Randolph Fritz <rfritz333@...>:

> Additional thought.  Take a look at:
>  <http://wiki.python.org/moin/boost.python/PointersAndSmartPointers#OSGexample>
> & see if it helps clarify some of the problems.
>
> --
> Randolph Fritz
>  design machine group, architecture department, university of washington
> rfritz@... -or- rfritz333@...
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig@...
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

--
Michele De Stefano
http://www.linkedin.com/in/micdestefano
http://xoomer.virgilio.it/michele_de_stefano
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Michele De Stefano]

I was forgetting to give an example of the Python code I would like to write:

>>> from petsc4py import PETSc

... now some uninteresting code for building a PETSc.Mat python object ...
... the result will be a PETSc.Mat m object ....
... once created, if I write "m" and press RETURN, this is the output ...

>>> m
<petsc4py.PETSc.Mat object at 0xb7bea548>

... so, basically, I already have a PETSc Mat in Python, because it is
exposed by the petsc4py package ...

... Now I want to pass this Python object to my "print_Mat" function ...
... What I'd like to write is ...

>>> form my_extension import *
>>> print_Mat(m)

I hope now it's all clear.

Thank you in advance.
Michele

2009/10/26 Michele De Stefano <micdestefano@...>:

> Ok. Let me clarify (I looked at the link you suggested, but I've not
> found the answers I was looking for).
>
> There exist a "petsc4py" package, which can be extended using its C
> API. This package, already exposes PETSc types (and also the Mat
> type). petsc4py is not programmed using Boost Python.
>
> Now, assume I already have a C++ function that takes a Mat parameter;
> to make it simple, let's take
>
> void print_Mat(Mat m) {
>   ... // The implementation is not interesting
> };
>
> I'd like to expose this function to Python, with a seamless
> integration mechanism, using Boost Python.
>
> Unfortunately, if I use the petsc4py API, I have to expose a wrapper
> for "print_Mat", in this way:
>
> void print_Mat_Wrap(const object& mat_obj) {
>    Mat m( PyPetscMat_Get(mat_obj.ptr()) );
>    ... // The rest is not important
> }
>
> I don't know anything of the petsc4py implementation, but I know that
> it already exposes PETSc Mat to Python. I know also that, using the
> petsc4py C API, the only way to retrieve a Mat from a
> PyObject* is through the PyPetscMat_Get function, whose signature is this one:
>
> Mat (*PyPetscMat_Get)(PyObject *);
>
> I know for sure that Mat objects are not allocated with the C++ new
> operator (I don't know if this is a useful information for you).
>
> What I'd really like to do, would be to write a from-Python-converter
> from a PyObject* (containing a petsc4py exposed Mat) into a Mat, in
> order to be able to directly expose my "void print_Mat(Mat m)" without
> writing any wrapper ... but I didn't manage to properly make it ...
>
> Is this possible ?
>
>
> 2009/10/26 Randolph Fritz <rfritz333@...>:
>> Additional thought.  Take a look at:
>>  <http://wiki.python.org/moin/boost.python/PointersAndSmartPointers#OSGexample>
>> & see if it helps clarify some of the problems.
>>
>> --
>> Randolph Fritz
>>  design machine group, architecture department, university of washington
>> rfritz@... -or- rfritz333@...
>>
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig@...
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>>
>
>
>
> --
> Michele De Stefano
> http://www.linkedin.com/in/micdestefano
> http://xoomer.virgilio.it/michele_de_stefano
>

--
Michele De Stefano
http://www.linkedin.com/in/micdestefano
http://xoomer.virgilio.it/michele_de_stefano
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Randolph Fritz-2]

I think what you want is here:
  <http://www.boost.org/doc/libs/1_39_0/libs/python/doc/tutorial/doc/html/python/object.html#python.extracting_c___objects>

and here:
  <http://www.boost.org/doc/libs/1_40_0/libs/python/doc/v2/extract.html>

Take a look at that--see if it helps.

--
Randolph Fritz
  design machine group, architecture department, university of washington
rfritz@... -or- rfritz333@...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Michele De Stefano]

That's not what I'd like to do. I'd like to make a "from-Python"
converter. I don't want to make a wrapper that takes a
boost::python::object and then call extract ...

The problem in my converter (see the previous e-mails) is that Python
does not even enter into the "convertible" method ...


2009/10/26 Randolph Fritz <rfritz333@...>:

> I think what you want is here:
>  <http://www.boost.org/doc/libs/1_39_0/libs/python/doc/tutorial/doc/html/python/object.html#python.extracting_c___objects>
>
> and here:
>  <http://www.boost.org/doc/libs/1_40_0/libs/python/doc/v2/extract.html>
>
> Take a look at that--see if it helps.
>
> --
> Randolph Fritz
>  design machine group, architecture department, university of washington
> rfritz@... -or- rfritz333@...
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig@...
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

--
Michele De Stefano
http://www.linkedin.com/in/micdestefano
http://xoomer.virgilio.it/michele_de_stefano
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Randolph Fritz-2]

On 2009-10-27, Michele De Stefano <micdestefano@...> wrote:
> That's not what I'd like to do. I'd like to make a "from-Python"
> converter. I don't want to make a wrapper that takes a
> boost::python::object and then call extract ...
>
> The problem in my converter (see the previous e-mails) is that Python
> does not even enter into the "convertible" method ...
>

Hunh.  Well, I'm out of ideas, then.  Sorry.

--
Randolph Fritz
  design machine group, architecture department, university of washington
rfritz@... -or- rfritz333@...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Ravi-41]

On Friday 23 October 2009 04:38:30 Michele De Stefano wrote:
> typedef struct _p_Mat*           Mat;

Creating a converter to/from a raw pointer is rather tricky (due to the way
argument type deduction handles pointers & references seamlessly, IIUC). You
should either expose _p_Mat or wrap the raw pointer in a struct and then
expose converters to it:
  struct Mat { struct _p_Mat *ptr; };
If you come up with a better way, I'd be interested.

Regards,
Ravi

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Michele De Stefano]

Thank you very much.

I had not thought to this solution. And I don't think there's a better one.

Thank you again.
Michele

2009/10/31 Ravi <lists_ravi@...>:

> On Friday 23 October 2009 04:38:30 Michele De Stefano wrote:
>> typedef struct _p_Mat*           Mat;
>
> Creating a converter to/from a raw pointer is rather tricky (due to the way
> argument type deduction handles pointers & references seamlessly, IIUC). You
> should either expose _p_Mat or wrap the raw pointer in a struct and then
> expose converters to it:
>  struct Mat { struct _p_Mat *ptr; };
> If you come up with a better way, I'd be interested.
>
> Regards,
> Ravi
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig@...
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

--
Michele De Stefano
http://www.linkedin.com/in/micdestefano
http://xoomer.virgilio.it/michele_de_stefano
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig