Nabble - Python - c++-sig

Re: Embedding Python - creating Python objects from C++

2009-11-30T11:09:48Z

On 29/11/09 7:50 PM, Sohail Somani wrote:
> // constructs a new instance of UserClass in Python using magic
> // but uses shared_ptr to keep alive
> shared_ptr<Base> base(/* magic here */);

This seemed to work:

// Boost Python/C++
class_<Base,shared_ptr<Base> >("Base")
.def(...)
;

# Python
class UserClass(Base):
...

// Back in C++
object = module["UserClass"]
shared_ptr<Base> ptr = extract<shared_ptr<Base> >(object());

I'll find out soon enough if it works like I'm expecting ;-)

--
Sohail Somani
http://uint32t.blogspot.com

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Embedding Python - creating Python objects from C++

2009-11-29T16:50:13Z

I have a class in C++ as follows:

class Base
{
virtual ~Base() /* ideally like = 0 but doesn't seem to work */;
};

I export this class to Python:

class_<Base>("Base",no_init);

The user is expected to create a derived class:

class UserClass(Base):
... stuff here ...

What I want to do then (in C++) is create instances of this Python class
that are stored using a shared pointer:

// constructs a new instance of UserClass in Python using magic
// but uses shared_ptr to keep alive
shared_ptr<Base> base(/* magic here */);

I understand that I can do something like module.attr("UserClass")() to
create an instance but that will return an object instance which doesn't
seem like it would play nice if I extracted a Base pointer from it.

So does anyone know how to implement the above referenced magic? It
sounds like it should be obvious, but I'm not sure.

Thanks, and sorry if this is a newb question!

Sohail

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Getting Boost.Python Working on MacOS X

2009-11-25T15:44:52Z

On 2009-11-23, Martin Hediger <ma.hed@...> wrote:
>
> I have been trying to get Boost.Python working on my MBP with Mac OS
> 10.5, since I need it to compile the chemical modeling software
> "AVOGADRO" for modification reasons.

There's something wrong with the boost.python test script on Mac OS X;
see <https://svn.boost.org/trac/boost/ticket/2185>. That's probably
what's making problems for you. Boost.python does in fact work on
Mac, despite the test script problem; here's what I did to make it go:
<http://mail.python.org/pipermail/cplusplus-sig/2009-October/014871.html>

--
Randolph Fritz
design machine group, architecture department, university of washington
rfritz@... -or- rfritz333@...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: boost python and weak_ptr from a shared_ptr argument

2009-11-25T12:13:50Z

This is a known problem. Boost.Python builds shared_ptr instances when calling your functions, using a custom deleter. I think internally it just uses regular Python reference counting to manage the lifetime of your KBObject.

Unfortunately I don't have a nice suggestion for you. A few years ago I worked around the issue in this way:

struct KBOject_data {
std::string name;
};

struct KBOject {
boost::shared_ptr<KBOject_data> data;
};

I.e. by hiding the shared_ptr from Boost.Python.
It isn't pretty. Maybe someone else has a better solution?
(I've always been wondering if enable_shared_from_this could make this nice.)

----- Original Message ----
From: Jahn Fuchs <jfuchs@...>
To: cplusplus-sig@...
Sent: Wed, November 25, 2009 8:16:58 AM
Subject: [C++-sig] boost python and weak_ptr from a shared_ptr argument

Hello boost python users,

I really like boost python but I stumbled in a problem I don't know how to solve, maybe you have an idea:

I have a class KnowledgeBase that holds an shared_ptr to an KBObject and also a weak_ptr to an KBObject. If I set the shared_ptr with the set_shared_ptr function it works fine, but it doesn't work with the set_weak_ptr function (which takes a shared_ptr as an argument) ...

see the following C++ code, python export code and python test code to see what i mean. I made a simple example which I also tested and it shows the problem I have.

Thank you in advance.
jahn.
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: building modules with MSVC

2009-11-25T10:41:51Z

It's certainly possible.

* Create a project in Visual Studio, set the output type to DLL.

* Go to Linker->Output and name the DLL as mymodule.pyd (pyd's are just DLLs on Windows).

* Set the Additional Library paths to where you keep your boost python stub lib for linking.

* You might need to specify the folder to the python lib too.

* Including the boost python headers will automatically cause the stub lib to be linked against.

This is from memory so might be missing out a step or two.

Hope this helps,

Brian

2009/11/25 Simon Pickles <sipickles@...>

Hi,

Is it possible to build boost::python modules using MSVC as opposed to bjam?

I'd like MSVC to output a pyd of course.

Thanks

Simon
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

boost python and weak_ptr from a shared_ptr argument

2009-11-25T08:16:58Z

Hello boost python users,

I really like boost python but I stumbled in a problem I don't know how
to solve, maybe you have an idea:

I have a class KnowledgeBase that holds an shared_ptr to an KBObject and
also a weak_ptr to an KBObject. If I set the shared_ptr with the
set_shared_ptr function it works fine, but it doesn't work with the
set_weak_ptr function (which takes a shared_ptr as an argument) ...

see the following C++ code, python export code and python test code to
see what i mean. I made a simple example which I also tested and it
shows the problem I have.

Thank you in advance.
jahn.

// C++ CODE:
-------------------------------------------------------------------------
class KBObject {
public:
std::string name;
};

class KnowledgeBase {
public:
boost::shared_ptr<KBObject> obj_shared_ptr;
boost::weak_ptr<KBObject> obj_weak_ptr;

void set_weak_ptr(boost::shared_ptr<KBObject> obj_ptr) {
obj_weak_ptr = boost::weak_ptr<KBObject>(obj_ptr);
};

void set_shared_ptr(boost::shared_ptr<KBObject> obj_ptr) {
obj_shared_ptr = obj_ptr;
};

boost::shared_ptr<KBObject> get_shared_ptr() {
return obj_shared_ptr;
};

boost::shared_ptr<KBObject> get_weak_ptr() {
return obj_weak_ptr.lock();
};

void print_weak_ptr() {
if (boost::shared_ptr<KBObject>
obj_ptr = obj_weak_ptr.lock())

std::cout << obj_ptr.get()->name << std::endl;
else
std::cout << "NO OBJECT" << std::endl;
};
void print_shared_ptr() {
if (obj_shared_ptr.get()!=0)
std::cout << obj_shared_ptr.get()->name
<< std::endl;
else
std::cout << "NO OBJECT" << std::endl;
};
};

// PYTHON MODULE EXPORT ----------------------------------------------
BOOST_PYTHON_MODULE(kbpython)
{
using namespace boost::python;

class_<KBObject, boost::shared_ptr<KBObject> >("KBObject")
.def_readwrite("name", &KBObject::name)
;

class_<KnowledgeBase>("KnowledgeBase")
.def("set_shared_ptr", &KnowledgeBase::set_shared_ptr)
.def("set_weak_ptr", &KnowledgeBase::set_weak_ptr)
.def("get_shared_ptr", &KnowledgeBase::get_shared_ptr)
.def("get_weak_ptr", &KnowledgeBase::get_weak_ptr)
.def("print_weak_ptr", &KnowledgeBase::print_weak_ptr)
.def("print_shared_ptr", &KnowledgeBase::print_shared_ptr)
;
}

// PYTHON TEST PROGRAM -----------------------------------------------

#!/usr/bin/env python
import kbpython

kb = kbpython.KnowledgeBase()

obj = kbpython.KBObject()
obj.name = "myObject"
kb.set_shared_ptr(obj)
kb.print_shared_ptr()

obj_2 = kb.get_shared_ptr()
obj_2.name = "change_name"
kb.print_shared_ptr()

kb.set_weak_ptr(obj_2)
kb.print_weak_ptr()

------------------------------------------------------------------------

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

building modules with MSVC

2009-11-25T05:11:06Z

Hi,

Is it possible to build boost::python modules using MSVC as opposed to bjam?

I'd like MSVC to output a pyd of course.

Thanks

Simon
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: multithreading and interpreters

2009-11-24T07:11:36Z

I'm sorry, I meant massively *patched* BP

-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com@...
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com@py
thon.org] On Behalf Of Matthew Scouten (TT)
Sent: Tuesday, November 24, 2009 8:58 AM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] multithreading and interpreters

Multiple threads is do-able, but you have to manage the GIL manually.
Multiple interpreters is more difficult. I think that it has been done
before by someone who massively BP, but I have never tried it.

-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com@...
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com@py
thon.org] On Behalf Of Trigve Siver
Sent: Sunday, November 22, 2009 10:56 AM
To: cplusplus-sig@...
Subject: [C++-sig] multithreading and interpreters

Hi,
I'm working on project where we used boost.python (embedded).

In general I have the "modules" where each module is something like
sandbox, is
isolated from others modules and run in solo thread. I want to be able
to use
python (and boost python functions of course) from each module thread.
Something
like that each thread could have own interpreter without need to take
care of GIL and
like. Is it possible? I know there could be some problems with GIL and
so. I've found in
python C API docs Py_NewInterpreter() function which could be of help
(or not?).

So will this configuration work? Could be there some problems with
boost.python
when using multiple threads?

Thanks

Trigve

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: multithreading and interpreters

2009-11-24T06:57:37Z

Multiple threads is do-able, but you have to manage the GIL manually.
Multiple interpreters is more difficult. I think that it has been done
before by someone who massively BP, but I have never tried it.

-----Original Message-----
From:
cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com@...
[mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com@py
thon.org] On Behalf Of Trigve Siver
Sent: Sunday, November 22, 2009 10:56 AM
To: cplusplus-sig@...
Subject: [C++-sig] multithreading and interpreters

Hi,
I'm working on project where we used boost.python (embedded).

In general I have the "modules" where each module is something like
sandbox, is
isolated from others modules and run in solo thread. I want to be able
to use
python (and boost python functions of course) from each module thread.
Something
like that each thread could have own interpreter without need to take
care of GIL and
like. Is it possible? I know there could be some problems with GIL and
so. I've found in
python C API docs Py_NewInterpreter() function which could be of help
(or not?).

So will this configuration work? Could be there some problems with
boost.python
when using multiple threads?

Thanks

Trigve

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Getting Boost.Python Working on MacOS X

2009-11-23T04:05:36Z

Dear All

I have been trying to get Boost.Python working on my MBP with Mac OS
10.5, since I need it to compile the chemical modeling software
"AVOGADRO" for modification reasons.
I pretty much followed the instructions on both the Boost, Getting
Started guide and the Boost.Python Introduction. I managed to get the
"Lambda" and the "Regex" examples working perfectly, but unfortunatly
not the python "embedding" and "extending" examples, neither as well
the "hello world". Apparently, no
embedding/extending module is being built for me to import in the
python-test script following the build.
My setup is as follows:

Boost root: /usr/local/boost_1_40_0/ (so I call every command with sudo).

The lambda and regex example files (.cpp) were perfectly compiling in a
directory ~/programming_tests/, so my compiler is very likely to work.
In order to do so, I issued the command:

cpp+ -I example.cpp -o example
/usr/local/boost_1_40_0/lib/libboost_regex-mt.a

for some (possibly to the overall problem related) reason, it does not
work with the dynamic
library .dylib.

What would be the basic procedure to get the boost.python examples
running on Mac OS X?
Thanks a lot for any suggestions.

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

multithreading and interpreters

2009-11-22T08:55:38Z

Hi,
I'm working on project where we used boost.python (embedded).

In general I have the "modules" where each module is something like sandbox, is
isolated from others modules and run in solo thread. I want to be able to use
python (and boost python functions of course) from each module thread. Something
like that each thread could have own interpreter without need to take care of GIL and
like. Is it possible? I know there could be some problems with GIL and so. I've found in
python C API docs Py_NewInterpreter() function which could be of help (or not?).

So will this configuration work? Could be there some problems with boost.python
when using multiple threads?

Thanks

Trigve

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: dynamic compile and "to-Python converter ... second conversion method ignored"

2009-11-21T11:05:04Z

On Fri, Nov 20, 2009 at 10:44 PM, Stefan Seefeld <seefeld@...> wrote:

>> Now I'm sure I've interpreted "scoping conversion operators" differently
>> than you meant it. Can you clarify?
>
> This is mostly only hand-waving, sorry. I was wondering whether it was
> possible to make the registry not an application-global object, but a
> per-module object. And if that's possible (and thus, conversions are by
> default only available to the defining module), would it be possible to
> chain them explicitly, to get both, well-defined semantics, as well as no
> ODR violations.
>
> I haven't thought about whether this has to involve runtime overhead, and if
> so, how much. At least without the chaining it may actually be cheap. (Just
> hide the registry symbol from other modules.)
> And even with chaining, this is merely a lookup in a chain of converter
> maps, which likely can be reordered / optimized at the point where one
> registry is explicitly added to another.
> Still, this is just a rough idea. I don't have any implementation ideas yet.

IMHO, this could be the way to go. I already meat this problem in the past:
* std containers were exposed using different suites
* during the testing - initially all my tests run in the same process,
but after 30 tests it became very difficult to name C++ classes :-),
so I was forced to run every test in its own process.

The "per-module" registry could be a solution to the problem.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Boost.Python + OpenGL segmentation faults

2009-11-20T13:34:17Z

> Back in 2005/2006 all that worked even with PyOpenGL imported under
> Windows.

I'm using boost 1.39 to embed Python 2.6 + PyOpenGL 3 in MSVC 2008 under
Windows XP right now, without any difficulties. I'm not using PyOpenGL
directly from C++, but have all the relevant code in a file pygl.py, and
run

exec("from pygl import *");

in my C++ code. I'm not sure if any of this is helpful, but you're very
welcome to the code samples if they would be of any use.

Best wishes,
Mohan
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: dynamic compile and "to-Python converter ... second conversion method ignored"

2009-11-20T12:44:19Z

On 11/20/2009 03:23 PM, troy d. straszheim wrote:

>
>> For example, what does 'replaceable' mean ? Does it mean two bindings
>> are identical or equivalent ?
>
> No it means only that the behavior in the situation above is modified.
>
>> Does it merely mean the user allows the mapping to change? (This
>> would lead into undefined behavior, if you don't carefully control in
>> which order things are loaded / initialized.)
>
> UB as in what happens when there is an ODR violation? How?

I'm only talking about user-visible behavior, and assuming that it could
be implemented in a way that would avoid ODR violations.
(Whether that's actually possible is a different matter. I'm only
talking about semantics.)

>
>> A different approach might be to avoid collisions by 'scoping'
>> conversion operators, so they are only available to specific
>> extension modules.
>
> Interesting idea. As I interpret "scoping conversion operators", I'm
> not sure the cost in runtime and complexity would be worth it, OTOH I
> don't have much to go on here.
>
> > May a converter registry be explicitly imported into a module,
>> during initialization?
>
> Now I'm sure I've interpreted "scoping conversion operators"
> differently than you meant it. Can you clarify?

This is mostly only hand-waving, sorry. I was wondering whether it was
possible to make the registry not an application-global object, but a
per-module object. And if that's possible (and thus, conversions are by
default only available to the defining module), would it be possible to
chain them explicitly, to get both, well-defined semantics, as well as
no ODR violations.

I haven't thought about whether this has to involve runtime overhead,
and if so, how much. At least without the chaining it may actually be
cheap. (Just hide the registry symbol from other modules.)
And even with chaining, this is merely a lookup in a chain of converter
maps, which likely can be reordered / optimized at the point where one
registry is explicitly added to another.
Still, this is just a rough idea. I don't have any implementation ideas yet.

>
>> That would involve making dependency between extension modules
>> explicit ("module X imports converters provided by module Y").
>
> But, you can do that now; "module X imports the converter for T
> provided by module Y" just means "module X doesn't wrap T and won't
> run without Y", right?

Yes. I'm talking about the case where module X and module Y provide a
converter for T, so the user who wishes to use X and Y needs to specify
the desired behavior, which could be "each uses its own" or "either of
the two will be used globally".

Stefan

--

...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: dynamic compile and "to-Python converter ... second conversion method ignored"

2009-11-20T12:23:43Z

Stefan Seefeld wrote:
>
> It's an interesting thought, but it raises a lot of questions, and opens
> a huge can of worms. (Welcome in the world of 'DLL hell' !)
> I'm not sure what angle to attack this problem from. In particular, I
> believe before thinking about interface issues, we should clarify the
> problem domain, and in particular, what semantics we want.

I thought this was obvious, sorry. The 'worst' and most common scenario
is, precisely put: more than one python extension module over which we
have no control has wrapped type T. We need to use them simultaneously.
For instance, here are two modules that both wrap vector<double>.
With assertions enabled, this happens:

>>> import converter_collisions1_ext
>>> import converter_collisions2_ext
python:
/home/troy/Projects/boost/src/libs/python/src/converter/registry.cpp:212:
void boost::python::converter::registry::insert(PyObject* (*)(const
void*), boost::python::type_info, const PyTypeObject* (*)()): Assertion
`slot->m_to_python == 0' failed.
zsh: abort python

delightfully, without assertions there is only a warning:

>>> import converter_collisions1_ext
>>> import converter_collisions2_ext
__main__:1: RuntimeWarning: to-Python converter for std::vector<double,
std::allocator<double> > already registered; second conversion method
ignored.

So one resolution is to remove the 'assert' that triggers this crash and
leave the semantics as-is: second and later converter registrations are
warned about and ignored. I'm simply suggesting we give the user more
control over this behavior, because in a number of situations (it is up
to the user to determine what they are) such control would be useful.

> For example, what does 'replaceable' mean ? Does it mean two bindings
> are identical or equivalent ?

No it means only that the behavior in the situation above is modified.

> Does it merely mean the user allows the
> mapping to change? (This would lead into undefined behavior, if you
> don't carefully control in which order things are loaded / initialized.)

UB as in what happens when there is an ODR violation? How?

> A different approach might be to avoid collisions by 'scoping'
> conversion operators, so they are only available to specific extension
> modules.

Interesting idea. As I interpret "scoping conversion operators", I'm
not sure the cost in runtime and complexity would be worth it, OTOH I
don't have much to go on here.

> May a converter registry be explicitly imported into a module,
> during initialization?

Now I'm sure I've interpreted "scoping conversion operators" differently
than you meant it. Can you clarify?

> That would involve making dependency between extension modules explicit
> ("module X imports converters provided by module Y").

But, you can do that now; "module X imports the converter for T provided
by module Y" just means "module X doesn't wrap T and won't run without
Y", right?

-t

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: dynamic compile and "to-Python converter ... second conversion method ignored"

2009-11-20T09:29:27Z

On 11/20/2009 11:47 AM, troy d. straszheim wrote:

> Stefan Seefeld wrote:
>>
>> (It's a global symbol that is seen by all DSOs. Having two DSOs
>> provide that symbol, but with different content, is illegal.)
>>
>
> I'd like to fix this, now, it is starting to cause me pain.
>
> For instance, it also rears its ugly head if you import two libraries
> that both wrap std::vector<double>, for instance if you try to use the
> IceTray physics analysis suite together with Paul Kunz's wonderful
> Hippodraw.
>
> Another situation that came up recently was that the pair<T,U> inside
> std::map<T, U> and another container of T,U (that used pair<T,U>
> internally) collided. I think it should be possible to specify that a
> type's wrapper is replaceable (if you know, for instance, that two
> projects wrap std::pair<string,int> the same way).
>
> I haven't thought about the interface much yet.

It's an interesting thought, but it raises a lot of questions, and opens
a huge can of worms. (Welcome in the world of 'DLL hell' !)
I'm not sure what angle to attack this problem from. In particular, I
believe before thinking about interface issues, we should clarify the
problem domain, and in particular, what semantics we want.

For example, what does 'replaceable' mean ? Does it mean two bindings
are identical or equivalent ? Does it merely mean the user allows the
mapping to change ? (This would lead into undefined behavior, if you
don't carefully control in which order things are loaded / initialized.)

A different approach might be to avoid collisions by 'scoping'
conversion operators, so they are only available to specific extension
modules. May a converter registry be explicitly imported into a module,
during initialization ?
That would involve making dependency between extension modules explicit
("module X imports converters provided by module Y"). In general, I
think that would be cleanest. "Explicit is better than implicit." I'm
sure David would like that. ;-)

Regards,
Stefan

--

...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: dynamic compile and "to-Python converter ... second conversion method ignored"

2009-11-20T08:47:09Z

Stefan Seefeld wrote:

> On 11/20/2009 05:50 AM, Eilif Mueller wrote:
>> Hi,
>>
>> I'm using boost.python to interface with some dynamically generated
>> C++ code, employing some scipy.weave facilities to cache the
>> results. When I force a recompile and import, I get the following
>> message:
>>
>> RuntimeWarning: to-Python converter for __randint already
>> registered; second conversion method ignored.
>>
>> where __randint is the dynamically generated class. As such, the
>> changes to the code (__randint) are not represented in the newly
>> imported module.
>>
>> Is it possible to override this behaviour so that the new class
>> replaces the registration for the old one?
>>
>
> I very much doubt that what you want is possible: All extension
> modules using boost.python share a single global 'registry' that
> holds type-conversion information. Thus it is impossible to have that
> loaded while a separate process is adding new entries to it.
>
> (It's a global symbol that is seen by all DSOs. Having two DSOs
> provide that symbol, but with different content, is illegal.)
>

I'd like to fix this, now, it is starting to cause me pain.

For instance, it also rears its ugly head if you import two libraries
that both wrap std::vector<double>, for instance if you try to use the
IceTray physics analysis suite together with Paul Kunz's wonderful
Hippodraw.

Another situation that came up recently was that the pair<T,U> inside
std::map<T, U> and another container of T,U (that used pair<T,U>
internally) collided. I think it should be possible to specify that a
type's wrapper is replaceable (if you know, for instance, that two
projects wrap std::pair<string,int> the same way).

I haven't thought about the interface much yet. Here's something I just
decided against, a per-type global 'replacability' flag, that warns or not:

template <typename T>
void replacable<T>(bool quietly);

BOOST_PYTHON_MODULE(mymod)
{
replacable<std::pair<string,int> >(true);
}

But this doesn't help the situation where you're trying to use two
different python modules together, neither of which specifies that pairs
are replacable. It seems that boost.python should expose some
way to specify replacability at runtime. How about this. A type's
converters are 'locked', 'verbose', and 'nonthrowing' by default (like
they are now).
Boost python modules expose one function

def __lock__(type, lock, verbose, throw): ...

at module scope that allows you to specify whether

- types are locked down or replacable
- collisions should be complained about to stdout
- collision events should throw

e.g.

>>> import M1

# silently replace T's
>>> M1.__lock__(M1.T, lock=False, verbose=False, throw=False)

# verbosely refuse replacement of tees
>>> M1.__lock__(M1.T, lock=True, verbose=True, throw=False)

And that this __lock__ function would also be available on the C++ side
for setting defaults:

BOOST_PYTHON_MODULE(M1)
{
class_<T>("T")
...

lock<T>(false, false, false); // silent, replacable, nonthrowing

}

Just a sketch. Thoughts?

-t

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: dynamic compile and "to-Python converter ... second conversion method ignored"

2009-11-20T05:00:56Z

On 11/20/2009 05:50 AM, Eilif Mueller wrote:

> Hi,
>
> I'm using boost.python to interface with some dynamically generated C++ code, employing some scipy.weave facilities to cache the results. When I force a recompile and import, I get the following message:
>
> RuntimeWarning: to-Python converter for __randint already registered; second conversion method ignored.
>
> where __randint is the dynamically generated class. As such, the changes to the code (__randint) are not represented in the newly imported module.
>
> Is it possible to override this behaviour so that the new class replaces the registration for the old one?
>

I very much doubt that what you want is possible: All extension modules
using boost.python share a single global 'registry' that holds
type-conversion information. Thus it is impossible to have that loaded
while a separate process is adding new entries to it.

(It's a global symbol that is seen by all DSOs. Having two DSOs provide
that symbol, but with different content, is illegal.)

Regards,
Stefan

--

...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

dynamic compile and "to-Python converter ... second conversion method ignored"

2009-11-20T02:50:01Z

Hi,

I'm using boost.python to interface with some dynamically generated C++ code, employing some scipy.weave facilities to cache the results. When I force a recompile and import, I get the following message:

RuntimeWarning: to-Python converter for __randint already registered; second conversion method ignored.

where __randint is the dynamically generated class. As such, the changes to the code (__randint) are not represented in the newly imported module.

Is it possible to override this behaviour so that the new class replaces the registration for the old one?

Thanks.

cheers,

Eilif

--
GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Boost.Python + OpenGL segmentation faults

2009-11-19T17:02:37Z

Just a remark:
We are heavily using Boost.Python extensions together with OpenGL, and have so for years.
It works for us without problems under Windows, Mac OS X, Linux.
We are not using PyOpenGL, though, but have our own Boost.Python-based wrappers for the
entire OpenGL library (http://cci.lbl.gov/gltbx/).
Back in 2005/2006 all that worked even with PyOpenGL imported under Windows.

I'd try running with valgrind to debug.

----- Original Message ----
From: Dimitri Tcaciuc <dtcaciuc@...>
To: cplusplus-sig@...
Sent: Thu, November 19, 2009 1:54:51 PM
Subject: [C++-sig] Boost.Python + OpenGL segmentation faults

Hello everyone,

I've recently posted a problem to SO
(http://stackoverflow.com/questions/1751408/boost-python-opengl-segmentation-faults),
where I haven't had much luck with answer.

My problem is that if I import OpenGL python libraries before
importing my B.P bindings, I get strange segmentation faults for no
apparent reason. I isolated a small test case which I'm attaching
below. The current boost library version I'm using is 1.37; you'll
have to make a change to Makefile if you have any other version
installed.

I'd greatly appreciate if anyone can give some insight on what's happening.

Dimitri.

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Boost.Python + OpenGL segmentation faults

2009-11-19T14:13:26Z

Dimitri Tcaciuc wrote:

> Hello everyone,
>
> I've recently posted a problem to SO
> (http://stackoverflow.com/questions/1751408/boost-python-opengl-segmentation-faults),
> where I haven't had much luck with answer.
>
> My problem is that if I import OpenGL python libraries before
> importing my B.P bindings, I get strange segmentation faults for no
> apparent reason. I isolated a small test case which I'm attaching
> below. The current boost library version I'm using is 1.37; you'll
> have to make a change to Makefile if you have any other version
> installed.
>
> I'd greatly appreciate if anyone can give some insight on what's happening.
>

The only thing that strikes me is that I'd just use the
vector_indexing_suite on the 'vector<A::ptr> elements'.

Beyond that, the next thing you'd want is a --with-pydebug
--without-pymalloc build of python (with associated opengl and
boost.python built against this library), get some stacktraces from
running under gdb, and run things under valgrind with the
valgrind-python.supp list from the Misc directory of the python
distribution. It is a bit of work to set up but you get a lot more
debuggability.

-t

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Boost.Python + OpenGL segmentation faults

2009-11-19T13:54:51Z

Hello everyone,

I've recently posted a problem to SO
(http://stackoverflow.com/questions/1751408/boost-python-opengl-segmentation-faults),
where I haven't had much luck with answer.

My problem is that if I import OpenGL python libraries before
importing my B.P bindings, I get strange segmentation faults for no
apparent reason. I isolated a small test case which I'm attaching
below. The current boost library version I'm using is 1.37; you'll
have to make a change to Makefile if you have any other version
installed.

I'd greatly appreciate if anyone can give some insight on what's happening.

Dimitri.

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

foo.cpp (2K) Download Attachment
glcrash.py (254 bytes) Download Attachment
Makefile (192 bytes) Download Attachment

Re: Efficient string passing

2009-11-19T05:56:39Z

On 11/19/2009 08:30 AM, Mohan Ganesalingam wrote:

>> I believe this is impossible: Python treats strings as immutable,
>> which means you can't manipulate them in-place, and there is no C API
>> function
>
> There is apparently one exception to this, which I was utilising:
> char* PyString_AsString(PyObject *string) Returns a NUL-terminated
> representation of the contents of string. The pointer refers to the
> internal buffer of string, not a copy. The data must not be modified
> in any way, unless the string was just created using
> PyString_FromStringAndSize(NULL, size).
>
> I.e. if you create a string using PyString_FromStringAndSize(0, n),
> then you can immediately modify its contents using PyString_AsString.
> AFAICS this functionality is (reasonably enough) not wrapped by
> boost::python::str, so you have to use the underlying API.

OK, perhaps I misunderstood your use-case. Indeed, you may allocate a
string using the Python C API, and have a small window within which you
can manipulate its content, before handing control back to the Python
runtime. If that is what you need, great.

For avoidance of doubt: I don't think it would be valid to modify that
string after you have handed control back to Python. The Python runtime
may well create hashes from it for efficient lookup, which build on the
immutability of this type of object.

Regards,
Stefan

--

...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Efficient string passing

2009-11-19T05:30:45Z

Thank you for the reply! As you spotted, I was making an embarrassingly
idiotic typo (due to too much cut and pasting)... it should of course have
been

handle<> h(result);
return object(h);

Once that was fixed, everything worked fine.

>I believe this is impossible: Python treats strings as immutable, which
>means you can't manipulate them in-place, and there is no C API function

There is apparently one exception to this, which I was utilising:

char* PyString_AsString(PyObject *string) Returns a NUL-terminated
representation of the contents of string. The pointer refers to the
internal buffer of string, not a copy. The data must not be modified in any
way, unless the string was just created using
PyString_FromStringAndSize(NULL, size).

I.e. if you create a string using PyString_FromStringAndSize(0, n), then
you can immediately modify its contents using PyString_AsString. AFAICS
this functionality is (reasonably enough) not wrapped by
boost::python::str, so you have to use the underlying API.

Thanks again for spotting that!

Best wishes,
Mohan
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Efficient string passing

2009-11-19T05:06:07Z

On 11/19/2009 07:30 AM, Mohan Ganesalingam wrote:
> Dear all,
>
> I recently sent an e-mail which (I think) didn't reach the list, about
> trying to return large strings from C++ into Python without copying them.

I believe this is impossible: Python treats strings as immutable, which
means you can't manipulate them in-place, and there is no C API function
that allows you to pass in a string by reference, as that would make
memory management impossibly complex.

> By looking at the C/Python API I've subsequently figured out roughly
> how to do this... something like
>
> Class World
> {
> object return_object()
> {
> PyObject *result = PyString_FromStringAndSize(0, 7);

This creates a new string, copying whatever you pass as input.

> // initialise the underlying string using PyString_AsString
> handle<> h(result);
> return object();

I'm not sure what your intent is, but returning a default-constructed
object is the C++ equivalent of None.

> }
>
> ...
>
>
> main_namespace["World"] = class_<World>("World")
> .def("return_object", &World::return_object);
>
> Unfortunately this doesn't work; when I call w.return_object(), I get
> back None.

...because above you "return object();", presumably.

> Returning a raw PyObject * instead of a boost::python::object works,
> but I can't tell if that involves a copy... and using
> return_value_policy<manage_new_object>() with a PyObject * return
> crashes.
>
> Any advice here would be very much appreciated!

I don't think you can get what you want using strings directly. You may
create and expose a wrapper type that provides a string-like interface,
with appropriate conversions.

Stefan

--

...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Efficient string passing

2009-11-19T04:30:16Z

Dear all,

I recently sent an e-mail which (I think) didn't reach the list, about
trying to return large strings from C++ into Python without copying them.
By looking at the C/Python API I've subsequently figured out roughly how to
do this... something like

Class World
{
object return_object()
{
PyObject *result = PyString_FromStringAndSize(0, 7);
// initialise the underlying string using PyString_AsString
handle<> h(result);
return object();
}

...

main_namespace["World"] = class_<World>("World")
.def("return_object", &World::return_object);

Unfortunately this doesn't work; when I call w.return_object(), I get back
None. Returning a raw PyObject * instead of a boost::python::object works,
but I can't tell if that involves a copy... and using
return_value_policy<manage_new_object>() with a PyObject * return crashes.

Any advice here would be very much appreciated!

Thank you and best wishes,
Mohan

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Boost.Python] tutorial python extension fails on import with ImportError: No module named hello_ext

2009-11-10T13:36:41Z

Hey all,

When using boost_1_38_0\libs\python\example\tutorial, I try to import the extension that is created (hello_ext) but get the exception, ImportError: No module named hello_ext

Here’s some details that might be needed:

OS: Windows Vista

Boost version: 1.38.0

Bjam version: 3.1.17

Python version: 2.6.2

My user-config.jam looks like this:

# MSVC configuration

using msvc : 9.0 ;

# Python configuration

using python

: 2.6 #version

: C:/Python26 #location

: C:/projects/external/python/Python-2.6.2/Include #includes

: C:/projects/external/python/Python-2.6.2/Lib #libraries

;

I am working with the tutorial example located here: boost_1_38_0\libs\python\example\tutorial. When I run Bjam after I cd to this directory, this tutorial example says that it passes all the tests and I can check the output in the resulting files within the hello.test directory. That's all well and good that the test can get this extension module to work, but I want to be able to import it myself in a python session or python file. That way I can see it work and then move on to creating my own extensions.

I modify Jamroot so that the last few lines where it creates tests are commented out. When I run bjam again, hello_ext.pyd remains (previously it was being created, used for the test, and then deleted). I try to run hello.py (which is the script that the test uses) from IDLE (which comes with the installation of python) and it throws an exception for my import statement

Traceback (most recent call last):

File "C:\Users\Charles\Desktop\boost_1_38_0\boost_1_38_0\libs\python\example\tutorial\hello.py", line 6, in <module>

import hello_ext

ImportError: No module named hello_ext

Hrmm. Okay. So I move the hello.py file into the same directory as the hello_ext.pyd file. Run it again using IDLE, still get the exception

Traceback (most recent call last):

File "C:\Users\Charles\Desktop\boost_1_38_0\boost_1_38_0\libs\python\example\tutorial\bin\msvc-9.0\debug\threading-multi\hello.py", line 6, in <module>

import hello_ext

ImportError: No module named hello_ext

So I guess I need some help here. I want to be able to create an extension module and then import it in a script. This is what I was led to believe I could do, but I do not see how. I have tried to provide all information that I think is necessary. Any help is greatly appreciated.

__________ Information from ESET NOD32 Antivirus, version of virus signature database 4593 (20091110) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: getting shape of numpy array

2009-11-10T05:35:13Z

Hello Nathan.

Find my answers below.

2009/11/10 Nathan Huesken <boost-python@...>:

> Hi,
>
> I am trying to interaface a numpy array using boost::python::numeric::array.
>
> void function(numeric::array& a)
> {
> tuple shape=extract<tuple>(a.getshape());
> double x;
> int i,j;
> for(i=0;i<shape[0];++i)
> for(j=0;j<shape[1];++j)
> x=extract<double>(a[make_tuple(i,j)]);
> }
>
> Now I had to add
> numeric::array::set_module_and_type("numpy", "ndarray");
>
> to the module, hope that was correct.
>
> When I call this function in python, I get:
> AttributeError: 'numpy.ndarray' object has no attribute 'getshape'

The error comes out because, actually, numpy.ndarray has no getshape method.

>
> What is the correct way to do it?
>

Within your function, declare

const tuple &shape = extract<tuple>(in.attr("shape"));

I suggest you to have a look also at my open source library (mds-utils):

http://code.google.com/p/mds-utils/

I've written some from/to Python converters that are able to convert a
Python sequence directly into a Boost uBLAS vector or matrix.
If you download the software, you can build the doxygen documentation
(follow the instructions) and you will find an example for each C++
class or function.

In other words, using my library, you could write your function as:

void function(const boost::numeric::ublas::matrix<double>& m) {
...
}

and export it to Python and pass, from Python, a Numpy 2D array.

Have a look at mds-utils and at its usage examples.

Bye,
Michele

> Thanks!
> Nathan
> --
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig@...
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

--
Michele De Stefano
http://www.linkedin.com/in/micdestefano
http://code.google.com/p/mds-utils
http://xoomer.virgilio.it/michele_de_stefano
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

getting shape of numpy array

2009-11-10T04:45:34Z

Hi,

I am trying to interaface a numpy array using boost::python::numeric::array.

void function(numeric::array& a)
{
tuple shape=extract<tuple>(a.getshape());
double x;
int i,j;
for(i=0;i<shape[0];++i)
for(j=0;j<shape[1];++j)
x=extract<double>(a[make_tuple(i,j)]);
}

Now I had to add
numeric::array::set_module_and_type("numpy", "ndarray");

to the module, hope that was correct.

When I call this function in python, I get:
AttributeError: 'numpy.ndarray' object has no attribute 'getshape'

What is the correct way to do it?

Thanks!
Nathan
--
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: unfriendly python error message (no line number)

2009-11-08T16:01:49Z

> No, it is not about template-related error messages :-)...
>
> I am wrapping a class:
> Obviously, the error is that I forgot '()' in the second get(). I
> am
> getting this error message:
>
> 0
> <bound method X.get of <Test.X object at 0x2b0db13d4cb0>>
>
> Is there a way to give user a more friendly error message in such
> case,
> with at least a line number where it happened?

This has nothing to do with boost.python. It's how python works. For example:

>>> class X(object):
def __init__(self):
self._x = 0
def get(self):
return self._x

>>> x = X()
>>> print x.get()
0
>>> print x.get
<bound method X.get of <__main__.X object at 0x02084FB0>>

And accessing 'x.get' is not in itself an error. You may want to pass that bound method to a function expecting a callable, for instance.

Alex
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

unfriendly python error message (no line number)

2009-11-08T15:20:03Z

No, it is not about template-related error messages :-)...

I am wrapping a class:
#include <boost/python.hpp>

struct X {
X() : _x(0) {}
int get() const { return _x; }
int _x;
};

BOOST_PYTHON_MODULE(Test) {
class_<X>("X")
.def("get", &X::get)
;
}

and testing it with python:
x = X();
print x.get()
print x.get

Obviously, the error is that I forgot '()' in the second get(). I am
getting this error message:

0
<bound method X.get of <Test.X object at 0x2b0db13d4cb0>>

Is there a way to give user a more friendly error message in such case,
with at least a line number where it happened?

Note that if mistype the attribute name, I am getting something I much
nicer:

Traceback (most recent call last):
File "test.py", line 5, in <module>
print x.foo
AttributeError: 'X' object has no attribute 'foo'

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: connecting library

2009-11-07T17:35:06Z

blackmet blackmet wrote:
> Hi, everyone.
>
> Prompt me please, how can I connect extern library to bjam.
> For example:
> connect libd3d9.a from mingw package.
>

The wrong list I think, asking on you are.

-t

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Iterators for heterogeneous container

2009-11-07T14:25:41Z

troy d. straszheim wrote:

> Thomas Daniel wrote:
>>
>> BOOST_PYTHON_MODULE(vegetables)
>> {
>> class_<Garden>("Garden")
>> .def("get_potatoes", &Garden::get_potatoes)
>> .def("get_tomatoes", &Garden::get_tomatoes)
>> ;
>> class_<TomatoIter>("TomatoIter")
>> .def("__iter__", &TomatoIter::get_next)
>> ;
>> }
>>
>> That at least compiles - unlike all my previous attempts that
>> generate three pages of template errors - but python complain:
>>
>> TypeError: iter() returned non-iterator of type 'Tomato'
>>
>> so now I am trying to figure out how to tell boost that get_tomatoes
>> returns an iterator ...
>>
>
> Please don't top-post, guys
>
> As you know, python iterators have a function next() that returns the
> next object in the iteree, or throw StopIteration when they're at the
> end. As you also know, python expects member functions __iter__() to
> return an iterator. Is the thing returned by TomatoIter::get_next an
> iterator, ie does it implement the iterator interface?
>
> -t
>
>

Thank you very much for the email, you did put me on the right track and
I finally figured out how to do it, with help from here also:
http://wiki.python.org/moin/boost.python/iterator

The solution involves creating a wrapper class around get_next(), which
throws StopIteration when done and a "pass_through" function to bind to
__iter__:

inline TomatoIter pass_through(const TomatoIter& iter) { return iter; }

Tomato next(TomatoIter& iter) {
Tomato tomato = iter.get_next();
if (!tomato) {
PyErr_SetString(PyExc_StopIteration, "No more data.");
throw_error_already_set();
}
return tomato;
}

Now, I can do this:
BOOST_PYTHON_MODULE(Garden)
{
......
class_<TomatoIter>("TomatoIter", no_init)
.def("next", next)
.def("__iter__", pass_through)
;

and it works.

Thomas

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

connecting library

2009-11-07T12:38:51Z

Hi, everyone.

Prompt me please, how can I connect extern library to bjam.

For example:

connect libd3d9.a from mingw package.

Thankful in advance.

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

Re: Iterators for heterogeneous container

2009-11-07T11:45:22Z

Thomas Daniel wrote:

>
> BOOST_PYTHON_MODULE(vegetables)
> {
> class_<Garden>("Garden")
> .def("get_potatoes", &Garden::get_potatoes)
> .def("get_tomatoes", &Garden::get_tomatoes)
> ;
> class_<TomatoIter>("TomatoIter")
> .def("__iter__", &TomatoIter::get_next)
> ;
> }
>
> That at least compiles - unlike all my previous attempts that generate
> three pages of template errors - but python complain:
>
> TypeError: iter() returned non-iterator of type 'Tomato'
>
> so now I am trying to figure out how to tell boost that get_tomatoes
> returns an iterator ...
>

Please don't top-post, guys

As you know, python iterators have a function next() that returns the
next object in the iteree, or throw StopIteration when they're at the
end. As you also know, python expects member functions __iter__() to
return an iterator. Is the thing returned by TomatoIter::get_next an
iterator, ie does it implement the iterator interface?

-t

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig