Segmentation fault for 64 Bit platforms in to_wkb function
|
View:
New views
2 Messages
—
Rating Filter:
Alert me
|
 |
Segmentation fault for 64 Bit platforms in to_wkb function
by Paul Balomiri
::
Rate this Message:
Hi Sean,
finally, the solution to the problem is attached to this mail. basically in ctypes SDLL/PyDLL loaders, the contained function arguments and return value are not reflected. By default when calling a function without supplying any argtypes list and restype ctype it is assumed that the respective argument can be casted to int. However, in case of c_void_p this only works on 32 bit platforms, where sizeof(c_size_t) = sizeof(c_int). All pointers, such as c_void_p, are of the same type as c_size_t. This produces to SEGFAULTs on 64 bit platforms where-ever specifying argtypes and restype has been omitted. The bug only shows up when the pointer type > 2^32, otherwise python converts the correct value to c_void_p. On linux, and in my case at least it takes about 1500 mallocs to hit one in the high memory area where pointers are > 2^32. This might well be the reason why the problem has not shown up until now. However, I had no time to thoroughly search for all imports used, but not declared. I would be grateful if you could include the patch into rev 1.0.13. I would have patched the trunk, but the file structure there seems pretty different from the 1.0.13 version, and I did't' find the c-declarations python file. Thank you for maintaining such a wonderful lib, Cheers Paul 2009/10/5 Paul Balomiri <paulbalomiri@...>: > Hi, > > I tried to find out where exactly BaseGeometry.__geom__ becomes int type. > > and here is what I found out: > >>>> p=Point(1.0,2.3) > pointaddr=10373776 >>>> p > <shapely.geometry.point.Point object at 0x7f7f7e8ebed0> >>>> p.__geom__.__class__ > <type 'int'> > > > OK, now going into the constructor of Point I could pinpoint > Line #53 in point.py > > Creating a sequence there yielda an int instead of a c_void_p: > >>>> cs = lgeos.GEOSCoordSeq_create(1, 2) >>>> cs.__class__ > <type 'int'> > > This seems strance as in > http://trac.gispython.org/lab/browser/Shapely/tags/rel-1.0.13/shapely/ctypes_declarations.py#L26 > tou correctly defined > > lgeos.GEOSCoordSeq_create.restype = ctypes.c_void_p > > So the only thing coming to my mind is that the c_type_declarations > might somehow not be included ? > > Do you have any hints where to go from here ? > > Cheers Paul > > 2009/10/5 Paul Balomiri <paulbalomiri@...>: >> Hi, >> Now having gone deeper into the matter i found out a bug, >> and can further get into details about when a segfault still occurs. >> The relevant fact i did not mention last time is that i'm running >> shapely on a 64 bit CPU. >> >> 1) http://trac.gispython.org/lab/browser/Shapely/tags/rel-1.0.13/shapely/geometry/base.py >> ay line 352 you use >> >> size = c_int() >> >> and later on pass it as size size arg to GEOS' GEOSGeomToWKB_buf. >> This is wrong on 64 bit processors. >> Please use size = c_size_t() (8 byte long on 64 bit) >> >> 2) I could find what may be a corelated symptom, and easily to check: >> the __geom__ type is int. This causes segfaults when the actual ponter type has >> a value >2^32-1. I could not follow the code to the point where __geom__ is set, >> but am searching for it. Perhaps you can give me a hint ? >> >> >> 2009/10/5 Sean Gillies <sean.gillies@...>: >>> On Oct 3, 2009, at 7:52 PM, Paul Balomiri wrote: >>> >>>> Hi, >>>> >>>> I encountered a similar problem to >>>> http://lists.osgeo.org/pipermail/geos-devel/2009-July/004280.html , >>>> which is >>>> at the exact same execution point. >>>> >>>> I am using shapely 1.0.13. As the lib is linked with PyDLL into >>>> python, the GIL (Global Interpreter Log) is used to schedule library >>>> entrance ( = no python code (in any thread) runs concurrently), >>>> so my first guess was, that an init error or an error in handling the >>>> global Context might cause the segfault. >>>> >>>> The backtrace I got from gdb is >>>> >>>> >>>> [Switching to Thread 0x7fffe7f076f0 (LWP 6598)] >>>> geos::io::WKBWriter::write (this=0x7fffeff120c0, g=@0xd8005270, >>>> os=@0x7fffeff11f50) at WKBWriter.cpp:77 >>>> 77 switch (g.getGeometryTypeId()) { >>>> Current language: auto; currently c++ >>>> (gdb) where >>>> #0 geos::io::WKBWriter::write (this=0x7fffeff120c0, g=@0xd8005270, >>>> os=@0x7fffeff11f50) at WKBWriter.cpp:77 >>>> #1 0x00007fffe6883549 in GEOSGeomToWKB_buf_r (extHandle=0xa5beb0, >>>> g=0xd8005270, size=0x14407f0) at geos_ts_c.cpp:948 >>>> #2 0x00007fffe6aa53b8 in ffi_call_unix64 () at >>>> /build/buildd/python2.6-2.6.2/Modules/_ctypes/libffi/src/x86/ >>>> unix64.S:75 >>>> Python stacktrace: >>>> >>>> [Python Stack] >>>> >>>> #46 0x0000000000417ae9 in _start () at ../sysdeps/x86_64/elf/start.S: >>>> 113 >>>> >>>> A full stacktrace is included as an attachment. I have augmented the >>>> stacktrace with the python >>>> stackframe where the stack frame is in python code. >>>> >>>> I was wondering why GEOSGeomToWKB_buf is called, but >>>> GEOSGeomToWKB_buf_r appears on the stacktrace. >>>> >>>> I do use threads in the application, but only the main thread has >>>> calling points to shapely. >>>> Also the application uses SQLAlchemy with the user defined Geometry >>>> type from shapely. The gluecode is part of mapfish. >>>> >>>> Perhaps you have some ideas about why the segfault shows up. I'm not >>>> familiar with the way >>>> shapely handles GEOS c pointers. In gdb the geos geometry object >>>> g=@0xd8005270 points to an adress >>>> >>>> trying to access the geometry object reveals >>>> >>>> (gdb) p g >>>> $1 = (const geos::geom::Geometry &) @0xd8005270: <error reading >>>> variable> >>>> >>>> >>>> regards, >>>> Paul >>> >>> Hi Paul, >>> >>> I'm sorry you're experiencing trouble with Shapely. If you can write a >>> Python program that doesn't depend on MapFish or SQLAlchemy or PostGIS >>> (I don't have that stack running right now) that reliably crashes, >>> I'll get right on it. Maybe it's just a matter of writing out WKB from >>> multiple threads at once. >>> >>> If you're using GEOS 3.1, the non-reentrant API is just a thin layer >>> over the reentrant one, so EOSGeomToWKB_buf_r will always be called. >>> >>> Cheers, >>> >>> -- >>> Sean >>> >>> _______________________________________________ >>> Community mailing list >>> Community@... >>> http://lists.gispython.org/mailman/listinfo/community >>> >> >> >> >> -- >> paulbalomiri@... >> > > > > -- > paulbalomiri@... > -- paulbalomiri@... _______________________________________________ Community mailing list Community@... http://lists.gispython.org/mailman/listinfo/community |
|
|
Re: Segmentation fault for 64 Bit platforms in to_wkb function
by Sean Gillies-2
::
Rate this Message:
|
Shapely.patch.rev1458 (3K) | Free embeddable forum powered by Nabble | Forum Help |
