Embed perl and multi-threading (Re: segmentation fault in event-driven library wrapped with swig)

[Sri-26]

Hi all,

In digging further into this, I feel that the pthread_getspecific() functions are not finding the perl interpreter instance corresponding to the thread in my library.  I guess its because I havent really created one yet? To test if this was the cause, I modified my SWIG_Wrap_bsa_disc_callback method to create a new perl-interpreter when my_perl is NULL and the crash is now occuring at the call_sv.  I realize its not the right way to do it, but not sure exactly the steps required.  I think the solution likely involves my library thread creating a interpreter instance and saving relevant data (what is relevant?) via pthread_setspecific method/SET_CONTEXT().

Would be great if anyone can shed some light or confirm if I'm on the right track.

Thx

void SWIG_Wrap_bsa_disc_callback(int event, tMy_MSG *p_data)
{
      int new_perl = 0;
      printf("entering perl callback handler...\n");
      dTHX;
      printf("dTHX done...\n");

      if ( my_perl != NULL )
         printf ("my_perl == %ul\n", my_perl);
      else
      {
         printf ("my_perl was NULL\n");
         my_perl = perl_alloc();
         PERL_SET_CONTEXT(my_perl);
         new_perl = 1;
      }
/*
      printf("Destruct level : %d...\n", PL_perl_destruct_level);
*/
      dSP;
      printf("DSP done...\n");
      SV * sv = bsa_perl_disc_callback;
      if (sv == (SV*)NULL)
         croak("Internal error...\n");

      printf("Preparing to call perl callback...\n");
      PUSHMARK(SP);
      XPUSHs(sv_2mortal(newSViv(event)));
      printf("newSViv of %d done...\n", event);
      XPUSHs(SWIG_NewPointerObj(SWIG_as_voidptr(p_data), SWIGTYPE_p_tMy_MSG, SWIG_SHADOW));
      /* XPUSHs(sv_2mortal(newSVsv((SV *)p_data))); */
      printf("newSVsv...\n");
      PUTBACK;

      printf("calling the Perl sub...\n");
      /* Call the Perl sub */
      call_sv(sv, G_DISCARD);
      printf("Perl sub completed successfully...\n");

      if ( new_perl )
        perl_free(my_perl);
}


--- On Fri, 10/30/09, Sri <w2learn@...> wrote:

> From: Sri <w2learn@...>
> Subject: segmentation fault in event-driven library wrapped with swig
> To: swig-user@...
> Date: Friday, October 30, 2009, 8:35 PM
> Hello SWIG, perl experts:
>
> With good help from swig-users posts & Stephen Chan in
> particular, I've figured out how to wrap a C event-driven
> library with SWIG.  SWIG is really nice tool!  I'm able to
> call APIs and register perl-sub as callback into the C
> library. If I invoke the perl callback as part of an API
> invoked from Perl script, the callback is called (in perls
> main thread).  However, when the same callback handler
> invokes the callback via a different thread (based on async
> event), it crashes at the "dSP;" line.
>
> I've tried adding "dTX" before "dSP;" and also tried
> enabling more macros to match how my perl is built.  So far
> no luck.  The line that crashes is below (pre-processor
> output).  "dTHX done" is printed, but crashes before
> "DSP done" is printed.
>
>       printf("entering perl callback
> handler...\n");
>       register PerlInterpreter *my_perl
> __attribute__((unused)) = ((PerlInterpreter
> *)pthread_getspecific((*Perl_Gthr_key_ptr(((void *)0)))));
>       printf("dTHX done...\n");
>       register SV **sp =
> (*Perl_Tstack_sp_ptr(((PerlInterpreter
> *)pthread_getspecific((*Perl_Gthr_key_ptr(((void
> *)0)))))));  <-- CRASHES!!
>       printf("DSP done...\n");
>
>
> The perl-callback handler code
> ------------------------------
>
> void SWIG_Wrap_bsa_disc_callback(int event, tMy_MSG
> *p_data)
> {
>       printf("entering perl callback
> handler...\n");
>       dTHX;
>       printf("dTHX done...\n");
>       dSP;
>       printf("DSP done...\n");
>       SV * sv = bsa_perl_disc_callback;
>       if (sv == (SV*)NULL)
>          croak("Internal
> error...\n");
>
>       printf("Preparing to call perl
> callback...\n");
>       PUSHMARK(SP);
>       XPUSHs(sv_2mortal(newSViv(event)));
>       printf("newSViv of %d done...\n",
> event);
>      
> XPUSHs(SWIG_NewPointerObj(SWIG_as_voidptr(p_data),
> SWIGTYPE_p_tMy_MSG, SWIG_SHADOW));
>       /* XPUSHs(sv_2mortal(newSVsv((SV
> *)p_data))); */
>       printf("newSVsv...\n");
>       PUTBACK;
>
>       printf("calling the Perl sub...\n");
>       /* Call the Perl sub */
>       call_sv(sv, G_DISCARD);
>       printf("Perl sub completed
> successfully...\n");
> }
>
> Options when running SWIG:
> --------------------------
>
> swig -includeall -I. -I <lots of others proj specific
> dirs> -perl5 -outdir <outdir> -module
> <module> my_api.i
>
> Options when compliling _wrap.c:
> --------------------------------
>
> gcc -E -Dbool=char -fpic -D_REENTRANT -D_GNU_SOURCE
> -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe
> -isystem /usr/local/include -D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64 -DSWIG -g -DMULTIPLICITY
> -DPERL_IMPLICIT_CONTEXT -DTHREADS_HAVE_PIDS
> -DUSE_LARGE_FILES -DUSE_PERLIO -DUSE_REENTRANT_API
> -DBSA_CLIENT -I/usr/lib/perl/5.8/CORE/
>
> Output from perl -V
> -------------------
>
> Summary of my perl5 (revision 5 version 8 subversion 8)
> configuration:
>   Platform:
>     osname=linux, osvers=2.6.24-19-server,
> archname=i486-linux-gnu-thread-multi
>     uname='linux palmer 2.6.24-19-server #1 smp
> sat jul 12 00:40:01 utc 2008 i686 gnulinux '
>     config_args='-Dusethreads -Duselargefiles
> -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC
> -Darchname=i486-linux-gnu -Dprefix=/usr
> -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8
> -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
> -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
> -Dsitelib=/usr/local/share/perl/5.8.8
> -Dsitearch=/usr/local/lib/perl/5.8.8
> -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
> -Dsiteman1dir=/usr/local/man/man1
> -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1
> -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs
> -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Duseshrplib
> -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
>     hint=recommended, useposix=true,
> d_sigaction=define
>     usethreads=define use5005threads=undef
> useithreads=define usemultiplicity=define
>     useperlio=define d_sfio=undef
> uselargefiles=define usesocks=undef
>     use64bitint=undef use64bitall=undef
> uselongdouble=undef
>     usemymalloc=n, bincompat5005=undef
>   Compiler:
>     cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE
> -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe
> -I/usr/local/include -D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64',
>     optimize='-O2',
>     cppflags='-D_REENTRANT -D_GNU_SOURCE
> -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe
> -I/usr/local/include'
>     ccversion='', gccversion='4.2.3 (Ubuntu
> 4.2.3-2ubuntu7)', gccosandvers=''
>     intsize=4, longsize=4, ptrsize=4,
> doublesize=8, byteorder=1234
>     d_longlong=define, longlongsize=8,
> d_longdbl=define, longdblsize=12
>     ivtype='long', ivsize=4, nvtype='double',
> nvsize=8, Off_t='off_t', lseeksize=8
>     alignbytes=4, prototype=define
>   Linker and Libraries:
>     ld='cc', ldflags =' -L/usr/local/lib'
>     libpth=/usr/local/lib /lib /usr/lib
>     libs=-lgdbm -lgdbm_compat -ldb -ldl -lm
> -lpthread -lc -lcrypt
>     perllibs=-ldl -lm -lpthread -lc -lcrypt
>     libc=/lib/libc-2.7.so, so=so,
> useshrplib=true, libperl=libperl.so.5.8.8
>     gnulibc_version='2.7'
>   Dynamic Linking:
>     dlsrc=dl_dlopen.xs, dlext=so,
> d_dlsymun=undef, ccdlflags='-Wl,-E'
>     cccdlflags='-fPIC', lddlflags='-shared
> -L/usr/local/lib'
>
>
> Characteristics of this binary (from libperl):
>   Compile-time options: MULTIPLICITY
> PERL_IMPLICIT_CONTEXT
>                
>         PERL_MALLOC_WRAP
> THREADS_HAVE_PIDS USE_ITHREADS
>                
>         USE_LARGE_FILES USE_PERLIO
> USE_REENTRANT_API
>   Built under linux
>   Compiled at Jan 14 2009 22:34:36
>   @INC:
>     /etc/perl
>     /usr/local/lib/perl/5.8.8
>     /usr/local/share/perl/5.8.8
>     /usr/lib/perl5
>     /usr/share/perl5
>     /usr/lib/perl/5.8
>     /usr/share/perl/5.8
>     /usr/local/lib/site_perl
>     .
>
> output from gcc -v
> ------------------
>
>  gcc -v
> Using built-in specs.
> Target: i486-linux-gnu
> Configured with: ../src/configure -v
> --enable-languages=c,c++,fortran,objc,obj-c++,treelang
> --prefix=/usr --enable-shared --with-system-zlib
> --libexecdir=/usr/lib --without-included-gettext
> --enable-threads=posix --enable-nls
> --with-gxx-include-dir=/usr/include/c++/4.2
> --program-suffix=-4.2 --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr
> --enable-targets=all --enable-checking=release
> --build=i486-linux-gnu --host=i486-linux-gnu
> --target=i486-linux-gnu
> Thread model: posix
> gcc version 4.2.4 (Ubuntu 4.2.4-1ubuntu4)
>
>
>
>
>

     

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user

[Steve Chan-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sri,
    I suspect you are asking the wrong forum about this.

    This sounds like a perl specific issue with embedded perl and
multithreading. You could check through the groups at
http://www.nntp.perl.org/group/ and see if anyone there has experience
with multithreaded embedded perl.

    Steve

Sri wrote:

> Hi all,
>
> In digging further into this, I feel that the pthread_getspecific()
> functions are not finding the perl interpreter instance
> corresponding to the thread in my library.  I guess its because I
> havent really created one yet? To test if this was the cause, I
> modified my SWIG_Wrap_bsa_disc_callback method to create a new
> perl-interpreter when my_perl is NULL and the crash is now occuring
> at the call_sv.  I realize its not the right way to do it, but not
> sure exactly the steps required.  I think the solution likely
> involves my library thread creating a interpreter instance and
> saving relevant data (what is relevant?) via pthread_setspecific
> method/SET_CONTEXT().
>
> Would be great if anyone can shed some light or confirm if I'm on
> the right track.
>
> Thx
>
> void SWIG_Wrap_bsa_disc_callback(int event, tMy_MSG *p_data) { int
> new_perl = 0; printf("entering perl callback handler...\n"); dTHX;
> printf("dTHX done...\n");
>
> if ( my_perl != NULL ) printf ("my_perl == %ul\n", my_perl); else {
>  printf ("my_perl was NULL\n"); my_perl = perl_alloc();
> PERL_SET_CONTEXT(my_perl); new_perl = 1; } /* printf("Destruct
> level : %d...\n", PL_perl_destruct_level); */ dSP; printf("DSP
> done...\n"); SV * sv = bsa_perl_disc_callback; if (sv == (SV*)NULL)
>  croak("Internal error...\n");
>
> printf("Preparing to call perl callback...\n"); PUSHMARK(SP);
> XPUSHs(sv_2mortal(newSViv(event))); printf("newSViv of %d
> done...\n", event);
> XPUSHs(SWIG_NewPointerObj(SWIG_as_voidptr(p_data),
> SWIGTYPE_p_tMy_MSG, SWIG_SHADOW)); /* XPUSHs(sv_2mortal(newSVsv((SV
> *)p_data))); */ printf("newSVsv...\n"); PUTBACK;
>
> printf("calling the Perl sub...\n"); /* Call the Perl sub */
> call_sv(sv, G_DISCARD); printf("Perl sub completed
> successfully...\n");
>
> if ( new_perl ) perl_free(my_perl); }
>
>
> --- On Fri, 10/30/09, Sri <w2learn@...> wrote:
>
>> From: Sri <w2learn@...> Subject: segmentation fault in
>> event-driven library wrapped with swig To:
>> swig-user@... Date: Friday, October 30, 2009,
>> 8:35 PM Hello SWIG, perl experts:
>>
>> With good help from swig-users posts & Stephen Chan in
>> particular, I've figured out how to wrap a C event-driven library
>> with SWIG.  SWIG is really nice tool!  I'm able to call APIs and
>> register perl-sub as callback into the C library. If I invoke the
>> perl callback as part of an API invoked from Perl script, the
>> callback is called (in perls main thread).  However, when the
>> same callback handler invokes the callback via a different thread
>> (based on async event), it crashes at the "dSP;" line.
>>
>> I've tried adding "dTX" before "dSP;" and also tried enabling
>> more macros to match how my perl is built.  So far no luck.  The
>> line that crashes is below (pre-processor output).  "dTHX done"
>> is printed, but crashes before "DSP done" is printed.
>>
>> printf("entering perl callback handler...\n"); register
>> PerlInterpreter *my_perl __attribute__((unused)) =
>> ((PerlInterpreter
>> *)pthread_getspecific((*Perl_Gthr_key_ptr(((void *)0)))));
>> printf("dTHX done...\n"); register SV **sp =
>> (*Perl_Tstack_sp_ptr(((PerlInterpreter
>> *)pthread_getspecific((*Perl_Gthr_key_ptr(((void *)0)))))));  <--
>> CRASHES!! printf("DSP done...\n");
>>
>>
>> The perl-callback handler code ------------------------------
>>
>> void SWIG_Wrap_bsa_disc_callback(int event, tMy_MSG *p_data) {
>> printf("entering perl callback handler...\n"); dTHX; printf("dTHX
>> done...\n"); dSP; printf("DSP done...\n"); SV * sv =
>> bsa_perl_disc_callback; if (sv == (SV*)NULL) croak("Internal
>> error...\n");
>>
>> printf("Preparing to call perl callback...\n"); PUSHMARK(SP);
>> XPUSHs(sv_2mortal(newSViv(event))); printf("newSViv of %d
>> done...\n", event);
>>
>> XPUSHs(SWIG_NewPointerObj(SWIG_as_voidptr(p_data),
>> SWIGTYPE_p_tMy_MSG, SWIG_SHADOW)); /*
>> XPUSHs(sv_2mortal(newSVsv((SV *)p_data))); */
>> printf("newSVsv...\n"); PUTBACK;
>>
>> printf("calling the Perl sub...\n"); /* Call the Perl sub */
>> call_sv(sv, G_DISCARD); printf("Perl sub completed
>> successfully...\n"); }
>>
>> Options when running SWIG: --------------------------
>>
>> swig -includeall -I. -I <lots of others proj specific dirs>
>> -perl5 -outdir <outdir> -module <module> my_api.i
>>
>> Options when compliling _wrap.c: --------------------------------
>>
>>
>> gcc -E -Dbool=char -fpic -D_REENTRANT -D_GNU_SOURCE
>> -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -isystem
>> /usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
>> -DSWIG -g -DMULTIPLICITY -DPERL_IMPLICIT_CONTEXT
>> -DTHREADS_HAVE_PIDS -DUSE_LARGE_FILES -DUSE_PERLIO
>> -DUSE_REENTRANT_API -DBSA_CLIENT -I/usr/lib/perl/5.8/CORE/
>>
>> Output from perl -V -------------------
>>
>> Summary of my perl5 (revision 5 version 8 subversion 8)
>> configuration: Platform: osname=linux, osvers=2.6.24-19-server,
>> archname=i486-linux-gnu-thread-multi uname='linux palmer
>> 2.6.24-19-server #1 smp sat jul 12 00:40:01 utc 2008 i686
>> gnulinux ' config_args='-Dusethreads -Duselargefiles
>> -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu
>> -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8
>> -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr
>> -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5
>> -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.8
>> -Dsitearch=/usr/local/lib/perl/5.8.8
>> -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
>> -Dsiteman1dir=/usr/local/man/man1
>> -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
>> -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm
>> -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8
>> -Dd_dosuid -des' hint=recommended, useposix=true,
>> d_sigaction=define usethreads=define use5005threads=undef
>> useithreads=define usemultiplicity=define useperlio=define
>> d_sfio=undef uselargefiles=define usesocks=undef
>> use64bitint=undef use64bitall=undef uselongdouble=undef
>> usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags
>> ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
>> -fno-strict-aliasing -pipe -I/usr/local/include
>> -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O2',
>> cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
>> -fno-strict-aliasing -pipe -I/usr/local/include' ccversion='',
>> gccversion='4.2.3 (Ubuntu 4.2.3-2ubuntu7)', gccosandvers=''
>> intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
>> d_longlong=define, longlongsize=8, d_longdbl=define,
>> longdblsize=12 ivtype='long', ivsize=4, nvtype='double',
>> nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=4,
>> prototype=define Linker and Libraries: ld='cc', ldflags ='
>> -L/usr/local/lib' libpth=/usr/local/lib /lib /usr/lib libs=-lgdbm
>> -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt perllibs=-ldl
>> -lm -lpthread -lc -lcrypt libc=/lib/libc-2.7.so, so=so,
>> useshrplib=true, libperl=libperl.so.5.8.8 gnulibc_version='2.7'
>> Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef,
>> ccdlflags='-Wl,-E' cccdlflags='-fPIC', lddlflags='-shared
>> -L/usr/local/lib'
>>
>>
>> Characteristics of this binary (from libperl): Compile-time
>> options: MULTIPLICITY PERL_IMPLICIT_CONTEXT
>>
>> PERL_MALLOC_WRAP THREADS_HAVE_PIDS USE_ITHREADS
>>
>> USE_LARGE_FILES USE_PERLIO USE_REENTRANT_API Built under linux
>> Compiled at Jan 14 2009 22:34:36 @INC: /etc/perl
>> /usr/local/lib/perl/5.8.8 /usr/local/share/perl/5.8.8
>> /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.8
>> /usr/share/perl/5.8 /usr/local/lib/site_perl .
>>
>> output from gcc -v ------------------
>>
>> gcc -v Using built-in specs. Target: i486-linux-gnu Configured
>> with: ../src/configure -v
>> --enable-languages=c,c++,fortran,objc,obj-c++,treelang
>> --prefix=/usr --enable-shared --with-system-zlib
>> --libexecdir=/usr/lib --without-included-gettext
>> --enable-threads=posix --enable-nls
>> --with-gxx-include-dir=/usr/include/c++/4.2 --program-suffix=-4.2
>> --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc
>> --enable-mpfr --enable-targets=all --enable-checking=release
>> --build=i486-linux-gnu --host=i486-linux-gnu
>> --target=i486-linux-gnu Thread model: posix gcc version 4.2.4
>> (Ubuntu 4.2.4-1ubuntu4)
>>
>>
>>
>>
>>
>
>
>
>
> ------------------------------------------------------------------------------
>  Come build with us! The BlackBerry(R) Developer Conference in SF,
> CA is the only developer event you need to attend this year.
> Jumpstart your developing skills, take BlackBerry mobile
> applications to market and stay ahead of the curve. Join us from
> November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________ Swig-user mailing
> list Swig-user@...
> https://lists.sourceforge.net/lists/listinfo/swig-user

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.12 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkrvFT8ACgkQcVd2YI1BWAi1vQCcDzOZ2kydduJm2Cn3WTeMzVl0
fScAniNdTUCg/LLBhI2EXhij9AmUtdgS
=0XmJ
-----END PGP SIGNATURE-----


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Swig-user mailing list
Swig-user@...
https://lists.sourceforge.net/lists/listinfo/swig-user