Deadlock due to `pthread_mutex_lock'.

View: New views
6 Messages — Rating Filter:   Alert me  

Deadlock due to `pthread_mutex_lock'.

by Ludovic Courtès-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello,

When running `gnunetd' (0.8.0b), I get the following error:

  # gnunetd -d --config gnunetd.conf -L DEBUG
  Aug 31 18:53:23 INFO: Loading transports `udp tcp http nat'
  Aug 31 18:53:23 INFO: Loaded transport `udp'
  Aug 31 18:53:23 INFO: Loaded transport `tcp'
  Aug 31 18:53:23 INFO: Loaded transport `http'
  Aug 31 18:53:23 INFO: Loaded transport `nat'
  Aug 31 18:53:23 FATAL: Deadlock due to `pthread_mutex_lock'.
  Aug 31 18:53:23 FATAL: Internal error: assertion failed at statuscalls.c:402.
  Aborted (core dumped)

Here's a back-trace of each of the 4 threads:

  (gdb) bt
  #0  0xffffe424 in __kernel_vsyscall ()
  #1  0xb7c1ec37 in raise () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6
  #2  0xb7c20671 in abort () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6
  #3  0xb7f41fa5 in GNUNET_mutex_lock_at_file_line_ () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #4  0xb7f3f919 in resetStatusCalls () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #5  0xb7f23a5a in GNUNET_GC_set_configuration_value_string () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #6  0xb7f241b2 in GNUNET_GC_set_configuration_value_number () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #7  0xb7f2435d in GNUNET_GC_get_configuration_value_number () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #8  0xb7f3fbd0 in resetStatusCalls () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #9  0xb7f23a5a in GNUNET_GC_set_configuration_value_string () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #10 0xb7f23dfc in GNUNET_GC_get_configuration_value_string () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #11 0xb7f3f8db in resetStatusCalls () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #12 0xb7f23a5a in GNUNET_GC_set_configuration_value_string () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #13 0xb7f241b2 in GNUNET_GC_set_configuration_value_number () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #14 0xb7f2435d in GNUNET_GC_get_configuration_value_number () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #15 0xb7f5099e in connectionConfigChangeCallback () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetcore.so.0
  #16 0xb7f50d28 in GNUNET_CORE_connection_init () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetcore.so.0
  #17 0x0804967f in gnunet_main ()
  #18 0x08049ae0 in main ()
  (gdb) info threads
    4 process 3476  0xffffe424 in __kernel_vsyscall ()
    3 process 3477  0xffffe424 in __kernel_vsyscall ()
    2 process 3478  0xffffe424 in __kernel_vsyscall ()
  * 1 process 3475  0xffffe424 in __kernel_vsyscall ()
  (gdb) thread 2
  [Switching to thread 2 (process 3478)]#0  0xffffe424 in __kernel_vsyscall ()
  (gdb) bt
  #0  0xffffe424 in __kernel_vsyscall ()
  #1  0xb7d3ebb8 in recv () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
  #2  0xb71a650f in gaim_upnp_discover () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/GNUnet/libgnunetmodule_upnp.so
  #3  0xb71a4dd4 in discover_thread () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/GNUnet/libgnunetmodule_upnp.so
  #4  0xb7d3717b in start_thread () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
  #5  0xb7cbbcfe in clone () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6
  (gdb) thread 3
  [Switching to thread 3 (process 3477)]#0  0xffffe424 in __kernel_vsyscall ()
  (gdb) bt
  #0  0xffffe424 in __kernel_vsyscall ()
  #1  0xb7d3f096 in nanosleep () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
  #2  0xb7f425a1 in GNUNET_thread_sleep () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #3  0xb7f27943 in cron_main_method () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #4  0xb7d3717b in start_thread () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
  #5  0xb7cbbcfe in clone () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6
  (gdb) thread 4
  [Switching to thread 4 (process 3476)]#0  0xffffe424 in __kernel_vsyscall ()
  (gdb) bt
  #0  0xffffe424 in __kernel_vsyscall ()
  #1  0xb7cb4fd1 in select () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6
  #2  0xb7f38cda in selectThread () from /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil.so.3
  #3  0xb7d3717b in start_thread () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
  #4  0xb7cbbcfe in clone () from /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6

The `gnunetd.conf' file reads this:

  [PATHS]
  GNUNETD_HOME = /var/lib/gnunet

  [GNUNETD]
  APPLICATIONS = advertising getoption fs stats traffic
  TRANSPORTS = udp tcp http nat

And `/var/lib/gnunet' is empty.

Any idea what's wrong?

Thanks in advance,
Ludo'.



_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: Deadlock due to `pthread_mutex_lock'.

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Yep, I have an idea. Nobody ever tried to run the code without specifying
bandwidth limits explicitly (which causes a non-recursive mutex to be
acquired twice by the same thread => deadlock).  I'm fixing this in SVN, as a
workaround, specify the UP/DOWN bandwidth limits for gnunetd.

Christian

On Sunday 31 August 2008 10:58:43 am Ludovic Courtès wrote:

> Hello,
>
> When running `gnunetd' (0.8.0b), I get the following error:
>
>   # gnunetd -d --config gnunetd.conf -L DEBUG
>   Aug 31 18:53:23 INFO: Loading transports `udp tcp http nat'
>   Aug 31 18:53:23 INFO: Loaded transport `udp'
>   Aug 31 18:53:23 INFO: Loaded transport `tcp'
>   Aug 31 18:53:23 INFO: Loaded transport `http'
>   Aug 31 18:53:23 INFO: Loaded transport `nat'
>   Aug 31 18:53:23 FATAL: Deadlock due to `pthread_mutex_lock'.
>   Aug 31 18:53:23 FATAL: Internal error: assertion failed at
> statuscalls.c:402. Aborted (core dumped)
>
> Here's a back-trace of each of the 4 threads:
>
>   (gdb) bt
>   #0  0xffffe424 in __kernel_vsyscall ()
>   #1  0xb7c1ec37 in raise () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6 #2
> 0xb7c20671 in abort () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6 #3
> 0xb7f41fa5 in GNUNET_mutex_lock_at_file_line_ () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #4  0xb7f3f919 in resetStatusCalls () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #5  0xb7f23a5a in GNUNET_GC_set_configuration_value_string () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #6  0xb7f241b2 in GNUNET_GC_set_configuration_value_number () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #7  0xb7f2435d in GNUNET_GC_get_configuration_value_number () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #8  0xb7f3fbd0 in resetStatusCalls () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #9  0xb7f23a5a in GNUNET_GC_set_configuration_value_string () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #10 0xb7f23dfc in GNUNET_GC_get_configuration_value_string () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #11 0xb7f3f8db in resetStatusCalls () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #12 0xb7f23a5a in GNUNET_GC_set_configuration_value_string () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #13 0xb7f241b2 in GNUNET_GC_set_configuration_value_number () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #14 0xb7f2435d in GNUNET_GC_get_configuration_value_number () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-resetStatusCalls ()
fro0.8.0b/lib/libgnunetutil

>.so.3 #15 0xb7f5099e in connectionConfigChangeCallback () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetcore
>.so.0 #16 0xb7f50d28 in GNUNET_CORE_connection_init () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetcore
>.so.0 #17 0x0804967f in gnunet_main ()
>   #18 0x08049ae0 in main ()
>   (gdb) info threads
>     4 process 3476  0xffffe424 in __kernel_vsyscall ()
>     3 process 3477  0xffffe424 in __kernel_vsyscall ()
>     2 process 3478  0xffffe424 in __kernel_vsyscall ()
>   * 1 process 3475  0xffffe424 in __kernel_vsyscall ()
>   (gdb) thread 2
>   [Switching to thread 2 (process 3478)]#0  0xffffe424 in __kernel_vsyscall
> () (gdb) bt
>   #0  0xffffe424 in __kernel_vsyscall ()
>   #1  0xb7d3ebb8 in recv () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
> #2  0xb71a650f in gaim_upnp_discover () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/GNUnet/libgnu
>netmodule_upnp.so #3  0xb71a4dd4 in discover_thread () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/GNUnet/libgnu
>netmodule_upnp.so #4  0xb7d3717b in start_thread () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
> #5  0xb7cbbcfe in clone () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6 (gdb)
> thread 3
>   [Switching to thread 3 (process 3477)]#0  0xffffe424 in __kernel_vsyscall
> () (gdb) bt
>   #0  0xffffe424 in __kernel_vsyscall ()
>   #1  0xb7d3f096 in nanosleep () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
> #2  0xb7f425a1 in GNUNET_thread_sleep () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #3  0xb7f27943 in cron_main_method () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #4  0xb7d3717b in start_thread () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
> #5  0xb7cbbcfe in clone () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6 (gdb)
> thread 4
>   [Switching to thread 4 (process 3476)]#0  0xffffe424 in __kernel_vsyscall
> () (gdb) bt
>   #0  0xffffe424 in __kernel_vsyscall ()
>   #1  0xb7cb4fd1 in select () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6 #2
> 0xb7f38cda in selectThread () from
> /nix/store/8gn0z686l3ajgb60zgngs2w1ip7nqvws-gnunet-0.8.0b/lib/libgnunetutil
>.so.3 #3  0xb7d3717b in start_thread () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libpthread.so.0
> #4  0xb7cbbcfe in clone () from
> /nix/store/b2gmgy8y6nidjj7n34jx4sanrjdaxrw3-glibc-2.7/lib/libc.so.6
>
> The `gnunetd.conf' file reads this:
>
>   [PATHS]
>   GNUNETD_HOME = /var/lib/gnunet
>
>   [GNUNETD]
>   APPLICATIONS = advertising getoption fs stats traffic
>   TRANSPORTS = udp tcp http nat
>
> And `/var/lib/gnunet' is empty.
>
> Any idea what's wrong?
>
> Thanks in advance,
> Ludo'.
>
>
>
> _______________________________________________
> GNUnet-developers mailing list
> GNUnet-developers@...
> http://lists.gnu.org/mailman/listinfo/gnunet-developers




_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: Deadlock due to `pthread_mutex_lock'.

by Ludovic Courtès-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Christian,

Christian Grothoff <christian@...> writes:

> Yep, I have an idea. Nobody ever tried to run the code without specifying
> bandwidth limits explicitly (which causes a non-recursive mutex to be
> acquired twice by the same thread => deadlock).

Hmm, thanks.

I now have the following problem:

  Sep 01 23:49:08 ERROR: `lt_dlopenext' failed for library `libgnunetmodule_topology' with error: libgnunetmodule_topology.so: cannot open shared object file: No such file or directory

Indeed, I do have `libgnunetmodule_topology_default' but not
`libgnunetmodule_topology'.  What did I miss?

Thanks in advance,
Ludo'.



_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: Re: Deadlock due to `pthread_mutex_lock'.

by Christian Grothoff :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

You must have an entry

topology=topology_default

under [MODULES] in the configuration.

Christian

On Monday 01 September 2008 03:51:03 pm Ludovic Courtès wrote:

> Hi Christian,
>
> Christian Grothoff <christian@...> writes:
> > Yep, I have an idea. Nobody ever tried to run the code without specifying
> > bandwidth limits explicitly (which causes a non-recursive mutex to be
> > acquired twice by the same thread => deadlock).
>
> Hmm, thanks.
>
> I now have the following problem:
>
>   Sep 01 23:49:08 ERROR: `lt_dlopenext' failed for library
> `libgnunetmodule_topology' with error: libgnunetmodule_topology.so: cannot
> open shared object file: No such file or directory
>
> Indeed, I do have `libgnunetmodule_topology_default' but not
> `libgnunetmodule_topology'.  What did I miss?
>
> Thanks in advance,
> Ludo'.
>
>
>
> _______________________________________________
> GNUnet-developers mailing list
> GNUnet-developers@...
> http://lists.gnu.org/mailman/listinfo/gnunet-developers




_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: Deadlock due to `pthread_mutex_lock'.

by Ludovic Courtès-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Christian,

Christian Grothoff <christian@...> writes:

> You must have an entry
>
> topology=topology_default
>
> under [MODULES] in the configuration.

Hmm, that doesn't make any difference.  The file contains this:

  [PATHS]
  GNUNETD_HOME = /var/lib/gnunet

  [GNUNETD]
  APPLICATIONS = advertising getoption fs stats traffic
  TRANSPORTS = udp tcp http nat

  [LOAD]
  MAXNETDOWNBPSTOTAL = 50000
  MAXNETUPBPSTOTAL = 50000
  HARDUPLIMIT = 0
  MAXCPULOAD = 100
  INTERFACES = eth0

  [MODULES]
  TOPOLOGY = topology_default

Looking at the output of `strace', `gnunetd' does read this config file,
but it looks only for `libgnunetmodule_topology', not
`libgnunetmodule_topology_default'.

Any idea?

Thanks in advance,
Ludo'.



_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers

Re: Deadlock due to `pthread_mutex_lock'.

by Ludovic Courtès-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

ludo@... (Ludovic Courtès) writes:

> Christian Grothoff <christian@...> writes:
>
>> You must have an entry
>>
>> topology=topology_default
>>
>> under [MODULES] in the configuration.

>   [MODULES]
>   TOPOLOGY = topology_default

Indeed, using lower-case `topology' does fix it.

I'd be nice if an error or warning was raised when an unknown option
name is encountered in `gnunetd.conf'.  :-)

Thanks!

Ludo'.



_______________________________________________
GNUnet-developers mailing list
GNUnet-developers@...
http://lists.gnu.org/mailman/listinfo/gnunet-developers