[jira] Created: (FELIX-1816) deadlock on SystemBundle.stop()

deadlock on SystemBundle.stop()
-------------------------------

                 Key: FELIX-1816
                 URL: https://issues.apache.org/jira/browse/FELIX-1816
             Project: Felix
          Issue Type: Bug
    Affects Versions: felix-2.0.1
            Reporter: Dennis Geurts
         Attachments: trace.log

When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.

I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dennis Geurts updated FELIX-1816:
---------------------------------

    Attachment: trace.log

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>         Attachments: trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770887#action_12770887 ]

Karl Pauls commented on FELIX-1816:
-----------------------------------

This doesn't seem to be correct in the sense that the systembundle.stop isn't the only thing contributing to the deadlock. The shutdownhook in the launcher is running and causing your problems again. Are you ctrl-c'ing or sending kill signals to the jvm externally? Just have a look at the stacktrace:

"SIGTERM handler" daemon prio=9 tid=0x0000000102830800 nid=0x114b94000 in Object.wait() [0x0000000114b93000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000010637fa10> (a org.apache.felix.main.Main$1)
        at java.lang.Thread.join(Thread.java:1167)
        - locked <0x000000010637fa10> (a org.apache.felix.main.Main$1)
        at java.lang.Thread.join(Thread.java:1220)
        at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:79)
        at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:24)
        at java.lang.Shutdown.runHooks(Shutdown.java:79)
        at java.lang.Shutdown.sequence(Shutdown.java:123)
        at java.lang.Shutdown.exit(Shutdown.java:168)
        - locked <0x000000010ba5b830> (a java.lang.Class for java.lang.Shutdown)
        at java.lang.Terminator$1.handle(Terminator.java:35)
        at sun.misc.Signal$1.run(Signal.java:195)
        at java.lang.Thread.run(Thread.java:637)

The jvm is getting a SIGTERM. Notice, in the other issue the stacktrace didn't contain a SIGTERM but just the call to system exit. If you don't do anything externally please attach an example again so that i can try to reproduce.

Anyways, I will have to investigate as a control-c together with a systembundle.stop shouldn't deadlock in the first place. Thanks for reporting :-)

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>         Attachments: trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Karl Pauls reassigned FELIX-1816:
---------------------------------

    Assignee: Karl Pauls

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>            Assignee: Karl Pauls
>         Attachments: trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770898#action_12770898 ]

Karl Pauls commented on FELIX-1816:
-----------------------------------

Ok, I think I see what is gonig on. You probably just hit ctrl-c to terminate the jvm after it was deadlocking.

Nevermind, this looks like a bug. I will try to fix it (but you might not like the solution). Basically, the issue seems to be that you are trying to stop the system bundle while it is still starting. This shouldn't deadlock but we probably will have to throw some exception from the stop method. I will have to look into this some more first, however. Thanks again.

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>            Assignee: Karl Pauls
>         Attachments: trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dennis Geurts updated FELIX-1816:
---------------------------------

    Attachment: trace-nohook.log
                Activator.java
                felix-framework-2.0.1.tar.gz

Thanks for looking into the issue. It's much appreciated!

Indeed I get my _stacktraces_ from a 'kill -QUIT'; Thus the SIGTERM handler kicking in.

I hadn't noticed that the SIGTERM isn't there in the trace of the other issue (FELIX-1812). Is this due to the fact that System.exit already has been called ?

the blocking factors are:
"Felix Shutdown Hook" prio=5 tid=0x0000000102031000 nid=0x114c97000 in Object.wait() [0x0000000114c96000]
        ...
        at org.apache.felix.framework.Felix.acquireBundleLock(Felix.java:4481)
        - locked <0x0000000106451f38> (a [Ljava.lang.Object;)

and
"FelixStartLevel" daemon prio=5 tid=0x00000001020f7800 nid=0x114904000 in Object.wait() [0x0000000114903000]
        ...
        at org.apache.felix.framework.Felix.acquireBundleLock(Felix.java:4481)
        - locked <0x0000000106451f38> (a [Ljava.lang.Object;)


Please find attached an additional stacktrace, also recorded using a 'kill -QUIT'  while running WITHOUT the shutdown hook. Then, the end result is such, that the java process is not stopped. There's no deadlock however, since ctrl-c'ing stops the process.

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>            Assignee: Karl Pauls
>         Attachments: Activator.java, felix-framework-2.0.1.tar.gz, trace-nohook.log, trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Dennis Geurts-2]

Thanks to you Karl !

dennis

On 28 okt 2009, at 12:36, Karl Pauls (JIRA) wrote:

>
>    [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770898#action_12770898 
>  ]
>
> Karl Pauls commented on FELIX-1816:
> -----------------------------------
>
> Ok, I think I see what is gonig on. You probably just hit ctrl-c to  
> terminate the jvm after it was deadlocking.
>
> Nevermind, this looks like a bug. I will try to fix it (but you  
> might not like the solution). Basically, the issue seems to be that  
> you are trying to stop the system bundle while it is still starting.  
> This shouldn't deadlock but we probably will have to throw some  
> exception from the stop method. I will have to look into this some  
> more first, however. Thanks again.
>
>> deadlock on SystemBundle.stop()
>> -------------------------------
>>
>>                Key: FELIX-1816
>>                URL: https://issues.apache.org/jira/browse/FELIX-1816
>>            Project: Felix
>>         Issue Type: Bug
>>   Affects Versions: felix-2.0.1
>>           Reporter: Dennis Geurts
>>           Assignee: Karl Pauls
>>        Attachments: trace.log
>>
>>
>> When one calls SystemBundle.stop within BundleActivator.start
>> (BundleContext) a deadlock occurs.
>> I will attach a stacktrace.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>

Violence is the last refuge of the incompetent.
-------------------------------------------------
Dennis Geurts
luminis
Software Development B.V.
K.v.k Centraal Gelderland: 09 16 28 87
-------------------------------------------------
[phone]
dennis.geurts@...
www.luminis.nl
-------------------------------------------------

     [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Karl Pauls resolved FELIX-1816.
-------------------------------

       Resolution: Fixed
    Fix Version/s: felix-2.2.0

Ok, well, the problem is in this case that the system bundle is still in the STARTING state when it's stop method is called. That get us into the deadlock.

Now, a proper solution would probably include a timeout mechanism but for now i just resolved the issue by having the framework throw an exception as a result of the stop call (this is allowed by the spec and would happen with a timeout too - eventually). Downside for you is that you don't get what you want (i.e., the framework shutdown) but an exception inside your Activator.start. Again, what you would have to do is to at least wait for the framework to be in the ACTIVE state before calling the stop method (you can check the state and if its not ACTIVE then register a framework listener and wait for that event).

However, as a side effect this makes it possible again to call System.exit inside an activator.start method while the framework is still in the STARTING state. See FELIX-1812 for more on that.

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>            Assignee: Karl Pauls
>             Fix For: felix-2.2.0
>
>         Attachments: Activator.java, felix-framework-2.0.1.tar.gz, trace-nohook.log, trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771959#action_12771959 ]

Karl Pauls commented on FELIX-1816:
-----------------------------------

Please close this issue if you are satisfied with the change to trunk.

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>            Assignee: Karl Pauls
>             Fix For: felix-2.2.0
>
>         Attachments: Activator.java, felix-framework-2.0.1.tar.gz, trace-nohook.log, trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Dennis Geurts-2]

Thank you, Karl. I'll look into it as soon as possible.

Cheers, Dennis

Sent from my iPhone

On 30 okt 2009, at 15:40, "Karl Pauls (JIRA)" <jira@...> wrote:

>
>    [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771959#action_12771959 
>  ]
>
> Karl Pauls commented on FELIX-1816:
> -----------------------------------
>
> Please close this issue if you are satisfied with the change to trunk.
>
>> deadlock on SystemBundle.stop()
>> -------------------------------
>>
>>                Key: FELIX-1816
>>                URL: https://issues.apache.org/jira/browse/FELIX-1816
>>            Project: Felix
>>         Issue Type: Bug
>>   Affects Versions: felix-2.0.1
>>           Reporter: Dennis Geurts
>>           Assignee: Karl Pauls
>>            Fix For: felix-2.2.0
>>
>>        Attachments: Activator.java, felix-framework-2.0.1.tar.gz,  
>> trace-nohook.log, trace.log
>>
>>
>> When one calls SystemBundle.stop within BundleActivator.start
>> (BundleContext) a deadlock occurs.
>> I will attach a stacktrace.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>

     [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dennis Geurts closed FELIX-1816.
--------------------------------


verified, indeed the exception is thrown, with a very good explanation of the situation.

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>            Assignee: Karl Pauls
>             Fix For: felix-2.2.0
>
>         Attachments: Activator.java, felix-framework-2.0.1.tar.gz, trace-nohook.log, trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/FELIX-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Karl Pauls reopened FELIX-1816:
-------------------------------


I re-open this issue as I really don't like the solution (and it makes various other cases fail). I will have to get back to this and fix it correctly by introducing timeouts or something. The result for this case would be the same I imagine but for now i will rollback.

> deadlock on SystemBundle.stop()
> -------------------------------
>
>                 Key: FELIX-1816
>                 URL: https://issues.apache.org/jira/browse/FELIX-1816
>             Project: Felix
>          Issue Type: Bug
>    Affects Versions: felix-2.0.1
>            Reporter: Dennis Geurts
>            Assignee: Karl Pauls
>             Fix For: felix-2.2.0
>
>         Attachments: Activator.java, felix-framework-2.0.1.tar.gz, trace-nohook.log, trace.log
>
>
> When one calls SystemBundle.stop within BundleActivator.start(BundleContext) a deadlock occurs.
> I will attach a stacktrace.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.