[jira] Created: (JMOCK-214) Fail fast when run concurrently

Fail fast when run concurrently
-------------------------------

                 Key: JMOCK-214
                 URL: http://jira.codehaus.org/browse/JMOCK-214
             Project: jMock
          Issue Type: Improvement
          Components: Library
         Environment: any
            Reporter: Douglas Squirrel


Steve F suggested I file this enhancement when we saw this behaviour at youDevise.

When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).

This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)

Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167062#action_167062 ]

Douglas Squirrel commented on JMOCK-214:
----------------------------------------

I just noticed JMock-213 which says JMock should be threadsafe. Obviously this would be preferable - I assumed thread safety was not possible (it sounds hard!)

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167097#action_167097 ]

Joerg Schaible commented on JMOCK-214:
--------------------------------------

Actually the expectation should not take a special thread id into account. When you define exactly(N) calls, it does not matter if all of those are done from the same thread or not.

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167101#action_167101 ]

Steve Freeman commented on JMOCK-214:
-------------------------------------

@Joerg. Actually it does, you can get misleading messages if one thread is still making calls while the test thread is trying to display the result.

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167104#action_167104 ]

Joerg Schaible commented on JMOCK-214:
--------------------------------------

@Steve. This was addressed originally in JMOCK-59. All threads had been terminated before validation of the result happened ...

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167257#action_167257 ]

Nat Pryce commented on JMOCK-214:
---------------------------------

The problem with that approach is that threads calling a mock object might have been started by a third-party library (a messaging API, for example), so the test has no control over their ThreadGroup.

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167257#action_167257 ]

Nat Pryce edited comment on JMOCK-214 at 2/26/09 8:05 PM:
----------------------------------------------------------

The problem with that approach is that threads calling a mock object might have been started by a third-party library (a messaging API, for example), so the test has no control over their ThreadGroup.

JMock cannot automagically make the test synchronise with the threads started by the system under test.  The Mockery can be thread-safe but at some point the programmer has to take responsibility for synchronisation between the test and their code.

      was (Author: npryce):
    The problem with that approach is that threads calling a mock object might have been started by a third-party library (a messaging API, for example), so the test has no control over their ThreadGroup.
 

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167493#action_167493 ]

Sebastian Sickelmann commented on JMOCK-214:
--------------------------------------------

There is a way to "control" threads created and started by third-party libraries. It is a litle bit tricky but it works. I use it in my project mockinject.

In mockinject i use ThreadLocal variables to store individual state, so every unittest can run it its own thread. To adress the problem of rescueing the ThreadLocal to newly created threads (also from third-party libraries), I catch all Thread-creation-steps and inject own threads that does the rescueing process. It is a little bit tricky and slows down the execution-speed a bit but works realy nice (have some problems with swing-event-thread but that is another story)

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167532#action_167532 ]

Nat Pryce commented on JMOCK-214:
---------------------------------

I'm uncomfortable with changing how third-party code behaves and then testing against it.  I want to test that my code works with the third-party code as is, not after I've mucked about with how it creates threads.

I think this is a sidetrack. It's no good waiting for third-party threads to terminate.  A third-party library might start long-lived daemon threads that outlast any single use of that API by a test.  They might exist to keep a connection alive, for example.  In which case, waiting for the threads to terminate would make the test hang.

See JMOCK-215 for how we're solving the issue of a test waiting for threads to finish.  Feedback welcome.

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167633#action_167633 ]

Nat Pryce commented on JMOCK-214:
---------------------------------

I've implemented this.  However, getting a clear failure is difficult because any error is going to be thrown onto the background thread, not onto the test thread.  What I've done is throw ConcurrentModificationException on the background thread and print a warning to stderr in case the thread swallows the exception.  The test should fail with an expectation failure, but if it doesn't (e.g. if the test only contains 'allowing' clauses) at least the error log will contain obvious warnings.

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

     [ http://jira.codehaus.org/browse/JMOCK-214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nat Pryce resolved JMOCK-214.
-----------------------------

       Resolution: Fixed
    Fix Version/s: 2.6.0

Resolved for when the user does not change the Mockery's imposteriser.  I'm not entirely happy with the implementation but it covers 90% of cases I think.

> Fail fast when run concurrently
> -------------------------------
>
>                 Key: JMOCK-214
>                 URL: http://jira.codehaus.org/browse/JMOCK-214
>             Project: jMock
>          Issue Type: Improvement
>          Components: Library
>         Environment: any
>            Reporter: Douglas Squirrel
>             Fix For: 2.6.0
>
>
> Steve F suggested I file this enhancement when we saw this behaviour at youDevise.
> When you build a test that executes in multiple threads, and it calls methods on a mocked object, you can get odd behaviour. For instance, our exactly(N).of expectation failed no matter how we set N or how many threads actually called the method (including N).
> This is by design, but it would help newcomers to JMock who don't know this if they got a clear failure instead of odd behaviour (as you do with ConcurrentModificationException for non-threadsafe collections). For instance, JMock could record the thread id on construction of the Mockery and then check the thread ID is the same on future invocations. (Not sure what performance effect this would have though.)
> Happy to try writing a unit test though I'm not sure if you want to start running multiple threads in your test suite.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email