[jira] Created: (JMOCK-202) Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+

Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
--------------------------------------------------------------------------------------------------------------------------

                 Key: JMOCK-202
                 URL: http://jira.codehaus.org/browse/JMOCK-202
             Project: jMock
          Issue Type: Bug
    Affects Versions: 2.5.1, 2.5.0, 2.5.0-RC1
         Environment: JMock 2.5.0-RC1+
JUnit 4.4
Java 1.5.0_16
Ubuntu Linux 8.0.4
kernel 2.6.24-19-generic
            Reporter: Steven Cummings
         Attachments: JMockBugTest.java

This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.

When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

     [ http://jira.codehaus.org/browse/JMOCK-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nat Pryce updated JMOCK-202:
----------------------------

    Component/s: Library

> Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JMOCK-202
>                 URL: http://jira.codehaus.org/browse/JMOCK-202
>             Project: jMock
>          Issue Type: Bug
>          Components: Library
>    Affects Versions: 2.5.0-RC1, 2.5.0, 2.5.1
>         Environment: JMock 2.5.0-RC1+
> JUnit 4.4
> Java 1.5.0_16
> Ubuntu Linux 8.0.4
> kernel 2.6.24-19-generic
>            Reporter: Steven Cummings
>         Attachments: JMockBugTest.java
>
>
> This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.
> When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167930#action_167930 ]

Nat Pryce commented on JMOCK-202:
---------------------------------

It's a bit dodgy to test this by catching the very error types that the test framework uses to report test failures!  

JUnit completely changed the way that it implements test runners in version 4.5, which jMock 2.5.0 uses.  It could be the cause of this issue.  We'll look into it.

I can think of two workarounds that, I think, would make your test better.

1) Use an ErrorTranslator to make the mock objects throw a different kind of error, one that doesn't interfere with JUnit.

2) Test the matcher directly.

> Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JMOCK-202
>                 URL: http://jira.codehaus.org/browse/JMOCK-202
>             Project: jMock
>          Issue Type: Bug
>          Components: Library
>    Affects Versions: 2.5.0-RC1, 2.5.0, 2.5.1
>         Environment: JMock 2.5.0-RC1+
> JUnit 4.4
> Java 1.5.0_16
> Ubuntu Linux 8.0.4
> kernel 2.6.24-19-generic
>            Reporter: Steven Cummings
>         Attachments: JMockBugTest.java
>
>
> This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.
> When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=167935#action_167935 ]

Steven Cummings commented on JMOCK-202:
---------------------------------------

I agree that in the case of a matcher, I should simply test it directly. So I should have separated that out from the bug itself. Where it will be an issue is when one wants to test a custom asserter for a testing library.

> Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JMOCK-202
>                 URL: http://jira.codehaus.org/browse/JMOCK-202
>             Project: jMock
>          Issue Type: Bug
>          Components: Library
>    Affects Versions: 2.5.0-RC1, 2.5.0, 2.5.1
>         Environment: JMock 2.5.0-RC1+
> JUnit 4.4
> Java 1.5.0_16
> Ubuntu Linux 8.0.4
> kernel 2.6.24-19-generic
>            Reporter: Steven Cummings
>         Attachments: JMockBugTest.java
>
>
> This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.
> When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=171135#action_171135 ]

Nat Pryce commented on JMOCK-202:
---------------------------------

If the test throws AssertionError directly instead of making the Mockery throw it after the test has finished, the JMock test runner reports it correctly.

> Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JMOCK-202
>                 URL: http://jira.codehaus.org/browse/JMOCK-202
>             Project: jMock
>          Issue Type: Bug
>          Components: Library
>    Affects Versions: 2.5.0-RC1, 2.5.0, 2.5.1
>         Environment: JMock 2.5.0-RC1+
> JUnit 4.4
> Java 1.5.0_16
> Ubuntu Linux 8.0.4
> kernel 2.6.24-19-generic
>            Reporter: Steven Cummings
>         Attachments: JMockBugTest.java
>
>
> This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.
> When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

     [ http://jira.codehaus.org/browse/JMOCK-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nat Pryce updated JMOCK-202:
----------------------------

    Priority: Minor  (was: Major)

Downgrading severity to Minor because it is a wierd use case and there are two workarounds (using an ErrorTranslator or explicitly calling assertIsSatisfied on the Mockery).

> Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JMOCK-202
>                 URL: http://jira.codehaus.org/browse/JMOCK-202
>             Project: jMock
>          Issue Type: Bug
>          Components: Library
>    Affects Versions: 2.5.0-RC1, 2.5.0, 2.5.1
>         Environment: JMock 2.5.0-RC1+
> JUnit 4.4
> Java 1.5.0_16
> Ubuntu Linux 8.0.4
> kernel 2.6.24-19-generic
>            Reporter: Steven Cummings
>            Priority: Minor
>         Attachments: JMockBugTest.java
>
>
> This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.
> When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JMOCK-202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=171137#action_171137 ]

Nat Pryce commented on JMOCK-202:
---------------------------------

The behaviour requested by this JIRA clashes with vital existing behaviour.  JMock reports if expectations are not met when a test throws an expected exception.  Therefore the "is satisfied" assertion must happen after the "is exception expected" assertion.

> Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JMOCK-202
>                 URL: http://jira.codehaus.org/browse/JMOCK-202
>             Project: jMock
>          Issue Type: Bug
>          Components: Library
>    Affects Versions: 2.5.0-RC1, 2.5.0, 2.5.1
>         Environment: JMock 2.5.0-RC1+
> JUnit 4.4
> Java 1.5.0_16
> Ubuntu Linux 8.0.4
> kernel 2.6.24-19-generic
>            Reporter: Steven Cummings
>            Priority: Minor
>         Attachments: JMockBugTest.java
>
>
> This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.
> When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

     [ http://jira.codehaus.org/browse/JMOCK-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nat Pryce closed JMOCK-202.
---------------------------

    Resolution: Won't Fix

> Unsatisfied expectations are not correctly captured with JUnit4's @Test(expected=AssertionError.class) in JMock 2.5.0-RC1+
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: JMOCK-202
>                 URL: http://jira.codehaus.org/browse/JMOCK-202
>             Project: jMock
>          Issue Type: Bug
>          Components: Library
>    Affects Versions: 2.5.0-RC1, 2.5.0, 2.5.1
>         Environment: JMock 2.5.0-RC1+
> JUnit 4.4
> Java 1.5.0_16
> Ubuntu Linux 8.0.4
> kernel 2.6.24-19-generic
>            Reporter: Steven Cummings
>            Priority: Minor
>         Attachments: JMockBugTest.java
>
>
> This was working up through version 2.4.0. I am testing some custom matchers with JMock by writing expectations that I expect to be unsatisfied when the test completes, and therefore I expect an AssertionError. From JMock 2.5.0-RC1 and on, @RunWith(JMock.class) causes @Test(expected=AssertionError.class) to fail to capture the resulting exception.
> When I say JMock 2.5.0-RC1+, I mean I could reproduce the problem with versions 2.5.0-RC1, 2.5.0, 2.5.0.1, and 2.5.1.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email