Nabble - JUnit - User

Re: Categories in JUnit 4.8

2009-11-21T16:18:48Z

Hi David!

A minor thing: why did you decide to use an additional annotation and did not add a property to @SuiteClasses?

E.g., your example (on http://saffgreenbar.blogspot.com/2009/11/heres-test-code-snippet-public.html)

@RunWith(Categories.class)
@IncludeCategory(SlowTests.class)
@SuiteClasses( { A.class, B.class, C.class })
public static class SlowTestSuite {
}

could read

@RunWith(Suite.class)
@SuiteClasses( { A.class, B.class, C.class },
categories={SlowTests.class})
public static class SlowTestSuite {
}

This would make it more visible which tests a suite includes and reduces the number of annotations.

Regards,
Georg

Re: Re: Categories in JUnit 4.8

2009-11-21T03:24:38Z

2009/11/20 Cédric Beust ♔ <cbeust@...>:

> On Fri, Nov 20, 2009 at 7:07 PM, Georg <le_garcon_enerve@...> wrote:
>
>>
>>
>> Hi David,
>>
>> Did you gave thoughs to cases where one would like to exclude certain
>> tests? E.g.: run all tests that are not in (a) certain category(s)? This is
>> probably not difficult add in one way or another onto the concept of
>> categories, but the answer to "which test is now acctually included" could
>> become confusing in cases where a test is in an included and an excluded
>> category. I would like to see something here that does not require a lot of
>> browsing through hierarchies of suites nad categories. This may in my
>> approach not well solved either as suites can extend on other suites: if a
>> referenced suite includes a certain test, it is executed even if the
>> extending suite excludes a test.
>>
>> And more fundamentally: is the idea/need of exclusion meaningfull or just a
>> result of badly organising tests (I'd say no, but I may be wrong)?
>>
>
> For what it's worth, excluding tests is a very popular TestNG features. For
> example, I use it when a few tests are broken and need to remain so for a
> while: I put them in the group "broken", which I always exclude during my
> runs.
>
> When the time to ship approaches, I look at all the tests in the "broken"
> category and I go fix them then.

For this particular use case, I've found @Ignore generally useful. I
also really love Kevin Lawrence's idea of marking tests with bug
numbers. He does it like this:

@Test public void shouldntDeleteAllTheUsersFiles() {
bug(34456);
...
}

The test only starts failing once bug 34456 is marked as fixed in the
bug system. Once we have failing assumptions working as they truly
should, this will be easy to implement on the JUnit side, although the
bug-system integration would still be up to individual extenders or
teams.

All this is to say that I do get the idea of excluded categories, but
this particular instance doesn't immediately convince me.

David Saff

>
> --
> ***Cédric
> *
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Re: Re: Categories in JUnit 4.8

2009-11-21T03:11:20Z

On Fri, Nov 20, 2009 at 10:07 PM, Georg <le_garcon_enerve@...> wrote:
> Hi David,
>
> Did you gave thoughs to cases where one would like to exclude certain tests? E.g.: run all tests that are not in (a) certain category(s)? This is probably not difficult add in one way or another onto the concept of categories, but the answer to "which test is now acctually included" could become confusing in cases where a test is in an included and an excluded category. I would like to see something here that does not require a lot of browsing through hierarchies of suites nad categories. This may in my approach not well solved either as suites can extend on other suites: if a referenced suite includes a certain test, it is executed even if the extending suite excludes a test.

Georg,

We came very close to adding an ExcludeCategories annotation.
However, we decided to stay at a minimal feature set for Categories in
the initial release, figuring that if the whole API turned out to be
wrong, or if users just shrugged, then any time put into perfecting
the mini-language of category filter configuration would be wasted.
If the feature takes off, I think we'll certainly revisit that
decision (especially if a highly-voted github issue requests it, hint,
hint.)

However, Iyou touch on another concern: a simple category feature set
that is easy to understand, but leaves out 20% of valid use cases, may
be preferable to a more "feature complete" feature implementation that
requires pages of documentation to fully grok. I don't know that
we're in danger of either extreme yet.

David Saff

>
> And more fundamentally: is the idea/need of exclusion meaningfull or just a result of badly organising tests (I'd say no, but I may be wrong)?
>
> Cheers,
> Georg
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Re: Re: Categories in JUnit 4.8

2009-11-20T20:59:48Z

On Fri, Nov 20, 2009 at 7:07 PM, Georg <le_garcon_enerve@...> wrote:

>
>
> Hi David,
>
> Did you gave thoughs to cases where one would like to exclude certain
> tests? E.g.: run all tests that are not in (a) certain category(s)? This is
> probably not difficult add in one way or another onto the concept of
> categories, but the answer to "which test is now acctually included" could
> become confusing in cases where a test is in an included and an excluded
> category. I would like to see something here that does not require a lot of
> browsing through hierarchies of suites nad categories. This may in my
> approach not well solved either as suites can extend on other suites: if a
> referenced suite includes a certain test, it is executed even if the
> extending suite excludes a test.
>
> And more fundamentally: is the idea/need of exclusion meaningfull or just a
> result of badly organising tests (I'd say no, but I may be wrong)?
>

For what it's worth, excluding tests is a very popular TestNG features. For
example, I use it when a few tests are broken and need to remain so for a
while: I put them in the group "broken", which I always exclude during my
runs.

When the time to ship approaches, I look at all the tests in the "broken"
category and I go fix them then.

--
***Cédric
*

[Non-text portions of this message have been removed]

Re: Categories in JUnit 4.8

2009-11-20T19:07:28Z

Hi David,

Did you gave thoughs to cases where one would like to exclude certain tests? E.g.: run all tests that are not in (a) certain category(s)? This is probably not difficult add in one way or another onto the concept of categories, but the answer to "which test is now acctually included" could become confusing in cases where a test is in an included and an excluded category. I would like to see something here that does not require a lot of browsing through hierarchies of suites nad categories. This may in my approach not well solved either as suites can extend on other suites: if a referenced suite includes a certain test, it is executed even if the extending suite excludes a test.

And more fundamentally: is the idea/need of exclusion meaningfull or just a result of badly organising tests (I'd say no, but I may be wrong)?

Cheers,
Georg

Re: Categories in JUnit 4.8

2009-11-20T18:41:44Z

Sorry!

The URL is http://github.com/KentBeck/junit/issues/#issue/37 (can the issue number change? I have memories of this being number 41). The title is: Feature request: Making test suites less tedious.

On a side note: this does not allow for direct inheritance like in the categories, but for a combination of tests and suites, which, IMHO, achieves the same flexibility.

Cheers,
Georg

Re: Gathering and displaying supplemental information about test failures.

2009-11-20T16:15:14Z

Thanks David. I'll give the assertThat a close look.

On Fri, Nov 20, 2009 at 2:17 PM, David Saff <david@...> wrote:

>
>
> Landon,
>
> Definitely take a close look at the failure messages from the standard
> assert methods, and also the assertThat method added in 4.4. Good
> luck,
>
> David Saff
>
> On Fri, Nov 20, 2009 at 2:32 PM, Sunburned Surveyor
>
> <sunburned.surveyor@... <sunburned.surveyor%40gmail.com>> wrote:
> > David,
> >
> > In this particular case your technique would work.
> >
> > I wonder, when testing return values that are slightly more complicated,
> if
> > this would start to fall apart. I don't have specific examples I can give
> > you just yet.
> >
> > I will give this some more thought, and maybe try some code. Before I
> went
> > very far I wanted to make sure there wasn't an existing system I should
> use.
> >
> > Thanks for the responses. I have listened to the JUnit group for a while,
> > but this was my first post.
> >
> > Landon (AKA The Sunburned Surveyor)
> >
> >
> >
> > On Fri, Nov 20, 2009 at 11:22 AM, David Saff <david@...<david%40saff.net>>
> wrote:
> >
> >>
> >>
> >> Dear Sunburned,
> >>
> >> If I write
> >>
> >> assertEquals("(1,-1,0)", point.toString());
> >>
> >> and change the toString method, I get this (slightly garbled, but
> >> mostly helpful) failure:
> >>
> >> org.junit.ComparisonFailure: expected:<[(1,-1,0)]> but was:<[[1, -1,
> 0]]>
> >> at org.junit.Assert.assertEquals(Assert.java:123)...
> >>
> >> This seems to cover the one example you cite. Are there others?
> >>
> >> David Saff
> >>
> >>
> >> On Fri, Nov 20, 2009 at 1:06 PM, Sunburned Surveyor
> >> <sunburned.surveyor@... <sunburned.surveyor%40gmail.com><sunburned.surveyor%
> 40gmail.com>> wrote:
> >> > I have a question about the options for gathering and displaying
> >> > supplemental information about test failures. Please allow me to
> >> > illustrate my question with a simple example.
> >> >
> >> > I've got a class called Coordinate that represents a point in a 3D
> >> > coordinate system. I've overloaded the toString method of this class
> >> > to write out the X, Y, and Z values of the coordinate in a simple
> >> > String. Now I want to test my toString method using JUnit.
> >> >
> >> > If the Coordinate.toString test method fails (because the string was
> >> > not constructed in the correct format) I would like to provide the
> >> > programmer running the test with a copy of the incorrect string, since
> >> > it provides valuable clues about what is broken with the method under
> >> > test. I typically do this by placing System.err.println statements in
> >> > my test method. However, I realize that using System.err.println isn't
> >> > the most portable or elegant way to provide this information to the
> >> > programmer executing the test.
> >> >
> >> > What I really want is a GUI that allows the programmer to select a
> >> > failed test from a list of failed tests, and then to view supplemental
> >> > test data I provide him/her.
> >> >
> >> > I was thinking I could do this by completing the following tasks:
> >> >
> >> > - Implementing the RunListener interface in JUnit.
> >> > - Create an extension of Exception that could contain a HashMap of
> >> > simple parameters and that would be passed to the constructor of a
> >> > Failure object from within a test.
> >> > - Creating Failure objects in my tests that contain the supplemental
> >> > failrure data.
> >> > - Passing these failure objects to my RunListener implementaiton.
> >> > - Creating a FailureProcessor class that displays a GUI with the
> >> > information in my RunListener implemenation to show the programmer
> >> > executing my tests the supplemental failure data as described above.
> >> > (FailureProcessor would be an interface so that you could do different
> >> > things with the supplemental failure data, not just show it in a GUI.)
> >> >
> >> > Before I begin this work, I wanted to ask if there was a better way to
> >> > do this. I know I could use a logging framework from within my test
> >> > method, but I really want to work within the JUnut framework, not
> >> > outside of it. I also want a nice little GUI, not a log file.
> >> >
> >> > How are other programmers handling instances when they want to display
> >> > supplemental data on failed tests? Does my approach sound reasonable?
> >> >
> >> > The Sunburned Surveyor
> >> >
> >> >
> >> > ------------------------------------
> >> >
> >> > Yahoo! Groups Links
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>

[Non-text portions of this message have been removed]

Re: Gathering and displaying supplemental information about test failures.

2009-11-20T14:17:47Z

Landon,

Definitely take a close look at the failure messages from the standard
assert methods, and also the assertThat method added in 4.4. Good
luck,

David Saff

On Fri, Nov 20, 2009 at 2:32 PM, Sunburned Surveyor
<sunburned.surveyor@...> wrote:

> David,
>
> In this particular case your technique would work.
>
> I wonder, when testing return values that are slightly more complicated, if
> this would start to fall apart. I don't have specific examples I can give
> you just yet.
>
> I will give this some more thought, and maybe try some code. Before I went
> very far I wanted to make sure there wasn't an existing system I should use.
>
> Thanks for the responses. I have listened to the JUnit group for a while,
> but this was my first post.
>
> Landon (AKA The Sunburned Surveyor)
>
>
>
> On Fri, Nov 20, 2009 at 11:22 AM, David Saff <david@...> wrote:
>
>>
>>
>> Dear Sunburned,
>>
>> If I write
>>
>> assertEquals("(1,-1,0)", point.toString());
>>
>> and change the toString method, I get this (slightly garbled, but
>> mostly helpful) failure:
>>
>> org.junit.ComparisonFailure: expected:<[(1,-1,0)]> but was:<[[1, -1, 0]]>
>> at org.junit.Assert.assertEquals(Assert.java:123)...
>>
>> This seems to cover the one example you cite. Are there others?
>>
>> David Saff
>>
>>
>> On Fri, Nov 20, 2009 at 1:06 PM, Sunburned Surveyor
>> <sunburned.surveyor@... <sunburned.surveyor%40gmail.com>> wrote:
>> > I have a question about the options for gathering and displaying
>> > supplemental information about test failures. Please allow me to
>> > illustrate my question with a simple example.
>> >
>> > I've got a class called Coordinate that represents a point in a 3D
>> > coordinate system. I've overloaded the toString method of this class
>> > to write out the X, Y, and Z values of the coordinate in a simple
>> > String. Now I want to test my toString method using JUnit.
>> >
>> > If the Coordinate.toString test method fails (because the string was
>> > not constructed in the correct format) I would like to provide the
>> > programmer running the test with a copy of the incorrect string, since
>> > it provides valuable clues about what is broken with the method under
>> > test. I typically do this by placing System.err.println statements in
>> > my test method. However, I realize that using System.err.println isn't
>> > the most portable or elegant way to provide this information to the
>> > programmer executing the test.
>> >
>> > What I really want is a GUI that allows the programmer to select a
>> > failed test from a list of failed tests, and then to view supplemental
>> > test data I provide him/her.
>> >
>> > I was thinking I could do this by completing the following tasks:
>> >
>> > - Implementing the RunListener interface in JUnit.
>> > - Create an extension of Exception that could contain a HashMap of
>> > simple parameters and that would be passed to the constructor of a
>> > Failure object from within a test.
>> > - Creating Failure objects in my tests that contain the supplemental
>> > failrure data.
>> > - Passing these failure objects to my RunListener implementaiton.
>> > - Creating a FailureProcessor class that displays a GUI with the
>> > information in my RunListener implemenation to show the programmer
>> > executing my tests the supplemental failure data as described above.
>> > (FailureProcessor would be an interface so that you could do different
>> > things with the supplemental failure data, not just show it in a GUI.)
>> >
>> > Before I begin this work, I wanted to ask if there was a better way to
>> > do this. I know I could use a logging framework from within my test
>> > method, but I really want to work within the JUnut framework, not
>> > outside of it. I also want a nice little GUI, not a log file.
>> >
>> > How are other programmers handling instances when they want to display
>> > supplemental data on failed tests? Does my approach sound reasonable?
>> >
>> > The Sunburned Surveyor
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>>
>>
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Re: Gathering and displaying supplemental information about test failures.

2009-11-20T11:45:03Z

Big Mike,

Thanks for the info. I don't know that the system I have in mind will
require any tweaks to JUnit. I think all of my code can run along side of
it, so I won't need to fork JUnit at all. I will know more in a couple of
days.

I'm very much a JUnit rookie, so I won't presume I can improve it. :]

If I get some code working I will post a link to it so other JUnit users can
evaluate and make suggestions on improving it.

Landon

On Fri, Nov 20, 2009 at 11:33 AM, Forsberg, Mike <
mike.forsberg@...> wrote:

>
>
> You might consider extending/changing/refactoring JUnit itself using
> github.
>
> There you can fork JUnit itself and submit your idea for inclusion for
> everyone. It is a very easy system. Yet, Kent and David do an excellent job
> keeping the code consistent. (Enhancements are considered, but there is a
> high bar for inclusion into the code base.)
>
> With all that said, I think David Saff response covers your initial
> question.
>
>
> Big Mike
>
> From: junit@... <junit%40yahoogroups.com> [mailto:
> junit@... <junit%40yahoogroups.com>] On Behalf Of Sunburned
> Surveyor
> Sent: Friday, November 20, 2009 1:22 PM
> To: junit@... <junit%40yahoogroups.com>
> Subject: Re: [junit] Gathering and displaying supplemental information
> about test failures.
>
>
> Mike,
>
> I've never seen it. I am interested, but ultimately we are looking for a
> solution that will work outside of Eclipse. We are working on an open
> source
> library, and we can't know what IDE/build environment other contributors
> will be using.
>
> Maybe if I get something worked up they would consider making it a
> supplemental part of JUnit. In the worst case, I'll make a distro of my
> code
> available on SourceForge.
>
> Landon
>
> On Fri, Nov 20, 2009 at 11:20 AM, Forsberg, Mike <
> mike.forsberg@... <mike.forsberg%40globalfoundries.com>
> <mailto:mike.forsberg%40globalfoundries.com<mike.forsberg%2540globalfoundries.com>>>
> wrote:
>
> >
> >
> > Have you looked at how eclipse handles AssertEquals of two Strings.
> >
> > Not sure if this is a JUnit core functionality or not. But eclipse
> > highlights how the two strings differ.
> >
> > Just wondering if you've ever seen this functionality... I'll send screen
> > shots if not.
> >
> > Big Mike
> >
> > From: junit@... <junit%40yahoogroups.com><mailto:
> junit%40yahoogroups.com <junit%2540yahoogroups.com>> <junit%
> 40yahoogroups.com> [mailto:
> > junit@... <junit%40yahoogroups.com><mailto:
> junit%40yahoogroups.com <junit%2540yahoogroups.com>> <junit%
> 40yahoogroups.com>] On Behalf Of Sunburned
>
> > Surveyor
> > Sent: Friday, November 20, 2009 12:06 PM
> > To: junit@... <junit%40yahoogroups.com><mailto:
> junit%40yahoogroups.com <junit%2540yahoogroups.com>> <junit%
> 40yahoogroups.com>
>
> > Cc: Martin Davis; Martin Davis
> > Subject: [junit] Gathering and displaying supplemental information about
> > test failures.
> >
> >
> > I have a question about the options for gathering and displaying
> > supplemental information about test failures. Please allow me to
> > illustrate my question with a simple example.
> >
> > I've got a class called Coordinate that represents a point in a 3D
> > coordinate system. I've overloaded the toString method of this class
> > to write out the X, Y, and Z values of the coordinate in a simple
> > String. Now I want to test my toString method using JUnit.
> >
> > If the Coordinate.toString test method fails (because the string was
> > not constructed in the correct format) I would like to provide the
> > programmer running the test with a copy of the incorrect string, since
> > it provides valuable clues about what is broken with the method under
> > test. I typically do this by placing System.err.println statements in
> > my test method. However, I realize that using System.err.println isn't
> > the most portable or elegant way to provide this information to the
> > programmer executing the test.
> >
> > What I really want is a GUI that allows the programmer to select a
> > failed test from a list of failed tests, and then to view supplemental
> > test data I provide him/her.
> >
> > I was thinking I could do this by completing the following tasks:
> >
> > - Implementing the RunListener interface in JUnit.
> > - Create an extension of Exception that could contain a HashMap of
> > simple parameters and that would be passed to the constructor of a
> > Failure object from within a test.
> > - Creating Failure objects in my tests that contain the supplemental
> > failrure data.
> > - Passing these failure objects to my RunListener implementaiton.
> > - Creating a FailureProcessor class that displays a GUI with the
> > information in my RunListener implemenation to show the programmer
> > executing my tests the supplemental failure data as described above.
> > (FailureProcessor would be an interface so that you could do different
> > things with the supplemental failure data, not just show it in a GUI.)
> >
> > Before I begin this work, I wanted to ask if there was a better way to
> > do this. I know I could use a logging framework from within my test
> > method, but I really want to work within the JUnut framework, not
> > outside of it. I also want a nice little GUI, not a log file.
> >
> > How are other programmers handling instances when they want to display
> > supplemental data on failed tests? Does my approach sound reasonable?
> >
> > The Sunburned Surveyor
> >
> > [Non-text portions of this message have been removed]
> >
> >
> >
>
> [Non-text portions of this message have been removed]
>
> [Non-text portions of this message have been removed]
>
>
>

[Non-text portions of this message have been removed]

RE: Gathering and displaying supplemental information about test failures.

2009-11-20T11:33:41Z

You might consider extending/changing/refactoring JUnit itself using github.

There you can fork JUnit itself and submit your idea for inclusion for everyone. It is a very easy system. Yet, Kent and David do an excellent job keeping the code consistent. (Enhancements are considered, but there is a high bar for inclusion into the code base.)

With all that said, I think David Saff response covers your initial question.

Big Mike

From: junit@... [mailto:junit@...] On Behalf Of Sunburned Surveyor
Sent: Friday, November 20, 2009 1:22 PM
To: junit@...
Subject: Re: [junit] Gathering and displaying supplemental information about test failures.

Mike,

I've never seen it. I am interested, but ultimately we are looking for a
solution that will work outside of Eclipse. We are working on an open source
library, and we can't know what IDE/build environment other contributors
will be using.

Maybe if I get something worked up they would consider making it a
supplemental part of JUnit. In the worst case, I'll make a distro of my code
available on SourceForge.

Landon

On Fri, Nov 20, 2009 at 11:20 AM, Forsberg, Mike <
mike.forsberg@...<mailto:mike.forsberg%40globalfoundries.com>> wrote:

>
>
> Have you looked at how eclipse handles AssertEquals of two Strings.
>
> Not sure if this is a JUnit core functionality or not. But eclipse
> highlights how the two strings differ.
>
> Just wondering if you've ever seen this functionality... I'll send screen
> shots if not.
>
> Big Mike
>
> From: junit@...<mailto:junit%40yahoogroups.com> <junit%40yahoogroups.com> [mailto:
> junit@...<mailto:junit%40yahoogroups.com> <junit%40yahoogroups.com>] On Behalf Of Sunburned
> Surveyor
> Sent: Friday, November 20, 2009 12:06 PM
> To: junit@...<mailto:junit%40yahoogroups.com> <junit%40yahoogroups.com>
> Cc: Martin Davis; Martin Davis
> Subject: [junit] Gathering and displaying supplemental information about
> test failures.
>
>
> I have a question about the options for gathering and displaying
> supplemental information about test failures. Please allow me to
> illustrate my question with a simple example.
>
> I've got a class called Coordinate that represents a point in a 3D
> coordinate system. I've overloaded the toString method of this class
> to write out the X, Y, and Z values of the coordinate in a simple
> String. Now I want to test my toString method using JUnit.
>
> If the Coordinate.toString test method fails (because the string was
> not constructed in the correct format) I would like to provide the
> programmer running the test with a copy of the incorrect string, since
> it provides valuable clues about what is broken with the method under
> test. I typically do this by placing System.err.println statements in
> my test method. However, I realize that using System.err.println isn't
> the most portable or elegant way to provide this information to the
> programmer executing the test.
>
> What I really want is a GUI that allows the programmer to select a
> failed test from a list of failed tests, and then to view supplemental
> test data I provide him/her.
>
> I was thinking I could do this by completing the following tasks:
>
> - Implementing the RunListener interface in JUnit.
> - Create an extension of Exception that could contain a HashMap of
> simple parameters and that would be passed to the constructor of a
> Failure object from within a test.
> - Creating Failure objects in my tests that contain the supplemental
> failrure data.
> - Passing these failure objects to my RunListener implementaiton.
> - Creating a FailureProcessor class that displays a GUI with the
> information in my RunListener implemenation to show the programmer
> executing my tests the supplemental failure data as described above.
> (FailureProcessor would be an interface so that you could do different
> things with the supplemental failure data, not just show it in a GUI.)
>
> Before I begin this work, I wanted to ask if there was a better way to
> do this. I know I could use a logging framework from within my test
> method, but I really want to work within the JUnut framework, not
> outside of it. I also want a nice little GUI, not a log file.
>
> How are other programmers handling instances when they want to display
> supplemental data on failed tests? Does my approach sound reasonable?
>
> The Sunburned Surveyor
>
> [Non-text portions of this message have been removed]
>
>
>

[Non-text portions of this message have been removed]

[Non-text portions of this message have been removed]

Re: Gathering and displaying supplemental information about test failures.

2009-11-20T11:32:23Z

David,

In this particular case your technique would work.

I wonder, when testing return values that are slightly more complicated, if
this would start to fall apart. I don't have specific examples I can give
you just yet.

I will give this some more thought, and maybe try some code. Before I went
very far I wanted to make sure there wasn't an existing system I should use.

Thanks for the responses. I have listened to the JUnit group for a while,
but this was my first post.

Landon (AKA The Sunburned Surveyor)

On Fri, Nov 20, 2009 at 11:22 AM, David Saff <david@...> wrote:

>
>
> Dear Sunburned,
>
> If I write
>
> assertEquals("(1,-1,0)", point.toString());
>
> and change the toString method, I get this (slightly garbled, but
> mostly helpful) failure:
>
> org.junit.ComparisonFailure: expected:<[(1,-1,0)]> but was:<[[1, -1, 0]]>
> at org.junit.Assert.assertEquals(Assert.java:123)...
>
> This seems to cover the one example you cite. Are there others?
>
> David Saff
>
>
> On Fri, Nov 20, 2009 at 1:06 PM, Sunburned Surveyor
> <sunburned.surveyor@... <sunburned.surveyor%40gmail.com>> wrote:
> > I have a question about the options for gathering and displaying
> > supplemental information about test failures. Please allow me to
> > illustrate my question with a simple example.
> >
> > I've got a class called Coordinate that represents a point in a 3D
> > coordinate system. I've overloaded the toString method of this class
> > to write out the X, Y, and Z values of the coordinate in a simple
> > String. Now I want to test my toString method using JUnit.
> >
> > If the Coordinate.toString test method fails (because the string was
> > not constructed in the correct format) I would like to provide the
> > programmer running the test with a copy of the incorrect string, since
> > it provides valuable clues about what is broken with the method under
> > test. I typically do this by placing System.err.println statements in
> > my test method. However, I realize that using System.err.println isn't
> > the most portable or elegant way to provide this information to the
> > programmer executing the test.
> >
> > What I really want is a GUI that allows the programmer to select a
> > failed test from a list of failed tests, and then to view supplemental
> > test data I provide him/her.
> >
> > I was thinking I could do this by completing the following tasks:
> >
> > - Implementing the RunListener interface in JUnit.
> > - Create an extension of Exception that could contain a HashMap of
> > simple parameters and that would be passed to the constructor of a
> > Failure object from within a test.
> > - Creating Failure objects in my tests that contain the supplemental
> > failrure data.
> > - Passing these failure objects to my RunListener implementaiton.
> > - Creating a FailureProcessor class that displays a GUI with the
> > information in my RunListener implemenation to show the programmer
> > executing my tests the supplemental failure data as described above.
> > (FailureProcessor would be an interface so that you could do different
> > things with the supplemental failure data, not just show it in a GUI.)
> >
> > Before I begin this work, I wanted to ask if there was a better way to
> > do this. I know I could use a logging framework from within my test
> > method, but I really want to work within the JUnut framework, not
> > outside of it. I also want a nice little GUI, not a log file.
> >
> > How are other programmers handling instances when they want to display
> > supplemental data on failed tests? Does my approach sound reasonable?
> >
> > The Sunburned Surveyor
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
>

[Non-text portions of this message have been removed]

Re: Gathering and displaying supplemental information about test failures.

2009-11-20T11:22:28Z

Mike,

I've never seen it. I am interested, but ultimately we are looking for a
solution that will work outside of Eclipse. We are working on an open source
library, and we can't know what IDE/build environment other contributors
will be using.

Maybe if I get something worked up they would consider making it a
supplemental part of JUnit. In the worst case, I'll make a distro of my code
available on SourceForge.

Landon

On Fri, Nov 20, 2009 at 11:20 AM, Forsberg, Mike <
mike.forsberg@...> wrote:

>
>
> Have you looked at how eclipse handles AssertEquals of two Strings.
>
> Not sure if this is a JUnit core functionality or not. But eclipse
> highlights how the two strings differ.
>
> Just wondering if you've ever seen this functionality... I'll send screen
> shots if not.
>
> Big Mike
>
> From: junit@... <junit%40yahoogroups.com> [mailto:
> junit@... <junit%40yahoogroups.com>] On Behalf Of Sunburned
> Surveyor
> Sent: Friday, November 20, 2009 12:06 PM
> To: junit@... <junit%40yahoogroups.com>
> Cc: Martin Davis; Martin Davis
> Subject: [junit] Gathering and displaying supplemental information about
> test failures.
>
>
> I have a question about the options for gathering and displaying
> supplemental information about test failures. Please allow me to
> illustrate my question with a simple example.
>
> I've got a class called Coordinate that represents a point in a 3D
> coordinate system. I've overloaded the toString method of this class
> to write out the X, Y, and Z values of the coordinate in a simple
> String. Now I want to test my toString method using JUnit.
>
> If the Coordinate.toString test method fails (because the string was
> not constructed in the correct format) I would like to provide the
> programmer running the test with a copy of the incorrect string, since
> it provides valuable clues about what is broken with the method under
> test. I typically do this by placing System.err.println statements in
> my test method. However, I realize that using System.err.println isn't
> the most portable or elegant way to provide this information to the
> programmer executing the test.
>
> What I really want is a GUI that allows the programmer to select a
> failed test from a list of failed tests, and then to view supplemental
> test data I provide him/her.
>
> I was thinking I could do this by completing the following tasks:
>
> - Implementing the RunListener interface in JUnit.
> - Create an extension of Exception that could contain a HashMap of
> simple parameters and that would be passed to the constructor of a
> Failure object from within a test.
> - Creating Failure objects in my tests that contain the supplemental
> failrure data.
> - Passing these failure objects to my RunListener implementaiton.
> - Creating a FailureProcessor class that displays a GUI with the
> information in my RunListener implemenation to show the programmer
> executing my tests the supplemental failure data as described above.
> (FailureProcessor would be an interface so that you could do different
> things with the supplemental failure data, not just show it in a GUI.)
>
> Before I begin this work, I wanted to ask if there was a better way to
> do this. I know I could use a logging framework from within my test
> method, but I really want to work within the JUnut framework, not
> outside of it. I also want a nice little GUI, not a log file.
>
> How are other programmers handling instances when they want to display
> supplemental data on failed tests? Does my approach sound reasonable?
>
> The Sunburned Surveyor
>
> [Non-text portions of this message have been removed]
>
>
>

[Non-text portions of this message have been removed]

Re: Gathering and displaying supplemental information about test failures.

2009-11-20T11:22:16Z

Dear Sunburned,

If I write

assertEquals("(1,-1,0)", point.toString());

and change the toString method, I get this (slightly garbled, but
mostly helpful) failure:

org.junit.ComparisonFailure: expected:<[(1,-1,0)]> but was:<[[1, -1, 0]]>
at org.junit.Assert.assertEquals(Assert.java:123)...

This seems to cover the one example you cite. Are there others?

David Saff

On Fri, Nov 20, 2009 at 1:06 PM, Sunburned Surveyor
<sunburned.surveyor@...> wrote:

> I have a question about the options for gathering and displaying
> supplemental information about test failures. Please allow me to
> illustrate my question with a simple example.
>
> I've got a class called Coordinate that represents a point in a 3D
> coordinate system. I've overloaded the toString method of this class
> to write out the X, Y, and Z values of the coordinate in a simple
> String. Now I want to test my toString method using JUnit.
>
> If the Coordinate.toString test method fails (because the string was
> not constructed in the correct format) I would like to provide the
> programmer running the test with a copy of the incorrect string, since
> it provides valuable clues about what is broken with the method under
> test. I typically do this by placing System.err.println statements in
> my test method. However, I realize that using System.err.println isn't
> the most portable or elegant way to provide this information to the
> programmer executing the test.
>
> What I really want is a GUI that allows the programmer to select a
> failed test from a list of failed tests, and then to view supplemental
> test data I provide him/her.
>
> I was thinking I could do this by completing the following tasks:
>
> - Implementing the RunListener interface in JUnit.
> - Create an extension of Exception that could contain a HashMap of
> simple parameters and that would be passed to the constructor of a
> Failure object from within a test.
> - Creating Failure objects in my tests that contain the supplemental
> failrure data.
> - Passing these failure objects to my RunListener implementaiton.
> - Creating a FailureProcessor class that displays a GUI with the
> information in my RunListener implemenation to show the programmer
> executing my tests the supplemental failure data as described above.
> (FailureProcessor would be an interface so that you could do different
> things with the supplemental failure data, not just show it in a GUI.)
>
> Before I begin this work, I wanted to ask if there was a better way to
> do this. I know I could use a logging framework from within my test
> method, but I really want to work within the JUnut framework, not
> outside of it. I also want a nice little GUI, not a log file.
>
> How are other programmers handling instances when they want to display
> supplemental data on failed tests? Does my approach sound reasonable?
>
> The Sunburned Surveyor
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

RE: Gathering and displaying supplemental information about test failures.

2009-11-20T11:20:15Z

Have you looked at how eclipse handles AssertEquals of two Strings.

Not sure if this is a JUnit core functionality or not. But eclipse highlights how the two strings differ.

Just wondering if you've ever seen this functionality... I'll send screen shots if not.

Big Mike

From: junit@... [mailto:junit@...] On Behalf Of Sunburned Surveyor
Sent: Friday, November 20, 2009 12:06 PM
To: junit@...
Cc: Martin Davis; Martin Davis
Subject: [junit] Gathering and displaying supplemental information about test failures.

I have a question about the options for gathering and displaying
supplemental information about test failures. Please allow me to
illustrate my question with a simple example.

I've got a class called Coordinate that represents a point in a 3D
coordinate system. I've overloaded the toString method of this class
to write out the X, Y, and Z values of the coordinate in a simple
String. Now I want to test my toString method using JUnit.

If the Coordinate.toString test method fails (because the string was
not constructed in the correct format) I would like to provide the
programmer running the test with a copy of the incorrect string, since
it provides valuable clues about what is broken with the method under
test. I typically do this by placing System.err.println statements in
my test method. However, I realize that using System.err.println isn't
the most portable or elegant way to provide this information to the
programmer executing the test.

What I really want is a GUI that allows the programmer to select a
failed test from a list of failed tests, and then to view supplemental
test data I provide him/her.

I was thinking I could do this by completing the following tasks:

- Implementing the RunListener interface in JUnit.
- Create an extension of Exception that could contain a HashMap of
simple parameters and that would be passed to the constructor of a
Failure object from within a test.
- Creating Failure objects in my tests that contain the supplemental
failrure data.
- Passing these failure objects to my RunListener implementaiton.
- Creating a FailureProcessor class that displays a GUI with the
information in my RunListener implemenation to show the programmer
executing my tests the supplemental failure data as described above.
(FailureProcessor would be an interface so that you could do different
things with the supplemental failure data, not just show it in a GUI.)

Before I begin this work, I wanted to ask if there was a better way to
do this. I know I could use a logging framework from within my test
method, but I really want to work within the JUnut framework, not
outside of it. I also want a nice little GUI, not a log file.

How are other programmers handling instances when they want to display
supplemental data on failed tests? Does my approach sound reasonable?

The Sunburned Surveyor

[Non-text portions of this message have been removed]

Gathering and displaying supplemental information about test failures.

2009-11-20T10:06:22Z

I have a question about the options for gathering and displaying
supplemental information about test failures. Please allow me to
illustrate my question with a simple example.

I've got a class called Coordinate that represents a point in a 3D
coordinate system. I've overloaded the toString method of this class
to write out the X, Y, and Z values of the coordinate in a simple
String. Now I want to test my toString method using JUnit.

If the Coordinate.toString test method fails (because the string was
not constructed in the correct format) I would like to provide the
programmer running the test with a copy of the incorrect string, since
it provides valuable clues about what is broken with the method under
test. I typically do this by placing System.err.println statements in
my test method. However, I realize that using System.err.println isn't
the most portable or elegant way to provide this information to the
programmer executing the test.

What I really want is a GUI that allows the programmer to select a
failed test from a list of failed tests, and then to view supplemental
test data I provide him/her.

I was thinking I could do this by completing the following tasks:

- Implementing the RunListener interface in JUnit.
- Create an extension of Exception that could contain a HashMap of
simple parameters and that would be passed to the constructor of a
Failure object from within a test.
- Creating Failure objects in my tests that contain the supplemental
failrure data.
- Passing these failure objects to my RunListener implementaiton.
- Creating a FailureProcessor class that displays a GUI with the
information in my RunListener implemenation to show the programmer
executing my tests the supplemental failure data as described above.
(FailureProcessor would be an interface so that you could do different
things with the supplemental failure data, not just show it in a GUI.)

Before I begin this work, I wanted to ask if there was a better way to
do this. I know I could use a logging framework from within my test
method, but I really want to work within the JUnut framework, not
outside of it. I also want a nice little GUI, not a log file.

How are other programmers handling instances when they want to display
supplemental data on failed tests? Does my approach sound reasonable?

The Sunburned Surveyor

Re: Re: Categories in JUnit 4.8

2009-11-20T08:28:30Z

On Thu, Nov 19, 2009 at 11:24 AM, jlinkyh <jl@...> wrote:

>
>
>
>
>
>
>
> --- In junit@..., Kent Beck <kentb@...> wrote:
>>
>> I'm with David on being intrigued by the idea of using arbitrary
>> classes. However, I'm much more interested in experiences than in
>> votes. If someone said, "I went through my code and I could
>> definitely use classes as categories because of X, Y and/or Z," would
>> be compelling to me. Even more compelling is if someone would write
>> the test cases for the new scheme.
>
> Just skimmed through two of my OS projects. 80% of the test cases refer to a single class under test. For about 20% of those CUTs I have more than one test class; those could benefit most since the naming convention doesn't help to show the connection.
>
> The follow-up question is: What would I do with this meta info?

Some (many) people have asked for categories as a way of running only
some of their tests, and our primary goal is to help them. If you
were ever tempted to run only the subset of your tests that shared
some given CUT, this annotation strategy would let you do that. Does
that make your life better?

David Saff

Re: An "after all tests" event

2009-11-20T08:18:49Z

Stolsvik,

What you want (annotate some random class somewhere) isn't currently supported by JUnit (as far as I know).

But what you can do is make a suite. So you pick some random class somewhere and you annotated it with the @RunWith(Suite.class), add the class to the suite with @Suite.SuiteClasses annotation, and then on that class define a @BeforeClass and an @AfterClass method. Those @BeforeClass and @AfterClass methods will get run before and after all the other tests in the suite, respectively.

What you don't get from using this method is the nice convenient use of your IDE to just run all the tests. You have to put them together yourself. Right now in JUnit literally by specifying them, although David has promised some idea for allowing the Suite classes to be generated programatically.

In the mean time if you need something generated programatically, you can make a custom runner that extends Suite and pass an array of the classes that you construct programatically to the Suite constructor.

Even with this solution, you lose the ability to have your IDE just find all tests in the project, but since I need those tests run during the build anyway, that was not a loss for me.

--- In junit@..., "stolsvik" <online@...> wrote:

>
>
>
>
> Folks, thanks for your answers.
>
> --- In junit@..., David Saff <david@> wrote:
> >
> > Stolsvik,
> >
> > You can use @BeforeClass and @AfterClass on a suite to get this
> > behavior, in many circumstances.
>
> Could you tell me directly, or point to some resource, of how I set that up? Your sentence didn't immediately ring understanding with me.
>
> Basically, what I am requesting, is that I'd like to annotate some random method on some random class anywhere in the project with some annotation like @AfterTestRun - so that when I run either one single test class, or a package, or the entire project, with the Eclipse runner, this specific method would be invoked after the requested test(s) are run.
>
> So: How do I accomplish that with "@AfterClass on a suite"?
>
> Do I have to annotate all my test classes with some specific Suite-annotation? That would be doable. However, it would be just as awkward as I do with my logging init: Having to do some boilerplate bs on every single test class - only that forgetting to do this would not give me any immediate indication, as with the logging init; For the logging init, I immediately see if I have forgotten those three boilerplate lines ("static { TestHelper.setupLogging() }"), because log4j spews out some noise on Std.err if any part of the system starts to log on an uninitialized logging system.
>
> >
> > David Saff
> >
> > On Thu, Nov 19, 2009 at 7:09 AM, stolsvik <online@> wrote:
> > > I need some way to run a certain test after all other tests.
> > >
> > > I run my tests in Eclipse, and do not yet have a build system outside of it. I understand that had I done this programmatically, I could have attached a RunListener, implementing the testRunFinished(Result) method.
> > >
> > > Is there any way to achieve a similar effect within the JUnit runner in Eclipse?
> > >
> > > Rationale: I have actual unit tests, then bigger tests, then full-fledged "integration tests" or whatever one call them, where I take up my entire application-local server thingy and process a bunch of items, letting JUnit check after the dust settles whether the items was processed correctly. However, the internals are multi threaded, and also, this being a Swing app, lots of Runnables are sent over to the EDT. This is all good.
> > >
> > > However, I've sometimes seen that a thread crashes (uncaught exception), without this being noticed by any of the tests running (I've e.g. seen a ConcurrentModificationException take down the EDT, without having a clue as to why I got that - and the next event posted just brings up a new EDT, so I get a green bar, but with an uncaught error). What I'd want to achieve, is to install an uncaught exception handler, and then, as a final step to ALL permutations of tests (e.g. both for a single unit test, and also for the project-wide test run), run a test that checks whether any thread-killing exception was caught by the handler, and if so, fails the run. (I'd throw the exception that was caught, and in any case also log-dump any and all such exceptions).
> > >
> > > (As a very similar question, I'd also like to have a "before any tests" thing, like the RunListener.testRunStarted(Description) method, where I could set up logging. Now I do that by letting ALL tests have a static block that calls a specific static method which has a static boolean that tells whether the log system already is set up. This also works just nicely, but is a hassle nevertheless. However, it is not possible, as I see it, to solve the above question in the same manner.)
> > >
> > >
> > >
> > > ------------------------------------
> > >
> > > Yahoo! Groups Links
> > >
> > >
> > >
> > >
> >
>

Re: Re: Categories in JUnit 4.8

2009-11-20T05:24:49Z

All,

I can see that a short code snippet has been misleading about what our
proposal makes easy, possible, and impossible. I'll try to post a
longer feature summary soon.

David Saff

On Fri, Nov 20, 2009 at 3:24 AM, Bogdan
<bogdan.mocanu.notifications@...> wrote:

> I agree with Georg. I prefer an annotation where I can specify the subcategories where a test fits. Something like:
>
> @Category( "database.mysql.local" )
> public class MySQLDBTest {
> }
>
> Cons:
> - allows for typos to happen
> - no support from the IDE (this could be improvode if you define a list of constants in a class, such as DATABASE, DATABASE_MYSQL, DATABASE_MYSQL_LOCAL, etc)
>
> Pros:
> - allows for regex to be used when defining the Suites
> - allows for hierarchies of categories.
>
> The hierarchy of categories, I think, is the hardest part to implement with interfaces or base classes.
>
> Regards,
> Bogdan
>
> --- In junit@..., "Georg" <le_garcon_enerve@...> wrote:
>>
>> Hi!
>>
>> Something like this is nice to have, but, I don't quite like the idea of the interface. I see the danger of having to introduce an unmanagable number of interfaces lurking around the corner. That the compiler does some checking is certainly appreciable, though.
>> I this sense, I like the extension to Suite I proposed better (see http://github.com/KentBeck/junit), as it allows to name tests in a certain way and then use regexps to run them as groups. With very little effort, all tests (in a package), or class names matching some convention, can be run easily (it takes a bit of consistency in naming tests, though).
>>
>>
>> Cheers,
>> Georg
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Re: An "after all tests" event

2009-11-20T05:18:09Z

Folks, thanks for your answers.

--- In junit@..., David Saff <david@...> wrote:
>
> Stolsvik,
>
> You can use @BeforeClass and @AfterClass on a suite to get this
> behavior, in many circumstances.

Could you tell me directly, or point to some resource, of how I set that up? Your sentence didn't immediately ring understanding with me.

Basically, what I am requesting, is that I'd like to annotate some random method on some random class anywhere in the project with some annotation like @AfterTestRun - so that when I run either one single test class, or a package, or the entire project, with the Eclipse runner, this specific method would be invoked after the requested test(s) are run.

So: How do I accomplish that with "@AfterClass on a suite"?

Do I have to annotate all my test classes with some specific Suite-annotation? That would be doable. However, it would be just as awkward as I do with my logging init: Having to do some boilerplate bs on every single test class - only that forgetting to do this would not give me any immediate indication, as with the logging init; For the logging init, I immediately see if I have forgotten those three boilerplate lines ("static { TestHelper.setupLogging() }"), because log4j spews out some noise on Std.err if any part of the system starts to log on an uninitialized logging system.

>
> David Saff
>
> On Thu, Nov 19, 2009 at 7:09 AM, stolsvik <online@...> wrote:
> > I need some way to run a certain test after all other tests.
> >
> > I run my tests in Eclipse, and do not yet have a build system outside of it. I understand that had I done this programmatically, I could have attached a RunListener, implementing the testRunFinished(Result) method.
> >
> > Is there any way to achieve a similar effect within the JUnit runner in Eclipse?
> >
> > Rationale: I have actual unit tests, then bigger tests, then full-fledged "integration tests" or whatever one call them, where I take up my entire application-local server thingy and process a bunch of items, letting JUnit check after the dust settles whether the items was processed correctly. However, the internals are multi threaded, and also, this being a Swing app, lots of Runnables are sent over to the EDT. This is all good.
> >
> > However, I've sometimes seen that a thread crashes (uncaught exception), without this being noticed by any of the tests running (I've e.g. seen a ConcurrentModificationException take down the EDT, without having a clue as to why I got that - and the next event posted just brings up a new EDT, so I get a green bar, but with an uncaught error). What I'd want to achieve, is to install an uncaught exception handler, and then, as a final step to ALL permutations of tests (e.g. both for a single unit test, and also for the project-wide test run), run a test that checks whether any thread-killing exception was caught by the handler, and if so, fails the run. (I'd throw the exception that was caught, and in any case also log-dump any and all such exceptions).
> >
> > (As a very similar question, I'd also like to have a "before any tests" thing, like the RunListener.testRunStarted(Description) method, where I could set up logging. Now I do that by letting ALL tests have a static block that calls a specific static method which has a static boolean that tells whether the log system already is set up. This also works just nicely, but is a hassle nevertheless. However, it is not possible, as I see it, to solve the above question in the same manner.)
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>

Re: Categories in JUnit 4.8

2009-11-20T00:24:33Z

I agree with Georg. I prefer an annotation where I can specify the subcategories where a test fits. Something like:

@Category( "database.mysql.local" )
public class MySQLDBTest {
}

Cons:
- allows for typos to happen
- no support from the IDE (this could be improvode if you define a list of constants in a class, such as DATABASE, DATABASE_MYSQL, DATABASE_MYSQL_LOCAL, etc)

Pros:
- allows for regex to be used when defining the Suites
- allows for hierarchies of categories.

The hierarchy of categories, I think, is the hardest part to implement with interfaces or base classes.

Regards,
Bogdan

--- In junit@..., "Georg" <le_garcon_enerve@...> wrote:

>
> Hi!
>
> Something like this is nice to have, but, I don't quite like the idea of the interface. I see the danger of having to introduce an unmanagable number of interfaces lurking around the corner. That the compiler does some checking is certainly appreciable, though.
> I this sense, I like the extension to Suite I proposed better (see http://github.com/KentBeck/junit), as it allows to name tests in a certain way and then use regexps to run them as groups. With very little effort, all tests (in a package), or class names matching some convention, can be run easily (it takes a bit of consistency in naming tests, though).
>
>
> Cheers,
> Georg
>

Re: An "after all tests" event

2009-11-19T11:48:58Z

On Thu, Nov 19, 2009 at 11:44 AM, David Saff <david@...> wrote:

>
>
> As synonyms for BeforeClass and AfterClass?
>

No, as wrappers of an entire test suite:

BeforeSuite
BeforeClass
...
AfterClass
BeforeClass
AfterClass
...
AfterSuite

--
***Cédric
*

[Non-text portions of this message have been removed]

Re: An "after all tests" event

2009-11-19T11:45:18Z

Cedric,

That is what you get out of @BeforeClass and @AfterClass when you define them on the class with the @RunWith(Suite.class) annotation. It is the same effect.

Yishai

--- In junit@..., CÃ©dric Beust â" <cbeust@...> wrote:

>
> On Thu, Nov 19, 2009 at 11:24 AM, David Saff <david@...> wrote:
>
> >
> >
> > Stolsvik,
> >
> > You can use @BeforeClass and @AfterClass on a suite to get this
> > behavior, in many circumstances.
> >
>
> @BeforeSuite and @AfterSuite would be nice :-)
>
> --
> ***Cédric
> *
>
>
> [Non-text portions of this message have been removed]
>

Re: An "after all tests" event

2009-11-19T11:44:12Z

As synonyms for BeforeClass and AfterClass?

David Saff

2009/11/19 Cédric Beust ♔ <cbeust@...>:

> On Thu, Nov 19, 2009 at 11:24 AM, David Saff <david@...> wrote:
>
>>
>>
>> Stolsvik,
>>
>> You can use @BeforeClass and @AfterClass on a suite to get this
>> behavior, in many circumstances.
>>
>
> @BeforeSuite and @AfterSuite would be nice :-)
>
> --
> ***Cédric
> *
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Re: An "after all tests" event

2009-11-19T11:34:25Z

On Thu, Nov 19, 2009 at 11:24 AM, David Saff <david@...> wrote:

>
>
> Stolsvik,
>
> You can use @BeforeClass and @AfterClass on a suite to get this
> behavior, in many circumstances.
>

@BeforeSuite and @AfterSuite would be nice :-)

--
***Cédric
*

[Non-text portions of this message have been removed]

Re: An "after all tests" event

2009-11-19T11:24:25Z

Stolsvik,

You can use @BeforeClass and @AfterClass on a suite to get this
behavior, in many circumstances.

David Saff

On Thu, Nov 19, 2009 at 7:09 AM, stolsvik <online@...> wrote:

> I need some way to run a certain test after all other tests.
>
> I run my tests in Eclipse, and do not yet have a build system outside of it. I understand that had I done this programmatically, I could have attached a RunListener, implementing the testRunFinished(Result) method.
>
> Is there any way to achieve a similar effect within the JUnit runner in Eclipse?
>
> Rationale: I have actual unit tests, then bigger tests, then full-fledged "integration tests" or whatever one call them, where I take up my entire application-local server thingy and process a bunch of items, letting JUnit check after the dust settles whether the items was processed correctly. However, the internals are multi threaded, and also, this being a Swing app, lots of Runnables are sent over to the EDT. This is all good.
>
> However, I've sometimes seen that a thread crashes (uncaught exception), without this being noticed by any of the tests running (I've e.g. seen a ConcurrentModificationException take down the EDT, without having a clue as to why I got that - and the next event posted just brings up a new EDT, so I get a green bar, but with an uncaught error). What I'd want to achieve, is to install an uncaught exception handler, and then, as a final step to ALL permutations of tests (e.g. both for a single unit test, and also for the project-wide test run), run a test that checks whether any thread-killing exception was caught by the handler, and if so, fails the run. (I'd throw the exception that was caught, and in any case also log-dump any and all such exceptions).
>
> (As a very similar question, I'd also like to have a "before any tests" thing, like the RunListener.testRunStarted(Description) method, where I could set up logging. Now I do that by letting ALL tests have a static block that calls a specific static method which has a static boolean that tells whether the log system already is set up. This also works just nicely, but is a hassle nevertheless. However, it is not possible, as I see it, to solve the above question in the same manner.)
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Re: Categories in JUnit 4.8

2009-11-19T10:21:03Z

Well, of course the feature already has my vote ;), but in terms of concrete code, sure.

I have an DAOBase for the general persistence classes, and then specific DAOBase implementations for their respective DataObject.

So I could certainly see categorizing all general persistence tests to be about DAOBase and specific tests to be categorized according to the specific subclass that they exercise.

In addition, in certain hairy parts of the data model, the DAOBase subclass has itself subclasses, allowing me to categorize tests according to that area.

The advantage of such categories is that since all of the relevant tests on these are slow (they actually hit the database), I could test a specific area I'm working on before check-in in a reasonable time, and then rely on the build server to run everything after check in just in case, without a long wait (currently the tests take over an hour on the Build server).

Of course I could make my own arbitrary categories, whether interfaces or Strings, but it is just a parallel duplicate system. Or I could implement the JUnit interface on DAOBase, introducing a JUnit dependency I would rather not have on production code, and get the same effect.

In terms of a test case, I'd be happy to put one together, but I'm just not fully sure what you are looking for. The real value would seem to be shown in having a real class hierarchy that fits well, more than in just demonstrating it with a hypothetical test hierarchy. So the question is what characteristics would such a test have that would be compelling for you?

--- In junit@..., Kent Beck <kentb@...> wrote:

>
> I'm with David on being intrigued by the idea of using arbitrary classes. However, I'm much more interested in experiences than in votes. If someone said, "I went through my code and I could definitely use classes as categories because of X, Y and/or Z," would be compelling to me. Even more compelling is if someone would write the test cases for the new scheme.
>
> Regards,
>
> Kent
>
> On Nov 19, 2009, at 7:33 AM, David Saff wrote:
>
> > On Wed, Nov 18, 2009 at 7:46 PM, yme0987654321 <yme0987654321@...> wrote:
> > > The decision to use types to define categories is definitely surprising to see, so that has to count against it.
> > >
> > > However, after the initial shock wears off, I definitely see the power in it, using Java inheritance and static checking rather than free form strings and some arbitrary hierarchy delimiter in the string.
> > >
> > > However, I think the decision to use a specific new interface as the super type is not a good idea. It would seem you gain the same power by allowing any Class<?>. The advantage of doing this is it would allow real classes within the project to act as Categories, so that if, for example, I am testing a DAO framework, I can categorize individual tests with the specific DAO subclass they are targeting, and then run a category based on the DAO supertype. Much nicer than just calling it all SlowTests. In practice that won't happen often if a custom interface is required because it will introduce a dependency on a JUnit type in the production code.
> >
> > Huh. The first time through, I thought "that's ridiculous". The
> > second time, I wondered if there's not something to this...
> >
> > Totally musing out loud, but you could rename the annotation as
> > TestOf, so that you'd have
> >
> > @TestOf(CustomStreamingPlatform.class)
> >
> > Hmm...
> >
> > > I'm also curious if you considered using an annotation as the category? That is having the category take a Class<?> that represents an annotation? You could support meta-annotations to allow a hierarchy of categories as well as supporting any subclasses or implementations of types so annotated. Coding for that would be nasty business, but it wouldn't be complicated from a client point of view, and would allow the same flexibility of inheritance without the dependency on a JUnit type.
> >
> > I wanted to stay away from meta-annotations unless the reasons were
> > _really_ compelling.
> >
> > > I would prefer just an arbitrary Class<?>, though. Inheritance is well understood by any Java programmer (unlike meta-annotations) and opens up a synergy between the category hierarchy and the code hierarchy.
> >
> > Can I get another vote on this extension idea?
> >
> > David
> >
>
>
>
> [Non-text portions of this message have been removed]
>

RE: Categories in JUnit 4.8

2009-11-19T09:45:30Z

>To my way of thinking, categories should be handled by the IDE--they are meta-data
>that belongs with the test runner, not with the tests themselves. Since IDEs don't
>provide a good solution to running subsets of tests, we can add some value by *an*
>implementation.

I am sure that you guys spent a lot of time debating this point back in the day, but I would like to point out that this is one of the hidden drawbacks of not providing your own graphical test runner. With your own test runner, users could evaluate new features quickly and easily and then apply pressure to their IDE's to provide support, thus expanding your user base.

Also, it does help those of us who prefer Emacs over Eclipse feel a little less left out in the cold :-).

Rocco

-----Original Message-----
From: junit@... [mailto:junit@...] On Behalf Of Kent Beck
Sent: Tuesday, November 17, 2009 4:44 PM
To: junit@...
Subject: Re: [junit] Categories in JUnit 4.8

Malte,

Thank you for the question. We went back and forth on this. The two positive features of using interfaces as "tags" is that the compiler can check them and inheritance works. The need for otherwise useless interfaces is the cost of this decision.

To my way of thinking, categories should be handled by the IDE--they are meta-data that belongs with the test runner, not with the tests themselves. Since IDEs don't provide a good solution to running subsets of tests, we can add some value by *an* implementation.

That said, we are open to changing to strings if experience shows that the interface-based categories don't work in practice.

Regards,

Kent

On Nov 17, 2009, at 1:34 PM, Malte Finsterwalder wrote:

> Hi David,
>
> Why are the categories interfaces? I don't really like, that I have to
> create empty interfaces. Would it be easier to use strings? Even
> though this opens up the possibilites for typos.
>
> Greetings,
> Malte
>

[Non-text portions of this message have been removed]

------------------------------------

Yahoo! Groups Links

Re: Categories in JUnit 4.8

2009-11-19T08:24:51Z

--- In junit@..., Kent Beck <kentb@...> wrote:
>
> I'm with David on being intrigued by the idea of using arbitrary
> classes. However, I'm much more interested in experiences than in
> votes. If someone said, "I went through my code and I could
> definitely use classes as categories because of X, Y and/or Z," would
> be compelling to me. Even more compelling is if someone would write
> the test cases for the new scheme.

Just skimmed through two of my OS projects. 80% of the test cases refer to a single class under test. For about 20% of those CUTs I have more than one test class; those could benefit most since the naming convention doesn't help to show the connection.

The follow-up question is: What would I do with this meta info?

Johannes

Re: Re: Categories in JUnit 4.8

2009-11-19T08:10:56Z

2009/11/19 Kent Beck <kentb@...>:
> I'm with David on being intrigued by the idea of using arbitrary classes. However, I'm much more interested in experiences than in votes. If someone said, "I went through my code and I could definitely use classes as categories because of X, Y and/or Z," would be compelling to me. Even more compelling is if someone would write the test cases for the new scheme.

In my corrent codebase I could think of something like this:

Some of our tests use a class "PersistentTestSupport", that helps in
setting up tests that use a database.
I could use the same class as a marker for tests that need persistence
and are as such slower.

Greetings,
Malte

Re: Re: Categories in JUnit 4.8

2009-11-19T08:02:03Z

Georg,

Can you recheck that URL? The one you sent just points to the junit
home page on github.

David Saff

On Wed, Nov 18, 2009 at 9:32 PM, Georg <le_garcon_enerve@...> wrote:

> Hi!
>
> Something like this is nice to have, but, I don't quite like the idea of the interface. I see the danger of having to introduce an unmanagable number of interfaces lurking around the corner. That the compiler does some checking is certainly appreciable, though.
> I this sense, I like the extension to Suite I proposed better (see http://github.com/KentBeck/junit), as it allows to name tests in a certain way and then use regexps to run them as groups. With very little effort, all tests (in a package), or class names matching some convention, can be run easily (it takes a bit of consistency in naming tests, though).
>
>
> Cheers,
> Georg
>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Re: Re: Categories in JUnit 4.8

2009-11-19T07:38:51Z

I'm with David on being intrigued by the idea of using arbitrary classes. However, I'm much more interested in experiences than in votes. If someone said, "I went through my code and I could definitely use classes as categories because of X, Y and/or Z," would be compelling to me. Even more compelling is if someone would write the test cases for the new scheme.

Regards,

Kent

On Nov 19, 2009, at 7:33 AM, David Saff wrote:

> On Wed, Nov 18, 2009 at 7:46 PM, yme0987654321 <yme0987654321@...> wrote:
> > The decision to use types to define categories is definitely surprising to see, so that has to count against it.
> >
> > However, after the initial shock wears off, I definitely see the power in it, using Java inheritance and static checking rather than free form strings and some arbitrary hierarchy delimiter in the string.
> >
> > However, I think the decision to use a specific new interface as the super type is not a good idea. It would seem you gain the same power by allowing any Class<?>. The advantage of doing this is it would allow real classes within the project to act as Categories, so that if, for example, I am testing a DAO framework, I can categorize individual tests with the specific DAO subclass they are targeting, and then run a category based on the DAO supertype. Much nicer than just calling it all SlowTests. In practice that won't happen often if a custom interface is required because it will introduce a dependency on a JUnit type in the production code.
>
> Huh. The first time through, I thought "that's ridiculous". The
> second time, I wondered if there's not something to this...
>
> Totally musing out loud, but you could rename the annotation as
> TestOf, so that you'd have
>
> @TestOf(CustomStreamingPlatform.class)
>
> Hmm...
>
> > I'm also curious if you considered using an annotation as the category? That is having the category take a Class<?> that represents an annotation? You could support meta-annotations to allow a hierarchy of categories as well as supporting any subclasses or implementations of types so annotated. Coding for that would be nasty business, but it wouldn't be complicated from a client point of view, and would allow the same flexibility of inheritance without the dependency on a JUnit type.
>
> I wanted to stay away from meta-annotations unless the reasons were
> _really_ compelling.
>
> > I would prefer just an arbitrary Class<?>, though. Inheritance is well understood by any Java programmer (unlike meta-annotations) and opens up a synergy between the category hierarchy and the code hierarchy.
>
> Can I get another vote on this extension idea?
>
> David
>

[Non-text portions of this message have been removed]

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/junit/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/junit/join
(Yahoo! ID required)

<*> To change settings via email:
junit-digest@...
junit-fullfeatured@...

<*> To unsubscribe from this group, send an email to:
junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/

Re: Re: Categories in JUnit 4.8

2009-11-19T07:33:13Z

On Wed, Nov 18, 2009 at 7:46 PM, yme0987654321 <yme0987654321@...> wrote:
> The decision to use types to define categories is definitely surprising to see, so that has to count against it.
>
> However, after the initial shock wears off, I definitely see the power in it, using Java inheritance and static checking rather than free form strings and some arbitrary hierarchy delimiter in the string.
>
> However, I think the decision to use a specific new interface as the super type is not a good idea. It would seem you gain the same power by allowing any Class<?>. The advantage of doing this is it would allow real classes within the project to act as Categories, so that if, for example, I am testing a DAO framework, I can categorize individual tests with the specific DAO subclass they are targeting, and then run a category based on the DAO supertype. Much nicer than just calling it all SlowTests. In practice that won't happen often if a custom interface is required because it will introduce a dependency on a JUnit type in the production code.

Huh. The first time through, I thought "that's ridiculous". The
second time, I wondered if there's not something to this...

Totally musing out loud, but you could rename the annotation as
TestOf, so that you'd have

@TestOf(CustomStreamingPlatform.class)

Hmm...

> I'm also curious if you considered using an annotation as the category? That is having the category take a Class<?> that represents an annotation? You could support meta-annotations to allow a hierarchy of categories as well as supporting any subclasses or implementations of types so annotated. Coding for that would be nasty business, but it wouldn't be complicated from a client point of view, and would allow the same flexibility of inheritance without the dependency on a JUnit type.

I wanted to stay away from meta-annotations unless the reasons were
_really_ compelling.

> I would prefer just an arbitrary Class<?>, though. Inheritance is well understood by any Java programmer (unlike meta-annotations) and opens up a synergy between the category hierarchy and the code hierarchy.

Can I get another vote on this extension idea?

David

An "after all tests" event

2009-11-19T04:09:37Z

I need some way to run a certain test after all other tests.

I run my tests in Eclipse, and do not yet have a build system outside of it. I understand that had I done this programmatically, I could have attached a RunListener, implementing the testRunFinished(Result) method.

Is there any way to achieve a similar effect within the JUnit runner in Eclipse?

Rationale: I have actual unit tests, then bigger tests, then full-fledged "integration tests" or whatever one call them, where I take up my entire application-local server thingy and process a bunch of items, letting JUnit check after the dust settles whether the items was processed correctly. However, the internals are multi threaded, and also, this being a Swing app, lots of Runnables are sent over to the EDT. This is all good.

However, I've sometimes seen that a thread crashes (uncaught exception), without this being noticed by any of the tests running (I've e.g. seen a ConcurrentModificationException take down the EDT, without having a clue as to why I got that - and the next event posted just brings up a new EDT, so I get a green bar, but with an uncaught error). What I'd want to achieve, is to install an uncaught exception handler, and then, as a final step to ALL permutations of tests (e.g. both for a single unit test, and also for the project-wide test run), run a test that checks whether any thread-killing exception was caught by the handler, and if so, fails the run. (I'd throw the exception that was caught, and in any case also log-dump any and all such exceptions).

(As a very similar question, I'd also like to have a "before any tests" thing, like the RunListener.testRunStarted(Description) method, where I could set up logging. Now I do that by letting ALL tests have a static block that calls a specific static method which has a static boolean that tells whether the log system already is set up. This also works just nicely, but is a hassle nevertheless. However, it is not possible, as I see it, to solve the above question in the same manner.)

Re: Any way to multi-thread tests?

2009-11-19T00:18:13Z

Just to give you another idea to think about:

Is it possible to simulate time and speed up the tests that way?

Just a simple idea: Replace new Date() with something like
TimeServer.now() and make that configurable.
Or can you explicitly call the mark-logik, so it's not time dependent?
Or alike?

Greetings,
Malte

2009/11/19 Pat Farrell <pfarrell@...>:

> I've got a suite of tests for assorted classes, and its all working
> fine. except it takes too long. By clock, not by resources.
>
> One of the classes I'm testing is a working set cache, and in order to
> test it, you have to wait until a cycle or two have gone by to let the
> mark logic work.
>
> This takes a long time, during which the main JUnit test is just doing a
> Thread.sleep(bignumber);
>
> What I'd love to do is start the cache testing code, and then continue
> with other tests that are not time dependant while the cache test is
> doing the big sleep();
>
> It would not be hard to use Java's concurancy tests to resync things,
> but I don't have any idea if, or how, to get subsequent tests to fire off.
>
> Is this possible?
>
> Thanks
> Pat
> --
> Pat Farrell
> http://www.pfarrell.com/
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

Any way to multi-thread tests?

2009-11-18T20:49:26Z

I've got a suite of tests for assorted classes, and its all working
fine. except it takes too long. By clock, not by resources.

One of the classes I'm testing is a working set cache, and in order to
test it, you have to wait until a cycle or two have gone by to let the
mark logic work.

This takes a long time, during which the main JUnit test is just doing a
Thread.sleep(bignumber);

What I'd love to do is start the cache testing code, and then continue
with other tests that are not time dependant while the cache test is
doing the big sleep();

It would not be hard to use Java's concurancy tests to resync things,
but I don't have any idea if, or how, to get subsequent tests to fire off.

Is this possible?

Thanks
Pat
--
Pat Farrell
http://www.pfarrell.com/