How to build a test suite automatically in Junit4?

[ju_10101]

Hi,

I've just updated to 4.7 from 3.8.2.  I have a large project with a lot of classes and packages, but I also annotate tests for fast and slow.  I create my FastSuite to find all tests classes annotated with SmallTest.  That way, these can be run first in the continuous environment, and the slow ones run later, on a different machine, so I don't backup the queue, etc.  

It seems the way to make suites in 4.7 is to explicitly list each classname as an annotation to your suite class, but I don't want to do that because a) I have a lot, and b) I don't want to not run new tests because someone didn't go update the manual list of test classes somewhere.  

My old way doesn't work anymore.  Can anyone tell me how to accomplish the same in junit 4.7??


My code in 3.8.2 was (sorry for the formatting!):

    public static Test suite() {
        final String smallTestName = SmallTest.class.getCanonicalName();
        TestSuite suite = new TestSuite();

        Enumeration testClasses = new LoadingTestCollector().collectTests();
        while (testClasses.hasMoreElements()) {
            final String className = testClasses.nextElement().toString();

            try {
                Annotation[] annotations = Class.forName(className).getAnnotations();
                for (Annotation a : annotations) {
                    if (a.annotationType().getCanonicalName().equals(smallTestName)) {
                        System.out.println("HELLOOO- working on class " + className);
                        suite.addTestSuite(Class.forName(className));
                    }
                }
            }    catch (ClassNotFoundException e) {
                System.err.println("Error: Couldn't deal with " + className);
            }
        }
        return suite;
    }


The compiler error is: addTestSuite(java.lang.Class<? extends junit.framework.TestCase>) in junit.framework.TestSuite cannot be applied to (java.lang.Class<capture#337 of ?>)
    [javac]                         suite.addTestSuite(Class.forName(className));

[David Saff]

ju_10101,

Are you able to use pre-release code?  HEAD at github includes a
Category system, which you could combine with Link's cpsuite runner to
get what you want.

   David Saff

On Thu, Oct 22, 2009 at 7:31 PM, ju_10101 <ju_10101@...> wrote:

> Hi,
>
> I've just updated to 4.7 from 3.8.2.  I have a large project with a lot of classes and packages, but I also annotate tests for fast and slow.  I create my FastSuite to find all tests classes annotated with SmallTest.  That way, these can be run first in the continuous environment, and the slow ones run later, on a different machine, so I don't backup the queue, etc.
>
> It seems the way to make suites in 4.7 is to explicitly list each classname as an annotation to your suite class, but I don't want to do that because a) I have a lot, and b) I don't want to not run new tests because someone didn't go update the manual list of test classes somewhere.
>
> My old way doesn't work anymore.  Can anyone tell me how to accomplish the same in junit 4.7??
>
>
> My code in 3.8.2 was (sorry for the formatting!):
>
>    public static Test suite() {
>        final String smallTestName = SmallTest.class.getCanonicalName();
>        TestSuite suite = new TestSuite();
>
>        Enumeration testClasses = new LoadingTestCollector().collectTests();
>        while (testClasses.hasMoreElements()) {
>            final String className = testClasses.nextElement().toString();
>
>            try {
>                Annotation[] annotations = Class.forName(className).getAnnotations();
>                for (Annotation a : annotations) {
>                    if (a.annotationType().getCanonicalName().equals(smallTestName)) {
>                        System.out.println("HELLOOO- working on class " + className);
>                        suite.addTestSuite(Class.forName(className));
>                    }
>                }
>            }    catch (ClassNotFoundException e) {
>                System.err.println("Error: Couldn't deal with " + className);
>            }
>        }
>        return suite;
>    }
>
>
> The compiler error is: addTestSuite(java.lang.Class<? extends junit.framework.TestCase>) in junit.framework.TestSuite cannot be applied to (java.lang.Class<capture#337 of ?>)
>    [javac]                         suite.addTestSuite(Class.forName(className));
>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

[ju_10101]

I just got folks in the office to agree to upgrade - might be a harder battle to get a nightly build, but I"ll take a look.

With the 4.7 release then, there's no way to build a dynamic test suite?  Any estimate when the next release might be?  I might just drop back down to 3.8 in the meantime.

Thanks!


--- In junit@..., David Saff <david@...> wrote:

>
> ju_10101,
>
> Are you able to use pre-release code?  HEAD at github includes a
> Category system, which you could combine with Link's cpsuite runner to
> get what you want.
>
>    David Saff
>
> On Thu, Oct 22, 2009 at 7:31 PM, ju_10101 <ju_10101@...> wrote:
> > Hi,
> >
> > I've just updated to 4.7 from 3.8.2.  I have a large project with a lot of classes and packages, but I also annotate tests for fast and slow.  I create my FastSuite to find all tests classes annotated with SmallTest.  That way, these can be run first in the continuous environment, and the slow ones run later, on a different machine, so I don't backup the queue, etc.
> >
> > It seems the way to make suites in 4.7 is to explicitly list each classname as an annotation to your suite class, but I don't want to do that because a) I have a lot, and b) I don't want to not run new tests because someone didn't go update the manual list of test classes somewhere.
> >
> > My old way doesn't work anymore.  Can anyone tell me how to accomplish the same in junit 4.7??
> >
> >
> > My code in 3.8.2 was (sorry for the formatting!):
> >
> >    public static Test suite() {
> >        final String smallTestName = SmallTest.class.getCanonicalName();
> >        TestSuite suite = new TestSuite();
> >
> >        Enumeration testClasses = new LoadingTestCollector().collectTests();
> >        while (testClasses.hasMoreElements()) {
> >            final String className = testClasses.nextElement().toString();
> >
> >            try {
> >                Annotation[] annotations = Class.forName(className).getAnnotations();
> >                for (Annotation a : annotations) {
> >                    if (a.annotationType().getCanonicalName().equals(smallTestName)) {
> >                        System.out.println("HELLOOO- working on class " + className);
> >                        suite.addTestSuite(Class.forName(className));
> >                    }
> >                }
> >            }    catch (ClassNotFoundException e) {
> >                System.err.println("Error: Couldn't deal with " + className);
> >            }
> >        }
> >        return suite;
> >    }
> >
> >
> > The compiler error is: addTestSuite(java.lang.Class<? extends junit.framework.TestCase>) in junit.framework.TestSuite cannot be applied to (java.lang.Class<capture#337 of ?>)
> >    [javac]                         suite.addTestSuite(Class.forName(className));
> >
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>

[David Saff]

ju_10101,

Ask tomorrow for an ETA on the next release.

   David

On Fri, Oct 23, 2009 at 11:50 AM, ju_10101 <ju_10101@...> wrote:

> I just got folks in the office to agree to upgrade - might be a harder battle to get a nightly build, but I"ll take a look.
>
> With the 4.7 release then, there's no way to build a dynamic test suite?  Any estimate when the next release might be?  I might just drop back down to 3.8 in the meantime.
>
> Thanks!
>
>
> --- In junit@..., David Saff <david@...> wrote:
>>
>> ju_10101,
>>
>> Are you able to use pre-release code?  HEAD at github includes a
>> Category system, which you could combine with Link's cpsuite runner to
>> get what you want.
>>
>>    David Saff
>>
>> On Thu, Oct 22, 2009 at 7:31 PM, ju_10101 <ju_10101@...> wrote:
>> > Hi,
>> >
>> > I've just updated to 4.7 from 3.8.2.  I have a large project with a lot of classes and packages, but I also annotate tests for fast and slow.  I create my FastSuite to find all tests classes annotated with SmallTest.  That way, these can be run first in the continuous environment, and the slow ones run later, on a different machine, so I don't backup the queue, etc.
>> >
>> > It seems the way to make suites in 4.7 is to explicitly list each classname as an annotation to your suite class, but I don't want to do that because a) I have a lot, and b) I don't want to not run new tests because someone didn't go update the manual list of test classes somewhere.
>> >
>> > My old way doesn't work anymore.  Can anyone tell me how to accomplish the same in junit 4.7??
>> >
>> >
>> > My code in 3.8.2 was (sorry for the formatting!):
>> >
>> >    public static Test suite() {
>> >        final String smallTestName = SmallTest.class.getCanonicalName();
>> >        TestSuite suite = new TestSuite();
>> >
>> >        Enumeration testClasses = new LoadingTestCollector().collectTests();
>> >        while (testClasses.hasMoreElements()) {
>> >            final String className = testClasses.nextElement().toString();
>> >
>> >            try {
>> >                Annotation[] annotations = Class.forName(className).getAnnotations();
>> >                for (Annotation a : annotations) {
>> >                    if (a.annotationType().getCanonicalName().equals(smallTestName)) {
>> >                        System.out.println("HELLOOO- working on class " + className);
>> >                        suite.addTestSuite(Class.forName(className));
>> >                    }
>> >                }
>> >            }    catch (ClassNotFoundException e) {
>> >                System.err.println("Error: Couldn't deal with " + className);
>> >            }
>> >        }
>> >        return suite;
>> >    }
>> >
>> >
>> > The compiler error is: addTestSuite(java.lang.Class<? extends junit.framework.TestCase>) in junit.framework.TestSuite cannot be applied to (java.lang.Class<capture#337 of ?>)
>> >    [javac]                         suite.addTestSuite(Class.forName(className));
>> >
>> >
>> >
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

[yme0987654321]

The way I do this currently is with a custom runner like this:

public class CustomRunner extends Suite {

    public CustomRunner(Class<?> setupClass) throws InitializationError {
        super(setupClass, staticMethodWhichGetsTheClassesAsAnArrayForTheSuite());
    }
}

And then Run an arbitrary class annotated with that Runner using @RunWith (I did it as a nested class so it was all right there, just be sure to declare it static). In that arbitrary class, you can do @BeforeClass and @AfterClass that might do interesting things.

I put in a request (http://sourceforge.net/tracker/index.php?func=detail&aid=2786193&group%5Fid=15278&atid=365278) with my preferred way of resolving this, but I don't think anything was done on it.

By the way, I'd be happy to code up the implementation and submit it as a patch if David/Kent would agree on the approach (a new runner, changes to the current runner, whatever. I don't really care, I would just like to see it as part of the framework).

I think the above workaround is reasonable enough (although not really that obvious) that I can certainly understand not prioritizing addressing this.

Yishai

--- In junit@..., David Saff <david@...> wrote:

>
> ju_10101,
>
> Ask tomorrow for an ETA on the next release.
>
>    David
>
> On Fri, Oct 23, 2009 at 11:50 AM, ju_10101 <ju_10101@...> wrote:
> > I just got folks in the office to agree to upgrade - might be a harder battle to get a nightly build, but I"ll take a look.
> >
> > With the 4.7 release then, there's no way to build a dynamic test suite?  Any estimate when the next release might be?  I might just drop back down to 3.8 in the meantime.
> >
> > Thanks!
> >
> >
> > --- In junit@..., David Saff <david@> wrote:
> >>
> >> ju_10101,
> >>
> >> Are you able to use pre-release code?  HEAD at github includes a
> >> Category system, which you could combine with Link's cpsuite runner to
> >> get what you want.
> >>
> >>    David Saff
> >>
> >> On Thu, Oct 22, 2009 at 7:31 PM, ju_10101 <ju_10101@> wrote:
> >> > Hi,
> >> >
> >> > I've just updated to 4.7 from 3.8.2.  I have a large project with a lot of classes and packages, but I also annotate tests for fast and slow.  I create my FastSuite to find all tests classes annotated with SmallTest.  That way, these can be run first in the continuous environment, and the slow ones run later, on a different machine, so I don't backup the queue, etc.
> >> >
> >> > It seems the way to make suites in 4.7 is to explicitly list each classname as an annotation to your suite class, but I don't want to do that because a) I have a lot, and b) I don't want to not run new tests because someone didn't go update the manual list of test classes somewhere.
> >> >
> >> > My old way doesn't work anymore.  Can anyone tell me how to accomplish the same in junit 4.7??
> >> >
> >> >
> >> > My code in 3.8.2 was (sorry for the formatting!):
> >> >
> >> >    public static Test suite() {
> >> >        final String smallTestName = SmallTest.class.getCanonicalName();
> >> >        TestSuite suite = new TestSuite();
> >> >
> >> >        Enumeration testClasses = new LoadingTestCollector().collectTests();
> >> >        while (testClasses.hasMoreElements()) {
> >> >            final String className = testClasses.nextElement().toString();
> >> >
> >> >            try {
> >> >                Annotation[] annotations = Class.forName(className).getAnnotations();
> >> >                for (Annotation a : annotations) {
> >> >                    if (a.annotationType().getCanonicalName().equals(smallTestName)) {
> >> >                        System.out.println("HELLOOO- working on class " + className);
> >> >                        suite.addTestSuite(Class.forName(className));
> >> >                    }
> >> >                }
> >> >            }    catch (ClassNotFoundException e) {
> >> >                System.err.println("Error: Couldn't deal with " + className);
> >> >            }
> >> >        }
> >> >        return suite;
> >> >    }
> >> >
> >> >
> >> > The compiler error is: addTestSuite(java.lang.Class<? extends junit.framework.TestCase>) in junit.framework.TestSuite cannot be applied to (java.lang.Class<capture#337 of ?>)
> >> >    [javac]                         suite.addTestSuite(Class.forName(className));
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > ------------------------------------
> >> >
> >> > Yahoo! Groups Links
> >> >
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>

[David Saff]

Yishai,

Can you move the request to github?  I have an idea of a general way
to handle things like this, but it looks like it'll take backseat to
concurrency for at least a week.

   David Saff

On Tue, Oct 27, 2009 at 3:27 PM, yme0987654321 <yme0987654321@...> wrote:

> The way I do this currently is with a custom runner like this:
>
> public class CustomRunner extends Suite {
>
>    public CustomRunner(Class<?> setupClass) throws InitializationError {
>        super(setupClass, staticMethodWhichGetsTheClassesAsAnArrayForTheSuite());
>    }
> }
>
> And then Run an arbitrary class annotated with that Runner using @RunWith (I did it as a nested class so it was all right there, just be sure to declare it static). In that arbitrary class, you can do @BeforeClass and @AfterClass that might do interesting things.
>
> I put in a request (http://sourceforge.net/tracker/index.php?func=detail&aid=2786193&group%5Fid=15278&atid=365278) with my preferred way of resolving this, but I don't think anything was done on it.
>
> By the way, I'd be happy to code up the implementation and submit it as a patch if David/Kent would agree on the approach (a new runner, changes to the current runner, whatever. I don't really care, I would just like to see it as part of the framework).
>
> I think the above workaround is reasonable enough (although not really that obvious) that I can certainly understand not prioritizing addressing this.
>
> Yishai
>
> --- In junit@..., David Saff <david@...> wrote:
>>
>> ju_10101,
>>
>> Ask tomorrow for an ETA on the next release.
>>
>>    David
>>
>> On Fri, Oct 23, 2009 at 11:50 AM, ju_10101 <ju_10101@...> wrote:
>> > I just got folks in the office to agree to upgrade - might be a harder battle to get a nightly build, but I"ll take a look.
>> >
>> > With the 4.7 release then, there's no way to build a dynamic test suite?  Any estimate when the next release might be?  I might just drop back down to 3.8 in the meantime.
>> >
>> > Thanks!
>> >
>> >
>> > --- In junit@..., David Saff <david@> wrote:
>> >>
>> >> ju_10101,
>> >>
>> >> Are you able to use pre-release code?  HEAD at github includes a
>> >> Category system, which you could combine with Link's cpsuite runner to
>> >> get what you want.
>> >>
>> >>    David Saff
>> >>
>> >> On Thu, Oct 22, 2009 at 7:31 PM, ju_10101 <ju_10101@> wrote:
>> >> > Hi,
>> >> >
>> >> > I've just updated to 4.7 from 3.8.2.  I have a large project with a lot of classes and packages, but I also annotate tests for fast and slow.  I create my FastSuite to find all tests classes annotated with SmallTest.  That way, these can be run first in the continuous environment, and the slow ones run later, on a different machine, so I don't backup the queue, etc.
>> >> >
>> >> > It seems the way to make suites in 4.7 is to explicitly list each classname as an annotation to your suite class, but I don't want to do that because a) I have a lot, and b) I don't want to not run new tests because someone didn't go update the manual list of test classes somewhere.
>> >> >
>> >> > My old way doesn't work anymore.  Can anyone tell me how to accomplish the same in junit 4.7??
>> >> >
>> >> >
>> >> > My code in 3.8.2 was (sorry for the formatting!):
>> >> >
>> >> >    public static Test suite() {
>> >> >        final String smallTestName = SmallTest.class.getCanonicalName();
>> >> >        TestSuite suite = new TestSuite();
>> >> >
>> >> >        Enumeration testClasses = new LoadingTestCollector().collectTests();
>> >> >        while (testClasses.hasMoreElements()) {
>> >> >            final String className = testClasses.nextElement().toString();
>> >> >
>> >> >            try {
>> >> >                Annotation[] annotations = Class.forName(className).getAnnotations();
>> >> >                for (Annotation a : annotations) {
>> >> >                    if (a.annotationType().getCanonicalName().equals(smallTestName)) {
>> >> >                        System.out.println("HELLOOO- working on class " + className);
>> >> >                        suite.addTestSuite(Class.forName(className));
>> >> >                    }
>> >> >                }
>> >> >            }    catch (ClassNotFoundException e) {
>> >> >                System.err.println("Error: Couldn't deal with " + className);
>> >> >            }
>> >> >        }
>> >> >        return suite;
>> >> >    }
>> >> >
>> >> >
>> >> > The compiler error is: addTestSuite(java.lang.Class<? extends junit.framework.TestCase>) in junit.framework.TestSuite cannot be applied to (java.lang.Class<capture#337 of ?>)
>> >> >    [javac]                         suite.addTestSuite(Class.forName(className));
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > ------------------------------------
>> >> >
>> >> > Yahoo! Groups Links
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >
>> >
>> >
>> >
>> > ------------------------------------
>> >
>> > Yahoo! Groups Links
>> >
>> >
>> >
>> >
>>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

[yme0987654321]

Done, request ID 41.

--- In junit@..., David Saff <david@...> wrote:

>
> Yishai,
>
> Can you move the request to github?  I have an idea of a general way
> to handle things like this, but it looks like it'll take backseat to
> concurrency for at least a week.
>
>    David Saff
>
> On Tue, Oct 27, 2009 at 3:27 PM, yme0987654321 <yme0987654321@...> wrote:
> > The way I do this currently is with a custom runner like this:
> >
> > public class CustomRunner extends Suite {
> >
> >    public CustomRunner(Class<?> setupClass) throws InitializationError {
> >        super(setupClass, staticMethodWhichGetsTheClassesAsAnArrayForTheSuite());
> >    }
> > }
> >
> > And then Run an arbitrary class annotated with that Runner using @RunWith (I did it as a nested class so it was all right there, just be sure to declare it static). In that arbitrary class, you can do @BeforeClass and @AfterClass that might do interesting things.
> >
> > I put in a request (http://sourceforge.net/tracker/index.php?func=detail&aid=2786193&group%5Fid=15278&atid=365278) with my preferred way of resolving this, but I don't think anything was done on it.
> >
> > By the way, I'd be happy to code up the implementation and submit it as a patch if David/Kent would agree on the approach (a new runner, changes to the current runner, whatever. I don't really care, I would just like to see it as part of the framework).
> >
> > I think the above workaround is reasonable enough (although not really that obvious) that I can certainly understand not prioritizing addressing this.
> >
> > Yishai
> >
> > --- In junit@..., David Saff <david@> wrote:
> >>
> >> ju_10101,
> >>
> >> Ask tomorrow for an ETA on the next release.
> >>
> >>    David
> >>
> >> On Fri, Oct 23, 2009 at 11:50 AM, ju_10101 <ju_10101@> wrote:
> >> > I just got folks in the office to agree to upgrade - might be a harder battle to get a nightly build, but I"ll take a look.
> >> >
> >> > With the 4.7 release then, there's no way to build a dynamic test suite?  Any estimate when the next release might be?  I might just drop back down to 3.8 in the meantime.
> >> >
> >> > Thanks!
> >> >
> >> >
> >> > --- In junit@..., David Saff <david@> wrote:
> >> >>
> >> >> ju_10101,
> >> >>
> >> >> Are you able to use pre-release code?  HEAD at github includes a
> >> >> Category system, which you could combine with Link's cpsuite runner to
> >> >> get what you want.
> >> >>
> >> >>    David Saff
> >> >>
> >> >> On Thu, Oct 22, 2009 at 7:31 PM, ju_10101 <ju_10101@> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I've just updated to 4.7 from 3.8.2.  I have a large project with a lot of classes and packages, but I also annotate tests for fast and slow.  I create my FastSuite to find all tests classes annotated with SmallTest.  That way, these can be run first in the continuous environment, and the slow ones run later, on a different machine, so I don't backup the queue, etc.
> >> >> >
> >> >> > It seems the way to make suites in 4.7 is to explicitly list each classname as an annotation to your suite class, but I don't want to do that because a) I have a lot, and b) I don't want to not run new tests because someone didn't go update the manual list of test classes somewhere.
> >> >> >
> >> >> > My old way doesn't work anymore.  Can anyone tell me how to accomplish the same in junit 4.7??
> >> >> >
> >> >> >
> >> >> > My code in 3.8.2 was (sorry for the formatting!):
> >> >> >
> >> >> >    public static Test suite() {
> >> >> >        final String smallTestName = SmallTest.class.getCanonicalName();
> >> >> >        TestSuite suite = new TestSuite();
> >> >> >
> >> >> >        Enumeration testClasses = new LoadingTestCollector().collectTests();
> >> >> >        while (testClasses.hasMoreElements()) {
> >> >> >            final String className = testClasses.nextElement().toString();
> >> >> >
> >> >> >            try {
> >> >> >                Annotation[] annotations = Class.forName(className).getAnnotations();
> >> >> >                for (Annotation a : annotations) {
> >> >> >                    if (a.annotationType().getCanonicalName().equals(smallTestName)) {
> >> >> >                        System.out.println("HELLOOO- working on class " + className);
> >> >> >                        suite.addTestSuite(Class.forName(className));
> >> >> >                    }
> >> >> >                }
> >> >> >            }    catch (ClassNotFoundException e) {
> >> >> >                System.err.println("Error: Couldn't deal with " + className);
> >> >> >            }
> >> >> >        }
> >> >> >        return suite;
> >> >> >    }
> >> >> >
> >> >> >
> >> >> > The compiler error is: addTestSuite(java.lang.Class<? extends junit.framework.TestCase>) in junit.framework.TestSuite cannot be applied to (java.lang.Class<capture#337 of ?>)
> >> >> >    [javac]                         suite.addTestSuite(Class.forName(className));
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > ------------------------------------
> >> >> >
> >> >> > Yahoo! Groups Links
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> >
> >> > ------------------------------------
> >> >
> >> > Yahoo! Groups Links
> >> >
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>