tag:old.nabble.com,2006:forum-21511Nabble - Surefire2009-11-05T11:06:40Z<a href="http://maven.apache.org/surefire/surefire-api/" target="_top" rel="nofollow">Surefire</a> is a test framework project and belongs to the Apache Software Foundation.tag:old.nabble.com,2006:post-26220744Help with classpath issue2009-11-05T11:06:40Z2009-11-05T11:06:40ZBen DouglasHello,
<br><br>I'm currently running Maven 2.2.1. When I try to put a
<br>META-INF/hibernate.xml in the testResources directory (and therefore get
<br>copied over to the test-classes/META-INF dir) all the tests fail looking
<br>for a NamedQuery that is in one of my classes. However, when I run
<br>without the META-INF (and a clean before hand) the tests find the
<br>NamedQuery in the classes directory just fine. I've also found that if
<br>I copy over all the source classes into the test-classes directory with
<br>the META-INF/hibernate.xml the tests pass as well. It *looks* like the
<br>target/classes directory is put in the classpath that surefire gets, but
<br>not sure if it's setting up correctly or not.
<br><br>Please let me know if any more information is needed, or if this should
<br>be a thread in another project... just figured I'd start here ;)
<br><br>Thanks,
<br>Ben
<br><br>Here's the surefire...tmp file that sets up the classPath:
<br><br>#surefire
<br>#Thu Nov 05 14:00:14 GMT-05:00 2009
<br>enableAssertions=true
<br>report.0.types=java.lang.Boolean
<br>report.2.types=java.io.File|java.lang.Boolean
<br>report.0.params=true
<br>report.1.types=java.io.File|java.lang.Boolean
<br>testSuite.0.params=/home/bdouglas/eclipse-ws/RegressionSystem/TestcaseManager/target/test-classes|[**/Test*.java,
<br>**/*Test.java, **/*TestCase.java]|[**/*$*]
<br>report.1.params=/home/bdouglas/eclipse-ws/RegressionSystem/TestcaseManager/target/surefire-reports|true
<br>classPathUrl.39=/home/bdouglas/.m2/repository/org/jvnet/mimepull/1.2/mimepull-1.2.jar
<br>classPathUrl.38=/home/bdouglas/.m2/repository/com/sun/org/apache/xml/internal/resolver/20050927/resolver-20050927.jar
<br>classPathUrl.37=/home/bdouglas/.m2/repository/com/sun/xml/stream/sjsxp/1.0.1/sjsxp-1.0.1.jar
<br>classPathUrl.36=/home/bdouglas/.m2/repository/org/jvnet/staxex/stax-ex/1.2/stax-ex-1.2.jar
<br>classPathUrl.35=/home/bdouglas/.m2/repository/com/sun/xml/stream/buffer/streambuffer/0.7/streambuffer-0.7.jar
<br>classPathUrl.34=/home/bdouglas/.m2/repository/com/sun/xml/messaging/saaj/saaj-impl/1.3.1/saaj-impl-1.3.1.jar
<br>report.2.params=/home/bdouglas/eclipse-ws/RegressionSystem/TestcaseManager/target/surefire-reports|true
<br>classPathUrl.33=/home/bdouglas/.m2/repository/com/sun/xml/bind/jaxb-impl/2.1.7/jaxb-impl-2.1.7.jar
<br>classPathUrl.32=/home/bdouglas/.m2/repository/javax/jws/jsr181-api/1.0-MR1/jsr181-api-1.0-MR1.jar
<br>surefireClassPathUrl.2=/home/bdouglas/.m2/repository/org/apache/maven/surefire/surefire-api/2.4.3/surefire-api-2.4.3.jar
<br>classPathUrl.31=/home/bdouglas/.m2/repository/javax/annotation/jsr250-api/1.0/jsr250-api-1.0.jar
<br>surefireClassPathUrl.1=/home/bdouglas/.m2/repository/junit/junit/4.0/junit-4.0.jar
<br>classPathUrl.30=/home/bdouglas/.m2/repository/javax/xml/soap/saaj-api/1.3/saaj-api-1.3.jar
<br>surefireClassPathUrl.0=/home/bdouglas/.m2/repository/org/apache/maven/surefire/surefire-junit4/2.4.3/surefire-junit4-2.4.3.jar
<br>classPathUrl.9=/home/bdouglas/.m2/repository/dom4j/dom4j/1.6.1/dom4j-1.6.1.jar
<br>classPathUrl.8=/home/bdouglas/.m2/repository/asm/asm-attrs/1.5.3/asm-attrs-1.5.3.jar
<br>classPathUrl.7=/home/bdouglas/.m2/repository/commons-collections/commons-collections/2.1.1/commons-collections-2.1.1.jar
<br>classPathUrl.6=/home/bdouglas/.m2/repository/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar
<br>classPathUrl.5=/home/bdouglas/.m2/repository/net/sf/ehcache/ehcache/1.2.3/ehcache-1.2.3.jar
<br>classPathUrl.4=/home/bdouglas/.m2/repository/org/hibernate/hibernate/3.2.6.ga/hibernate-3.2.6.ga.jar
<br>classPathUrl.3=/home/bdouglas/.m2/repository/javax/persistence/persistence-api/1.0/persistence-api-1.0.jar
<br>classPathUrl.2=/home/bdouglas/.m2/repository/junit/junit/4.5/junit-4.5.jar
<br>classPathUrl.1=/home/bdouglas/eclipse-ws/RegressionSystem/TestcaseManager/target/classes
<br>report.2=org.apache.maven.surefire.report.XMLReporter
<br>classPathUrl.0=/home/bdouglas/eclipse-ws/RegressionSystem/TestcaseManager/target/test-classes
<br>childDelegation=false
<br>report.1=org.apache.maven.surefire.report.BriefFileReporter
<br>report.0=org.apache.maven.surefire.report.ForkingConsoleReporter
<br>classPathUrl.29=/home/bdouglas/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar
<br>classPathUrl.28=/home/bdouglas/.m2/repository/javax/xml/stream/stax-api/1.0/stax-api-1.0.jar
<br>classPathUrl.27=/home/bdouglas/.m2/repository/javax/xml/bind/jaxb-api/2.1/jaxb-api-2.1.jar
<br>classPathUrl.26=/home/bdouglas/.m2/repository/javax/xml/ws/jaxws-api/2.1/jaxws-api-2.1.jar
<br>classPathUrl.25=/home/bdouglas/.m2/repository/com/sun/xml/ws/jaxws-rt/2.1.4/jaxws-rt-2.1.4.jar
<br>classPathUrl.24=/home/bdouglas/.m2/repository/mysql/mysql-connector-java/5.1.10/mysql-connector-java-5.1.10.jar
<br>classPathUrl.23=/home/bdouglas/.m2/repository/c3p0/c3p0/0.9.1.2/c3p0-0.9.1.2.jar
<br>classPathUrl.22=/home/bdouglas/.m2/repository/log4j/log4j/1.2.12/log4j-1.2.12.jar
<br>classPathUrl.21=/home/bdouglas/.m2/repository/org/slf4j/slf4j-log4j12/1.4.2/slf4j-log4j12-1.4.2.jar
<br>classPathUrl.20=/home/bdouglas/.m2/repository/javassist/javassist/3.4.GA/javassist-3.4.GA.jar
<br>failIfNoTests=false
<br>useManifestOnlyJar=true
<br>classPathUrl.19=/home/bdouglas/.m2/repository/javax/transaction/jta/1.1/jta-1.1.jar
<br>classPathUrl.18=/home/bdouglas/.m2/repository/org/hibernate/hibernate-core/3.3.0.SP1/hibernate-core-3.3.0.SP1.jar
<br>testSuite.0.types=java.io.File|java.util.ArrayList|java.util.ArrayList
<br>classPathUrl.17=/home/bdouglas/.m2/repository/org/hibernate/hibernate-annotations/3.4.0.GA/hibernate-annotations-3.4.0.GA.jar
<br>classPathUrl.16=/home/bdouglas/.m2/repository/org/slf4j/slf4j-api/1.4.2/slf4j-api-1.4.2.jar
<br>classPathUrl.15=/home/bdouglas/.m2/repository/org/hibernate/hibernate-commons-annotations/3.1.0.GA/hibernate-commons-annotations-3.1.0.GA.jar
<br>classPathUrl.14=/home/bdouglas/.m2/repository/org/hibernate/ejb3-persistence/1.0.2.GA/ejb3-persistence-1.0.2.GA.jar
<br>classPathUrl.13=/home/bdouglas/.m2/repository/org/hibernate/hibernate-entitymanager/3.4.0.GA/hibernate-entitymanager-3.4.0.GA.jar
<br>classPathUrl.12=/home/bdouglas/.m2/repository/asm/asm/1.5.3/asm-1.5.3.jar
<br>classPathUrl.11=/home/bdouglas/.m2/repository/cglib/cglib/2.1_3/cglib-2.1_3.jar
<br>classPathUrl.10=/home/bdouglas/.m2/repository/antlr/antlr/2.7.6/antlr-2.7.6.jar
<br>testSuite.0=org.apache.maven.surefire.junit4.JUnit4DirectoryTestSuite
<br>classPathUrl.40=/home/bdouglas/.m2/repository/log4j/apache-log4j-extras/1.0/apache-log4j-extras-1.0.jar
<br>useSystemClassLoader=true
<br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-26218448deploy rpm to rpm repository using maven + mojo rpm plugin2009-11-05T08:59:41Z2009-11-05T08:59:41ZTranquilHi guys,
<br><br>i'm just starting to play around with maven as a candidate for our whole
<br>build/deploy system for java applications.
<br><br>we're been programming so far using Perl, packaging with Rpm on Fedora and
<br>Deploying it though a local yum repository.
<br><br><br>We still haven't decided if we could ditch the yum/rpm idea all together and
<br>swith to mvn repository,
<br>but in case we won't,
<br><br>Do you know* i**f there is a way to config mvn to upload the rpm (using
<br>SCP?) to the yum repository* after
<br>it finished building the RPM (today we use a build system based on make
<br>files and in the end it scp's the rpm to the repository and run make there
<br>to update the list of rpms).
<br><br>thank you.
<br><br>--
<br>Eyal Edri
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-25989676Using apache ant inside a surefire test2009-10-21T03:02:01Z2009-10-21T03:02:01ZKriegelstein, ThomasHi there,
<br><br>
<br><br>I didn't find a solution or even suggestion online to the following
<br>issue I'm having. I've developed a small ant-Task and build it with
<br>maven2. The ant task is supposed to throw a BuildException under certain
<br>circumstances. To test the task, I've created two junit tests, which
<br>invoke ant's Launcher during their test methods. One invocation is
<br>expected to pass, one is expected to fail and produce the
<br>BuildException.
<br><br>
<br><br>Within the test method, I've surrounded the invocation to the Launchers
<br>run method with try and catch, the Exception still propagates to
<br>surrounding surefire plugin and causes the maven2 build to fail.
<br><br>
<br><br>What should I do, to prevent the surefire plugin from seeing the
<br>BuildException thrown by the ant task?
<br><br>
<br><br>Greetings,
<br><br> Thomas
<br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-25798060Re: Concurrent Junit (SUREFIRE-555)2009-10-07T21:04:24Z2009-10-07T21:04:24Zdfabulich<br>I think this list is no longer in use. Instead just use the main
<br>maven-dev list.
<br><br>-Dan
<br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-25793870Concurrent Junit (SUREFIRE-555)2009-10-07T13:30:37Z2009-10-07T13:30:37ZKristian Rosenvold-2I am the author of the patches in
<br><a href="http://jira.codehaus.org/browse/SUREFIRE-555" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-555</a>. I was wondering if anyone
<br>would have the time to review this patch and hopefully accept it for
<br>inclusion in surefire. I am more than willing to make further adjustments if
<br>necessary.
<br><br>The patch initially had some external dependencies that were a bit
<br>complicated, and these have been removed.
<br><br>The only known issue with this patch at the moment is that the execution
<br>times for the tests are totally incorrect. This will require a bigger
<br>rewrite of the reporting module. (I can do this as a separate patch if
<br>needed, but this patch is large enough already)
<br><br>Regards,
<br><br>Kristian Rosenvold
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-25463686generating centralized findbugs report2009-09-15T00:51:13Z2009-09-15T00:51:13ZMircea Markus-2Hi,
<br><br>I have a project that has the following structure:
<br><br>- pom.xml
<br>- project_1
<br> - pom.xml (extends ../pom.xml)
<br>- other
<br> - other1
<br> - pom.xml (extends ../pom.xml)
<br> - other
<br> - pom.xml (extends ../pom.xml)
<br> pom.xml (extends ../pom.xml)
<br>- project_2
<br> pom.xml (extens ../pom.xml)
<br><br><br>I would like that whenever I run "mvn site" to have a single,
<br>centralized findbugs report in the <root>/site, rather than multiple
<br>individual ones in subprojects.
<br>Is that possible?
<br><br>Thank you,
<br>Mircea
<br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-24364859Yet another question about annotations in Junit Tests2009-07-06T12:12:24Z2009-07-06T12:12:24ZCooke, Peter M CTR DMDCHello:
<br> Sorry I am new to Maven and Surefire but I think my problem should be
<br>posted to this group.
<br><br>I like many others am having a problem running Junit 4.4 test cases when
<br>running maven. I have not seen a definitive solution to the problem.
<br><br>I can manually execute the following class and the test fails in the
<br>setUp() method as expected. When I run this with maven, the @before
<br>method is never called. I have seen a couple of different postings in
<br>various places that indicate that indicate that is fixed with the code
<br>changes for running TestNG. I have also seen some postings that
<br>indicate that this problem is not fixed and may never get fixed.
<br><br>My question is should maven and surefire be able to run this class and
<br>fail in the setUp() method as coded?
<br>Is there something I can change in my pom to make it run in surefire as
<br>a 4.4 Junit?
<br><br><br>Currently using Maven 2.1.0, surefire 2.4.3
<br>
<br> <plugin>
<br> <groupId>org.apache.maven.plugins</groupId>
<br> <artifactId>maven-surefire-plugin</artifactId>
<br> <configuration>
<br> <forkMode>always</forkMode>
<br> <redirectTestOutputToFile>true</redirectTestOutputToFile>
<br> <!-- Add the following argument to skip the tests
<br>-Dmaven.test.skip=true -->
<br> <includes>
<br> <include>**/*Tests.java</include>
<br> </includes>
<br>
<br> </configuration>
<br> </plugin>
<br><br><br>package support;
<br><br>import junit.framework.Assert;
<br><br>import org.junit.After;
<br>import org.junit.Before;
<br>import org.junit.Test;
<br>import org.junit.internal.runners.JUnit4ClassRunner;
<br>import org.junit.runner.RunWith;
<br><br><br><br>@RunWith(JUnit4ClassRunner.class)
<br>public class FooTests {
<br>
<br> @Before
<br> public void setUp() throws Exception {
<br> Assert.fail("foo");
<br> }
<br>
<br> @After
<br> public void tearDown() throws Exception {
<br> }
<br>
<br> @Test
<br> public void testFoo() {
<br> }
<br>
<br> public static void main(String[] args) {
<br> org.junit.runner.JUnitCore.main("support.FooTests");
<br> }
<br>}
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-23993109Debugging Hints2009-06-11T22:30:29Z2009-06-11T22:30:29ZSteven NunezGreetings all,
<br><br>Does anyone have any debugging hints for surefire? I've got a perplexing problem, not helped by the fact that surefire doesn't appear to tell me the source of the problem.
<br><br>I've got a working surefire/testng system working, and have gone to add tests for a second class. This class is nearly identical to the first, except that it processes a different data set. The trouble is that surefire won't even load the second set of tests. The first one still works fine, but as soon as the second class is added, 'BUILD FAILURE', with no indication of why. After running maven with '--debug', I still get nothing. The TestSuite.txt file is empty, and the console output doesn't indicate what the problem might be.
<br><br>The last line of the below shell excerpt says "... 16 more"; it sure would be useful to know how to obtain those last 16 errors. Does anyone know how?
<br><br>Cheers,
<br> - Steve Nunez
<br><br>-------------------------------------------------------
<br> T E S T S
<br>-------------------------------------------------------
<br>Running TestSuite
<br>[INFO] ------------------------------------------------------------------------
<br>[ERROR] BUILD FAILURE
<br>[INFO] ------------------------------------------------------------------------
<br>[INFO] There are test failures.
<br><br>Please refer to /Users/nunez/Illation/Development/Scratch/test-app/trunk/drools-4.0.7/target/surefire-reports for the individual test results.
<br>[INFO] ------------------------------------------------------------------------
<br>[DEBUG] Trace
<br>org.apache.maven.BuildFailureException: There are test failures.
<br><br>Please refer to /Users/nunez/Illation/Development/Scratch/test-app/trunk/drools-4.0.7/target/surefire-reports for the individual test results.
<br>at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:699)
<br>at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:540)
<br>at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:519)
<br>at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:371)
<br>at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:332)
<br>at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:181)
<br>at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:356)
<br>at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:137)
<br>at org.apache.maven.cli.MavenCli.main(MavenCli.java:356)
<br>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<br>at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
<br>at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
<br>at java.lang.reflect.Method.invoke(Method.java:585)
<br>at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
<br>at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
<br>at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
<br>at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
<br>Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures.
<br><br>Please refer to /Users/nunez/Illation/Development/Scratch/test-app/trunk/drools-4.0.7/target/surefire-reports for the individual test results.
<br>at org.apache.maven.plugin.surefire.SurefirePlugin.execute(SurefirePlugin.java:575)
<br>at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:483)
<br>at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:678)
<br>... 16 more
<br>[INFO] ------------------------------------------------------------------------
<br>[INFO] Total time: 7 seconds
<br>[INFO] Finished at: Fri Jun 12 15:23:11 EST 2009
<br>[INFO] Final Memory: 9M/23M
<br>[INFO] ------------------------------------------------------------------------
<br><br><br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-23617063Why isn't Surefire more easier and more sensible to use?2009-05-18T18:22:04Z2009-05-18T18:22:04ZBen Hutchison-2Ive recently started using Surefire as part of a Maven project.
<br><br>I was really surprised how inconvenient it was to use, and how things
<br>about it seemed, well... stupid.
<br><br>1. When running a suite of tests with "surefire:test
<br><<a href="http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html" target="_top" rel="nofollow">http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html</a>>",
<br>the main thing you want to see a summary of all failed tests. It is THE
<br>most important and normal thing to look at. I was surprised to discover
<br>Surefire doesnt produce this essential item by default. Instead, it
<br>creates many individual reports, one per test, whether they passed or
<br>failed.
<br><br>2. Then, one has to run a second plugin and special goal just to get the
<br>summary, which is what any normal person is going to be looking for. It
<br>puts the summary in a different directory from the individual files. And
<br>it doesnt produce this report as part of the normal build.
<br><br>3. It generates the XML intermediate files that the report needs. It
<br>also generates identical txt files, for no apparent reason. And
<br>bizarrely, you can turn the XML off, but you cant turn the txt off.
<br><br>And then you get weird situations where because the tests fail, the
<br>build fails, and this actually stops the report generation which will
<br>tell you what test failed :(
<br><a href="http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel</a><br><a href="http://www.eclipse.org/newsportal/article.php?id=216&group=eclipse.technology.iam#216" target="_top" rel="nofollow">http://www.eclipse.org/newsportal/article.php?id=216&group=eclipse.technology.iam#216</a><br><br>The whole thing seems like a classic case of programmer-centric, not
<br>user-centric, thinking. This tool doesn't seem to have started with what
<br>a typical user would want and then worked backwards to meet this goal.
<br><br>I dont particularly enjoy criticizing some programmer's unpaid
<br>contribution, but Surefire is a widely used piece of infrastructure, and
<br>it should work better than it does.
<br><br>-Ben
<br><br><br>--
<br><br>
<br><br>*Ben Hutchison
<br>Senior Developer
<br>* Level 2 476 St Kilda Road Melbourne VIC 3004
<br>T 613 8807 5252 | F 613 8807 5203 | M 0423 879 534 |
<br>www.ibsglobalweb.com <<a href="http://www.ibsglobalweb.com/" target="_top" rel="nofollow">http://www.ibsglobalweb.com/</a>>
<br><br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-23443870Unknown source for assertion in surefire driven JUnit test with Tellurium2009-05-08T03:54:56Z2009-05-08T03:54:56ZErnst PerssonHi,
<br><br>I've started looking at the Tellurium web testing framework. I'm using
<br>their template with Maven, Surefire and JUnit to drive the tests. It's
<br>very easy to test it, just follow this guide.
<br><a href="http://code.google.com/p/aost/wiki/TenMinutesToTellurium" target="_top" rel="nofollow">http://code.google.com/p/aost/wiki/TenMinutesToTellurium</a><br><br>What happens is that when I have an assertion failure the JVM (?)
<br>doesn't provide the source file and line number! See stacktrace (1)
<br>This leads to a number of issues, i.e. you can't build reports. See
<br>stacktrace (2)
<br><br>I can't see that anything Tellurium-specific goes on here and neither
<br>can the Tellurium maintainer.
<br>Does anyone have a clue what might be causing this?
<br><br>Using surefire 2.4.3.
<br><br>Regards
<br>/Ernst Persson
<br><br>(1)
<br>more target/surefire-reports/test.SomethingTestCase.txt
<br>-------------------------------------------------------------------------------
<br>Test set: test.SomethingTestCase
<br>-------------------------------------------------------------------------------
<br>Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 18.251
<br>sec <<< FAILURE!
<br>testDuplicateName(test.SomethingTestCase) Time elapsed: 2.231 sec <<< FAILURE!
<br>java.lang.AssertionError: expected not same
<br> at org.junit.Assert.fail(Assert.java:74)
<br> at org.junit.Assert.failSame(Assert.java:437)
<br> at org.junit.Assert.assertNotSame(Assert.java:420)
<br> at org.junit.Assert.assertNotSame(Assert.java:430)
<br>>>>> at test.SomethingTestCase.testDuplicateName(Unknown Source)
<br> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
<br> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
<br> at java.lang.reflect.Method.invoke(Method.java:597)
<br> at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:59)
<br> at org.junit.internal.runners.MethodRoadie.runTestMethod(MethodRoadie.java:98)
<br> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:79)
<br> at org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:87)
<br> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:77)
<br> at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:42)
<br> at org.junit.internal.runners.JUnit4ClassRunner.invokeTestMethod(JUnit4ClassRunner.java:88)
<br> at org.junit.internal.runners.JUnit4ClassRunner.runMethods(JUnit4ClassRunner.java:51)
<br> at org.junit.internal.runners.JUnit4ClassRunner$1.run(JUnit4ClassRunner.java:44)
<br> at org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:27)
<br> at org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:37)
<br> at org.junit.internal.runners.JUnit4ClassRunner.run(JUnit4ClassRunner.java:42)
<br> at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
<br> at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
<br> at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
<br> at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
<br> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
<br> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
<br> at java.lang.reflect.Method.invoke(Method.java:597)
<br> at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
<br> at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)
<br><br>(2)
<br>[ERROR] FATAL ERROR
<br>[INFO] ------------------------------------------------------------------------
<br>[INFO] String index out of range: -1
<br>[INFO] ------------------------------------------------------------------------
<br>[INFO] Trace
<br>java.lang.StringIndexOutOfBoundsException: String index out of range: -1
<br> at java.lang.String.substring(String.java:1938)
<br> at org.apache.maven.plugins.surefire.report.SurefireReportGenerator.getErrorLineNumber(SurefireReportGenerator.java:638)
<br> at org.apache.maven.plugins.surefire.report.SurefireReportGenerator.constructFailureDetails(SurefireReportGenerator.java:600)
<br> at org.apache.maven.plugins.surefire.report.SurefireReportGenerator.doGenerateReport(SurefireReportGenerator.java:111)
<br> at org.apache.maven.plugins.surefire.report.SurefireReportMojo.executeReport(SurefireReportMojo.java:180)
<br> at org.apache.maven.reporting.AbstractMavenReport.generate(AbstractMavenReport.java:98)
<br> at org.apache.maven.plugins.site.ReportDocumentRenderer.renderDocument(ReportDocumentRenderer.java:139)
<br> at org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.renderModule(DefaultSiteRenderer.java:269)
<br> at org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.render(DefaultSiteRenderer.java:101)
<br> at org.apache.maven.plugins.site.SiteMojo.renderLocale(SiteMojo.java:133)
<br> at org.apache.maven.plugins.site.SiteMojo.execute(SiteMojo.java:100)
<br> at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:483)
<br> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:678)
<br> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:540)
<br> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:519)
<br> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:371)
<br> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:332)
<br> at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:181)
<br> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:356)
<br> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:137)
<br> at org.apache.maven.cli.MavenCli.main(MavenCli.java:356)
<br> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
<br> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
<br> at java.lang.reflect.Method.invoke(Method.java:597)
<br> at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
<br> at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
<br> at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
<br> at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
<br><br>PS I hope I'm not upsetting anyone by crossposting this, I really have
<br>no idea where the problem is.
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-22660743forking single tests from junit test suite2009-03-23T07:01:59Z2009-03-23T07:01:59ZChristoph HögerHi,
<br><br>I need to test a feature for my project on large test input files.
<br>To get that, I create TestCases from a single TestSuite dynamically. The
<br>relevant part of the pom looks like:
<br><br><div class='shrinkable-quote'><div class='shrinkable-quote'><br>> <plugin>
<br>> <groupId>org.apache.maven.plugins</groupId>
<br>> <artifactId>maven-surefire-plugin</artifactId>
<br>> <configuration>
<br>> <reportFormat>plain</reportFormat>
<br>> <trimStackTrace>false</trimStackTrace>
<br>> <forkMode>perTest</forkMode>
<br>> <argLine>-XX:+HeapDumpOnOutOfMemoryError</argLine>
<br>> <excludes>
<br>> <exclude>**/IncrementalParsePhase1Test.java</exclude>
<br>> <exclude>**/incremental/testparser/*.java</exclude>
<br>> </excludes>
<br>> <includes>
<br>> <include>**/IncrementalParsingTestSuite.java</include>
<br>> </includes>
<br>> </configuration>
<br>> </plugin>
</div></div>In my TestSuite there runs a loop:
<br><br><div class='shrinkable-quote'><div class='shrinkable-quote'><br>> for (int i = 0; i < Lexer.getCachedInput().size(); i++) {
<br>> //brake for large test cases
<br>> if (i % (1 + (Lexer.getCachedInput().size() / 20)) != 0)
<br>> continue;
<br>>
<br>> Range diffRange = new Range(i, i);
<br>>
<br>> IncrementalParsePhase1Test test = new IncrementalParsePhase1Test(
<br>> Lexer.getInputFileName(), Lexer.getCachedInput(),
<br>> generatedStates, diffRange, rootAbsy, Lexer.getCachedInput().get(i).getSavedConfiguration());
<br>> suite.addTest(test);
<br>> }
</div></div>The problem is, that maven does not seem to run every testCase in a new
<br>vm, but reuses the old one (and somehow prevents the GC from cleaning
<br>up old testcases), so earlier or later I run into a heap overflow.
<br><br>This is especially bad, as I want to measure the performance of some operations and the GC seems to interrupt my threads thus creating huge performance problems.
<br><br>So is it possible to make surefire run each test in a single vm?
<br><br>regards
<br><br>Christoph
<br><br /> <div class="small"><br/><img src="http://old.nabble.com/images/icon_attachment.gif" > <strong>signature.asc</strong> (204 bytes) <a href="http://old.nabble.com/attachment/22660743/0/signature.asc" target="_top">Download Attachment</a></div><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-22039344Junit target code dependencies are unresolved2009-02-16T07:27:09Z2009-02-16T07:27:09ZFrederic TuttleI've written a junit test case against a class that has dependencies (log4j
<br>for instance). I've declared those dependencies in my pom.xml. I've declared
<br>junit 3.8.1 as a test scoped dependency.
<br><br>When I run mvn test I get errors on each test indicating that junit is not
<br>resolving the references to the dependencies in the target code. From the
<br>surefire-reports--
<br><br> <error type="java.lang.Error" message="Unresolved compilation problems:
<br> The import org.apache cannot be resolved
<br> The import javax.xml.bind cannot be resolved
<br> The import javax.xml.bind cannot be resolved
<br> The import javax.xml.bind cannot be resolved
<br> Logger cannot be resolved to a type
<br> Logger cannot be resolved
<br> Logger cannot be resolved to a type
<br> Logger cannot be resolved to a type
<br> Logger cannot be resolved to a type
<br> Logger cannot be resolved to a type
<br> JAXBContext cannot be resolved to a type
<br> JAXBContext cannot be resolved
<br> Marshaller cannot be resolved to a type
<br> Marshaller cannot be resolved
<br> Logger cannot be resolved to a type
<br> Logger cannot be resolved to a type
<br> JAXBContext cannot be resolved to a type
<br> JAXBContext cannot be resolved
<br> Unmarshaller cannot be resolved to a type
<br> Logger cannot be resolved to a type
<br> Logger cannot be resolved to a type
<br> Logger cannot be resolved to a type
<br>...
<br><br>This is a very simple case and there is nothing special about it. I tried
<br>changing the dependency scopes and explicitly adding jars to the surefire
<br>config and nothing worked.
<br><br>Anyone else have this problem ?
<br>I'd be gratefull for some pointers.
<br>Thanks.
<br><br> -FT
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-21300757Expected Behavior When Using report-only2009-01-05T08:18:37Z2009-01-05T08:18:37ZleojhartivHello!
<br>
<br>I've read the JIRA found here:
<br><a href="http://jira.codehaus.org/browse/SUREFIRE-257" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-257</a><br>and tried to incorporate its advice into my POM.
<br>
<br>Currently, when I run "mvn clean install site", I get the following:
<br><br>
<br><br> C:\workspace\shares-trunk\shares-domain>mvn clean install site
<br>
<br> C:\workspace\shares-trunk\shares-domain>set MAVEN_OPTS=-Xmx256m
<br> [INFO] Scanning for projects...
<br> [INFO]
<br>------------------------------------------------------------------------
<br> [INFO] Building shares-domain
<br> [INFO] task-segment: [clean, install, site]
<br> [INFO]
<br>------------------------------------------------------------------------
<br> [INFO] [clean:clean]
<br> [INFO] Deleting directory
<br>C:\workspace\shares-trunk\shares-domain\target
<br> [INFO] [resources:resources]
<br> [INFO] Using default encoding to copy filtered resources.
<br> [INFO] [compiler:compile]
<br> [INFO] Compiling 101 source files to
<br>C:\workspace\shares-trunk\shares-domain\target\classes
<br> [INFO] [resources:testResources]
<br> [INFO] Using default encoding to copy filtered resources.
<br> [INFO] [compiler:testCompile]
<br> [INFO] Compiling 35 source files to
<br>C:\workspace\shares-trunk\shares-domain\target\test-classes
<br> [INFO] [surefire:test]
<br> [INFO] Surefire report directory:
<br>C:\workspace\shares-trunk\shares-domain\target\surefire-reports
<br>
<br> -------------------------------------------------------
<br> T E S T S
<br> -------------------------------------------------------
<br>
<br> ...<tests are run here - clears for confidentiality>...
<br>
<br> Results :
<br>
<br> Tests run: 54, Failures: 0, Errors: 0, Skipped: 0
<br>
<br> [INFO] [jar:jar]
<br> [INFO] Building jar:
<br>C:\workspace\shares-trunk\shares-domain\target\shares-domain-1.0-SNAPSHO
<br>T.jar
<br> [INFO] [install:install]
<br> [INFO] Installing
<br>C:\workspace\shares-trunk\shares-domain\target\shares-domain-1.0-SNAPSHO
<br>T.jar to \.m2\repository\com\fideli
<br>
<br>ty\shares\shares-domain\1.0-SNAPSHOT\shares-domain-1.0-SNAPSHOT.jar
<br> [INFO] Setting property: classpath.resource.loader.class =>
<br>'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
<br> [INFO] Setting property: velocimacro.messages.on => 'false'.
<br> [INFO] Setting property: resource.loader => 'classpath'.
<br> [INFO] Setting property: resource.manager.logwhenfound =>
<br>'false'.
<br> [INFO]
<br>**************************************************************
<br> [INFO] Starting Jakarta Velocity v1.4
<br> [INFO] RuntimeInstance initializing.
<br> [INFO] Default Properties File:
<br>org\apache\velocity\runtime\defaults\velocity.properties
<br> [INFO] Default ResourceManager initializing. (class
<br>org.apache.velocity.runtime.resource.ResourceManagerImpl)
<br> [INFO] Resource Loader Instantiated:
<br>org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader
<br> [INFO] ClasspathResourceLoader : initialization starting.
<br> [INFO] ClasspathResourceLoader : initialization complete.
<br> [INFO] ResourceCache : initialized. (class
<br>org.apache.velocity.runtime.resource.ResourceCacheImpl)
<br> [INFO] Default ResourceManager initialization complete.
<br> [INFO] Loaded System Directive:
<br>org.apache.velocity.runtime.directive.Literal
<br> [INFO] Loaded System Directive:
<br>org.apache.velocity.runtime.directive.Macro
<br> [INFO] Loaded System Directive:
<br>org.apache.velocity.runtime.directive.Parse
<br> [INFO] Loaded System Directive:
<br>org.apache.velocity.runtime.directive.Include
<br> [INFO] Loaded System Directive:
<br>org.apache.velocity.runtime.directive.Foreach
<br> [INFO] Created: 20 parsers.
<br> [INFO] Velocimacro : initialization starting.
<br> [INFO] Velocimacro : adding VMs from VM library template :
<br>VM_global_library.vm
<br> [ERROR] ResourceManager : unable to find resource
<br>'VM_global_library.vm' in any resource loader.
<br> [INFO] Velocimacro : error using VM library template
<br>VM_global_library.vm : org.apache.velocity.exception.ResourceNotFoundEx
<br> ception: Unable to find resource 'VM_global_library.vm'
<br> [INFO] Velocimacro : VM library template macro registration
<br>complete.
<br> [INFO] Velocimacro : allowInline = true : VMs can be defined
<br>inline in templates
<br> [INFO] Velocimacro : allowInlineToOverride = false : VMs defined
<br>inline may NOT replace previous VM definitions
<br> [INFO] Velocimacro : allowInlineLocal = false : VMs defined
<br>inline will be global in scope if allowed.
<br> [INFO] Velocimacro : initialization complete.
<br> [INFO] Velocity successfully started.
<br> [INFO] Preparing surefire-report:report
<br> [INFO] [resources:resources]
<br> [INFO] Using default encoding to copy filtered resources.
<br> [INFO] [compiler:compile]
<br> [INFO] Nothing to compile - all classes are up to date
<br> [INFO] [resources:testResources]
<br> [INFO] Using default encoding to copy filtered resources.
<br> [INFO] [compiler:testCompile]
<br> [INFO] Nothing to compile - all classes are up to date
<br> [INFO] [surefire:test]
<br> [INFO] Surefire report directory:
<br>C:\workspace\shares-trunk\shares-domain\target\surefire-reports
<br>
<br> -------------------------------------------------------
<br> T E S T S
<br> -------------------------------------------------------
<br>
<br>
<br> ...<tests are run here - clears for confidentiality>...
<br>
<br> Results :
<br>
<br> Tests run: 54, Failures: 0, Errors: 0, Skipped: 0
<br>
<br> [INFO] Preparing surefire-report:report-only
<br> [INFO] No goals needed for project - skipping
<br> [INFO] [site:site]
<br> [WARNING] No URL defined for the project - decoration links will
<br>not be resolved
<br> [INFO] Skipped "Surefire Report" report, file
<br>"surefire-report.html" already exists for the English version.
<br> [INFO] Generating "Source Xref" report.
<br> [WARNING] Unable to locate Javadoc to link to - DISABLED
<br> [INFO] Generating "Test Source Xref" report.
<br> [INFO] Generating "Plugin Management" report.
<br> [INFO] Generating "Mailing Lists" report.
<br> [INFO] Generating "Continuous Integration" report.
<br> [INFO] Generating "Dependency Management" report.
<br> Downloading:
<br><a href="http://cfesstbos1:18080/nexus/content/groups/public/joda-time/joda-time-" target="_top" rel="nofollow">http://cfesstbos1:18080/nexus/content/groups/public/joda-time/joda-time-</a><br>jsptags/1.0.2/joda-time-jsptags-1.0.2.po
<br><<a href="http://cfesstbos1:18080/nexus/content/groups/public/joda-time/joda-time" target="_top" rel="nofollow">http://cfesstbos1:18080/nexus/content/groups/public/joda-time/joda-time</a><br>-jsptags/1.0.2/joda-time-jsptags-1.0.2.po>
<br> m
<br> [INFO] Generating "Project License" report.
<br> [INFO] Generating "Project Team" report.
<br> [INFO] Generating "Source Repository" report.
<br> [INFO] Generating "About" report.
<br> [INFO] Generating "Issue Tracking" report.
<br> [INFO] Generating "Project Summary" report.
<br> [INFO] Generating "Project Plugins" report.
<br> [INFO] Generating "Dependencies" report.
<br> [INFO] Generating "Surefire Report" report.
<br> [INFO]
<br>------------------------------------------------------------------------
<br> [INFO] BUILD SUCCESSFUL
<br> [INFO]
<br>------------------------------------------------------------------------
<br> [INFO] Total time: 30 seconds
<br> [INFO] Finished at: Mon Jan 05 11:01:09 EST 2009
<br> [INFO] Final Memory: 35M/74M
<br> [INFO]
<br>------------------------------------------------------------------------
<br>
<br> C:\workspace\shares-trunk\shares-domain>
<br><br>As you can see, my tests are executed twice, once as part of the
<br>"install" goal and a second time under the "site" goal. My POM has the
<br>following declaration:
<br><br> <plugin>
<br> <groupId>org.apache.maven.plugins</groupId>
<br> <artifactId>maven-surefire-report-plugin</artifactId>
<br> <version>2.4.3</version>
<br> <configuration>
<br> <formMode>once</formMode>
<br> <xrefLocation>
<br> ${project.reporting.outputDirectory}/../xref-test
<br> </xrefLocation>
<br> <reportSets>
<br> <reportSet>
<br> <reports>
<br> <report>report-only</report>
<br> </reports>
<br> </reportSet>
<br> </reportSets>
<br> <systemProperties>
<br> <property>
<br> <name>env.props.path</name>
<br> <value>c:/environment.properties</value>
<br> </property>
<br> </systemProperties>
<br> <argLine>
<br> -Denv.props.path=c:/environment.properties
<br> </argLine>
<br> </configuration>
<br> </plugin>
<br><br>No other surefire configuration is specified. Given the conversation
<br>occuring in SUREFIRE-257, I would assume that my tests would only be
<br>executed once. Should this be the case or is it expected that the tests
<br>are run for each goal?
<br>
<br>Thanks!
<br>Leo
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-20428666How to specify a specialized Juni TestRunner in surefire?2008-11-10T13:37:48Z2008-11-10T13:37:48ZPinaki Poddar-3Hi,
<br> Is there a way in surefire to specify a specialized JUnit TestRunner
<br>class?
<br> If yes, how?
<br><br> Regards --
<br><br>Pinaki Poddar
<br>Application & Integration Middleware
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-20263493Running both Junit4 and TestNG test cases in single Maven project2008-10-31T03:43:19Z2008-10-31T03:43:19ZNalaka Gamage-3In my project I need to have both both Junit4 and TestNG test cases and
<br>build using Maven2.
<br><br>However I found that when I run 'mvn test' goll, Junit4 test cases are
<br>ignored by Maven surfire plugin and only TestNG test cases are run. However
<br>when I use JUnit3 and TestNG test cases together, both the test cases are
<br>picked correctly. Can anyone propose a solution?
<br><br>I have append my POM file for your reference.
<br><br>Thanks in advance.
<br><br>- Nalaka
<br><project>
<br> <dependencies>
<br> <dependency>
<br> <groupId>junit</groupId>
<br> <artifactId>junit</artifactId>
<br> <version>4.3.1</version>
<br> <scope>test</scope>
<br> </dependency>
<br> <dependency>
<br> <groupId>org.testng</groupId>
<br> <artifactId>testng</artifactId>
<br> <version>5.8</version>
<br> <scope>test</scope>
<br> <classifier>jdk15</classifier>
<br> </dependency>
<br> </dependencies>
<br> <build>
<br> <plugins>
<br> <plugin>
<br> <groupId>org.apache.maven.plugins</groupId>
<br> <artifactId>maven-compiler-plugin</artifactId>
<br> <version>2.0.2</version>
<br> <configuration>
<br> <source>1.5</source>
<br> <target>1.5</target>
<br> </configuration>
<br> </plugin>
<br> </plugins>
<br> </build>
<br></project>
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Users-f21512.html" embed="fixTarget[21512]" target="_top" >Surefire - Users</a></p>tag:old.nabble.com,2006:post-18492320Surefilre configuration in profile won't override testSourceDirectory2008-07-16T10:15:21Z2008-07-16T10:15:21ZvickiHi,
<br>I have a test integration profile in the project's main pom, where I am trying to override the testSourceDirectory specified in the project tag. The surefire ignores the new value of the testSourceDirectory. Please, help.
<br>Vicki<p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-18412308Surefire reports on Junit Parameterized tests2008-07-11T13:50:31Z2008-07-11T13:50:31ZvickiHi,
<br>I am using surefire to run Junit's 4.x parametrerized tests. Surefire reports them as 1 test. If one test out of the parameterized bundle fails - Surefire fails the entire encapsulating test. Is there a way to set the Surefire in such a way that it reports all the parameterized tests?
<br>Thanks,
<br>Vicki<p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15419116Re: TestNG @BeforeMethod not working on subclasses.2008-02-11T11:39:31Z2008-02-11T11:39:31Zdfabulichroadtripryan wrote:
<br><br>> I am trying to test using TestNG/Spring/ and Maven Surefire 2.4.1.
<br>>
<br>> My test suite runs great from within eclipse as individual tests or as a
<br>> whole suite. When I try and run the tests in Maven Surefire, however, they
<br>> fail. It appears Surefire is not calling the correct @BeforeMethod method.
<br>> (When running the whole suite, or an individual test.)
<br><br>As you might expect, all Surefire does is call TestNG directly; we don't
<br>directly call @BeforeMethod, @Test, etc.
<br><br>> From testing it, it seems Surefire calls the @BeforeMethod in the base
<br>> class, but not in extending classes if they override it. I am extending
<br>> Springs AbstractTestNGSpringContextTests, and it is calling the base class
<br>> methods, not my extending classes @BeforeMethod and @AfterMethod.
<br>> Interestingly, it seems to call the correct @BeforeClass method.
<br><br>If you're having a problem with a TestNG test that you believe should be
<br>working, please file a bug on jira.codehaus.org; please also attach a
<br>minimal Maven project that reproduces the problem. Ideally a test like
<br>this should be nothing more than a pom.xml file and a couple of TestNG
<br>classes in appropriate directories. (Please don't include any spring
<br>classes in this test case unless it's absolutely necessary to do so.)
<br><br>-Dan
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15418799TestNG @BeforeMethod not working on subclasses.2008-02-11T11:30:16Z2008-02-11T11:30:16ZroadtripryanI am trying to test using TestNG/Spring/ and Maven Surefire 2.4.1.
<br><br>My test suite runs great from within eclipse as individual tests or as a whole suite. When I try and run the tests in Maven Surefire, however, they fail. It appears Surefire is not calling the correct @BeforeMethod method. (When running the whole suite, or an individual test.)
<br><br>From testing it, it seems Surefire calls the @BeforeMethod in the base class, but not in extending classes if they override it. I am extending Springs AbstractTestNGSpringContextTests, and it is calling the base class methods, not my extending classes @BeforeMethod and @AfterMethod. Interestingly, it seems to call the correct @BeforeClass method.
<br><br>Any ideas?
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15324832Re: testng, surefire, and forkMode=always2008-02-06T16:15:23Z2008-02-06T16:15:23ZBenjamin Bentmann<blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">brettporter wrote:</div>
<div class="quote-message">I don't know why people even use it with JUnit
</div>
</div></blockquote>
I am not really after forking all my tests, they already take long enough... As Dan already guessed, I was more after a way to hack me through to single-class test-suites. If forkMode=always would have worked, I would have argued, "well, I want this behavior but without the JVM forking, please" ;-)
<br><br><br>Benjamin Bentmann
<br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15324776Re: testng, surefire, and forkMode=always2008-02-06T16:10:32Z2008-02-06T16:10:32ZBenjamin Bentmann<blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">What should I do here? Put in the hack and let the user figure out the
<br>TestNGException if they try to use multi-class dependent methods?</div>
</div></blockquote>
Please, give us the exception from TestNG if the setup is invalid, give us the mere test results if the setup is valid because no inter-class dependencies exist. I mean somebody who can setup a config file for a build tool is smart enough to figure out what an exception means or do you disagree? Just in case, a few lines of doc about the forkMode=always setting should successfully shield Surefire from users filling in unjustified issues.
<br><br>To me, it seems this discussion is primarily concerned about making Surefire bullet-proof against bad user config. As for me, I have this slightly arrogant attitude to feel superior to a machine and hence always feel kind of angry if a program wants to tell what I should better not do because it might fail.
<br><br><br>Benjamin Bentmann<p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15324537Re: [testng-users] testng, surefire, and forkMode=always2008-02-06T15:52:41Z2008-02-06T15:52:41ZBenjamin Bentmann<blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">Cédric Beust ♔ wrote:</div>
<div class="quote-message">Creating a TestNG instance for every @Test method? Seems completely useless
<br>to me.
</div>
</div></blockquote>
The intent is not to fork once per test method but once per test class. Just like I can say in the TestNG Eclipse plugin "run class XY" instead of selecting a group.
<br><br><br>Benjamin Bentmann<p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15323908Re: testng, surefire, and forkMode=always2008-02-06T15:08:17Z2008-02-06T15:08:17ZbrettporterMy inclination would be towards just an error that says its not
<br>supported under TestNG.
<br><br>I don't know why people even use it with JUnit, except when they have
<br>memory leaks in their tests, or tests that affect the state of the JVM
<br>that they don't tear down :)
<br><br>- Brett
<br><br>On 07/02/2008, at 9:59 AM, Dan Fabulich wrote:
<br><div class='shrinkable-quote'><br>>
<br>> Benjamin Bentmann has filed bug SUREFIRE-446 <a href="http://jira.codehaus.org/browse/SUREFIRE-446" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-446</a><br>>
<br>> Maven Surefire provides a facility for various "fork modes":
<br>>
<br>> * forkMode=once: The default mode, spawns a separate TestNG process
<br>> (like the TestNG Ant task)
<br>>
<br>> * forkMode=never: run TestNG in the same process as Maven, in an
<br>> isolated classloader
<br>>
<br>> * forkMode=always: launch TestNG N times for N tests; if one test
<br>> segfaults or calls System.exit or something, the other tests can run
<br>> to completion safely.
<br>>
<br>> I didn't test forkMode=always with TestNG in Surefire 2.4, and sure
<br>> enough it doesn't work at all. (It runs the tests but reports 0
<br>> tests passed/failed.)
<br>>
<br>> It does work basically OK for JUnit, where classes are normally
<br>> isolated from each other, but I'm not sure it even makes sense for
<br>> TestNG tests, where one test may depend on tests in other classes
<br>> (multi-class dependent methods).
<br>>
<br>> I can kind of hack it so it works OK if you don't have multi-class
<br>> dependent methods, but if you DO have multi-class dependent methods,
<br>> it fails with a TestNGException: "MyTest.testFoo() is depending on
<br>> nonexistent method otherClassSetUp".
<br>>
<br>> What should I do here? Put in the hack and let the user figure out
<br>> the TestNGException if they try to use multi-class dependent
<br>> methods? Fail with a clean error if the user tries to use
<br>> forkMode=always with TestNG? Something else?
<br>>
<br>> Should forkMode=always be a feature of TestNG somehow?
<br>>
<br>> -Dan
</div><br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15323895Re: [testng-users] testng, surefire, and forkMode=always2008-02-06T15:04:40Z2008-02-06T15:04:40ZCédric Beust ♔Creating a TestNG instance for every @Test method? Seems completely useless
<br>to me.
<br><br>Especially when you can control the threading behavior with the parallel
<br>attribute.
<br><br>--
<br>Cédric
<br><br><br>On Wed, Feb 6, 2008 at 2:59 PM, Dan Fabulich <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=15323895&i=0" target="_top" rel="nofollow">dan@...</a>> wrote:
<br><div class='shrinkable-quote'><br>>
<br>>
<br>> Benjamin Bentmann has filed bug SUREFIRE-446
<br>> <a href="http://jira.codehaus.org/browse/SUREFIRE-446" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-446</a><br>>
<br>> Maven Surefire provides a facility for various "fork modes":
<br>>
<br>> * forkMode=once: The default mode, spawns a separate TestNG process (like
<br>> the TestNG Ant task)
<br>>
<br>> * forkMode=never: run TestNG in the same process as Maven, in an isolated
<br>> classloader
<br>>
<br>> * forkMode=always: launch TestNG N times for N tests; if one test
<br>> segfaults or calls System.exit or something, the other tests can run to
<br>> completion safely.
<br>>
<br>> I didn't test forkMode=always with TestNG in Surefire 2.4, and sure enough
<br>> it doesn't work at all. (It runs the tests but reports 0 tests
<br>> passed/failed.)
<br>>
<br>> It does work basically OK for JUnit, where classes are normally isolated
<br>> from each other, but I'm not sure it even makes sense for TestNG tests,
<br>> where one test may depend on tests in other classes (multi-class dependent
<br>> methods).
<br>>
<br>> I can kind of hack it so it works OK if you don't have multi-class
<br>> dependent methods, but if you DO have multi-class dependent methods, it
<br>> fails with a TestNGException: "MyTest.testFoo() is depending on
<br>> nonexistent method otherClassSetUp".
<br>>
<br>> What should I do here? Put in the hack and let the user figure out the
<br>> TestNGException if they try to use multi-class dependent methods? Fail
<br>> with a clean error if the user tries to use forkMode=always with TestNG?
<br>> Something else?
<br>>
<br>> Should forkMode=always be a feature of TestNG somehow?
<br>>
<br>> -Dan
<br>>
<br>> --~--~---------~--~----~------------~-------~--~----~
<br>> You received this message because you are subscribed to the Google Groups
<br>> "testng-users" group.
<br>> To post to this group, send email to <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=15323895&i=1" target="_top" rel="nofollow">testng-users@...</a>
<br>> To unsubscribe from this group, send email to
<br>> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=15323895&i=2" target="_top" rel="nofollow">testng-users-unsubscribe@...</a>
<br>> For more options, visit this group at
<br>> <a href="http://groups.google.com/group/testng-users?hl=en" target="_top" rel="nofollow">http://groups.google.com/group/testng-users?hl=en</a><br>> -~----------~----~----~----~------~----~------~--~---
<br>>
<br>>
<br></div><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15323782testng, surefire, and forkMode=always2008-02-06T14:59:03Z2008-02-06T14:59:03Zdfabulich<br>Benjamin Bentmann has filed bug SUREFIRE-446
<br><a href="http://jira.codehaus.org/browse/SUREFIRE-446" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-446</a><br><br>Maven Surefire provides a facility for various "fork modes":
<br><br>* forkMode=once: The default mode, spawns a separate TestNG process (like
<br>the TestNG Ant task)
<br><br>* forkMode=never: run TestNG in the same process as Maven, in an isolated
<br>classloader
<br><br>* forkMode=always: launch TestNG N times for N tests; if one test
<br>segfaults or calls System.exit or something, the other tests can run to
<br>completion safely.
<br><br>I didn't test forkMode=always with TestNG in Surefire 2.4, and sure enough
<br>it doesn't work at all. (It runs the tests but reports 0 tests
<br>passed/failed.)
<br><br>It does work basically OK for JUnit, where classes are normally isolated
<br>from each other, but I'm not sure it even makes sense for TestNG tests,
<br>where one test may depend on tests in other classes (multi-class dependent
<br>methods).
<br><br>I can kind of hack it so it works OK if you don't have multi-class
<br>dependent methods, but if you DO have multi-class dependent methods, it
<br>fails with a TestNGException: "MyTest.testFoo() is depending on
<br>nonexistent method otherClassSetUp".
<br><br>What should I do here? Put in the hack and let the user figure out the
<br>TestNGException if they try to use multi-class dependent methods? Fail
<br>with a clean error if the user tries to use forkMode=always with TestNG?
<br>Something else?
<br><br>Should forkMode=always be a feature of TestNG somehow?
<br><br>-Dan
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15142735Re: Test Suites, Ant, Surefire, and JunitReport2008-01-28T11:00:17Z2008-01-28T11:00:17ZdfabulichBenjamin Bentmann wrote:
<br><br>> TestNG is not designed only for the fancy tests, it can run the simple unit
<br>> tests, too. Dan, to me it looks you/Surefire are pushing something like "If
<br>> you want unit tests, use JUnit, if you want function/integration tests, use
<br>> TestNG" by punishing those that employ TestNG for the simple use-case. I am
<br>> neither a JUnit nor TestNG fanatic but I can't follow your XOR-like argument
<br>> here.
<br><br>You CAN use TestNG for unit tests; the simple use case does work. It's
<br>just not as nice as you might wish it were. My argument is that this
<br>isn't very important because there's a better alternative available for
<br>unit tests.
<br><br>-Dan
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15138325Re: Test Suites, Ant, Surefire, and JunitReport2008-01-28T07:54:58Z2008-01-28T07:54:58ZBenjamin Bentmann<blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">As I said, I don't think you're using the right tool for the job here.
</div>
</div></blockquote>
Hm, I am not really in the mood to start off a discussion with my project manager ;-)
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">You can get the exact behavior you want from JUnit.
</div>
</div></blockquote>
And I could get it from TestNG if I passed each class in as an individual suite, couldn't I?
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">Again, if you want class-level logging, I encourage you to take up the
<br>matter with the TestNG team.
</div>
</div></blockquote>
Might be a good idea. As some kind of explanation why I started this discussion on the Surefire corner at all: Surefire 2.3.1 did what I wanted so the first impression after updating to Surefire 2.4 was that it got messed up.
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">Doesn't this option smell bad to you?
</div>
</div></blockquote>
It's a rather technical/sophisticated option, I agree about that. But aren't "useSystemClassLoader", "childDelegation" and "forkMode" likewise? Whoever develops tests needs to have proper knowledge about their requirements on the execution runtime and hence, needs to configure the test runner appropriately. The same would apply for this option, default it to "all", let the experts change it to meet their special needs.
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">Instead of making this option, wouldn't it be nice if it just logged
<br>correctly? If TestNG notified us when to log, we could log anything we
<br>wanted.
</div>
</div></blockquote>
I fear having just another lister in the TestNG framework is not enough. The problem I see here is the following: If Sufefire passes all test classes (say A and B) as a single suite to TestNG, what would TestNG prevent from running them like this:
<br> A.testOne
<br> B.testOne
<br> A.testTwo
<br> B.testTwo
<br> A.testThree
<br> B.testThree
<br>Our test methods have no dependencies that would require a deterministic order. Even if TestNG exposed events for starting/stopping a test class, I still would not get a guarantee that all methods from a single class are run without inter-leaving with methods from other classes to give me that class-level testing feeling. I hardly can see an argument how one could request a change on this behavior. A suite is the outer-most group of tests and as long as I do not specify dependencies, TestNG is free in ordering them.
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">Surefire is meant to be a wrapper around the underlying test framework;
</div>
</div></blockquote>
No problems with that. But it's Surefire and not the test framework that aggregates my test classes into a single suite. I am basically asking for an option to control the way how Surefire assembles the suites *before* it passes control to the test framework.
<br><br><br>Benjamin Bentmann<p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15136338Re: Test Suites, Ant, Surefire, and JunitReport2008-01-28T06:45:52Z2008-01-28T06:45:52ZBenjamin Bentmann<blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">Inter-class dependencies are one of TestNG's fundamental features
</div>
</div></blockquote>
Use of this feature is optional. For my sake, call it an "abuse" of TestNG if I run that simple unit tests without inter-class dependencies ;-)
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">[...] but TestNG is meant to support functional/integration tests.
</div>
</div></blockquote>
If I may requote the TestNG marketing slogan, again:
<br>> TestNG is designed to cover all categories of tests: *unit*, [...]
<br><br>TestNG is not designed only for the fancy tests, it can run the simple unit tests, too. Dan, to me it looks you/Surefire are pushing something like "If you want unit tests, use JUnit, if you want function/integration tests, use TestNG" by punishing those that employ TestNG for the simple use-case. I am neither a JUnit nor TestNG fanatic but I can't follow your XOR-like argument here.
<br><br><br>Benjamin Bentmann<p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15127052Re: Test Suites, Ant, Surefire, and JunitReport2008-01-27T18:06:03Z2008-01-27T18:06:03ZdfabulichBenjamin Bentmann wrote:
<br><br>> But what I really want is to avoid bothering with suites at all. Sure,
<br>> TestNG has a powerful suite concept, nice, maybe we may want to use it
<br>> someday in the future but for now, we have just the simple use-case with
<br>> non-related test classes, i.e. we are happy by grouping at the class
<br>> level.
<br><br>As I said, I don't think you're using the right tool for the job here.
<br>TestNG is meant to support all that fancy stuff, and behaves a little
<br>differently for that reason. You can get the exact behavior you want from
<br>JUnit.
<br><br>> We have no suite(s), only a directory with some unrelated test classes.
<br>> Now Surefire goes ahead and creates a suite, automatically, without my
<br>> control, assuming the entire directory is a single suite. This
<br>> assumption does not fit our needs.
<br><br>That's fair, but I claim that this assumption is a good one for people who
<br>are using TestNG, most of whom pick TestNG over JUnit because they need to
<br>do that weird stuff.
<br><br>Again, if you want class-level logging, I encourage you to take up the
<br>matter with the TestNG team. TestNG isn't meant to work this way today.
<br><div class='shrinkable-quote'><br>> dfabulich wrote:
<br>>>
<br>>> That option sounds really confusing. What would you call it? How would
<br>>> you document it? You'd have to know a lot about the guts of Surefire and
<br>>> TestNG for an option like that to make much sense.
<br>>>
<br>> Could think of something like "suiteMode=class|package|all":
<br>> This option controls how Surefire will assemble your test classes into a
<br>> suite. "class" means to run each class individually in its own suite.
<br>> "package" groups all classes from the same package into a separate suite.
<br>> "all" will aggregate all found test classes in a single suite. Each created
<br>> suite will then be passed to the testing framework for execution and will be
<br>> run independently of the other suites. Note that test classes that have
<br>> dependencies on each other, must be executed in the same suite. Therefore,
<br>> the values "class" and "package" should be used with care because Surefire
<br>> itself cannot detect such inter-class dependencies.
</div><br>Doesn't this option smell bad to you? It does to me!
<br><br>Instead of making this option, wouldn't it be nice if it just logged
<br>correctly? If TestNG notified us when to log, we could log anything we
<br>wanted.
<br><br>Surefire is meant to be a wrapper around the underlying test framework;
<br>it's not meant to provide features that the frameworks lack. I think this
<br>is one of those cases where trying to provide the feature in Surefire
<br>would be way harder, and would also be fighting the framework.
<br><br>> Say one starts of with JUnit, where Surefire executes classes
<br>> individually. Then one goes and converts his tests to use TestNG (they
<br>> have a tool for this). Conceptually, you have the same test design, but
<br>> now Surefire insists on running all classes in a single job/black-box. I
<br>> am sorry, but I still find this odd since I am lacking the required
<br>> knowledge about the guts of Surefire and TestNG ;-)
<br><br>For the record, I agree that the difference between TestNG and JUnit is
<br>strange. I claim that we only find it odd BECAUSE we have painfully
<br>learned about these guts... most people don't notice or care about the
<br>difference.
<br><br>-Dan
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15126834Re: Test Suites, Ant, Surefire, and JunitReport2008-01-27T17:37:13Z2008-01-27T17:37:13ZdfabulichJason van Zyl wrote:
<br><br>> Correct me if I'm wrong here Benjamin but I think he's talking about
<br>> stating what tests you want to run, and how they are grouped
<br>> declaratively. As a best practice don't use suites and come up with a
<br>> way in Surefire to group things and avoid grouping of things in code.
<br><br>Inter-class dependencies are one of TestNG's fundamental features; it's
<br>arguably the most important architectural difference between TestNG and
<br>JUnit. As Benjamin pointed out, they are declarative.
<br><br>> I think it would just be a practice for class to be self-contained, test
<br>> one thing, and allow wiring up of sets/groups externally.
<br><br>That's a good practice for unit tests, but TestNG is meant to support
<br>functional/integration tests. Unlike unit tests, which are meant to be
<br>isolated and test one thing, TestNG tests are supposed to test more than
<br>one thing together, and are therefore designed to inter-depend on one
<br>another.
<br><br>It's a pretty significant philosophical shift; the TestNG guys wrote a
<br>whole book about it, blah blah blah. :-)
<br><br>Dependent test methods
<br><a href="http://beust.com/weblog/archives/000171.html" target="_top" rel="nofollow">http://beust.com/weblog/archives/000171.html</a><br>Are dependent test methods really evil?
<br><a href="http://beust.com/weblog/archives/000259.html" target="_top" rel="nofollow">http://beust.com/weblog/archives/000259.html</a><br>Why unit tests are disappearing
<br><a href="http://beust.com/weblog/archives/000319.html" target="_top" rel="nofollow">http://beust.com/weblog/archives/000319.html</a><br><br>> If this model breaks down then maybe, as I've asked before, it makes sense to
<br>> create separate plugins for the various test tools.
<br><br>I'm still on the fence about that, but regardless I don't think this is a
<br>good candidate reason. Surefire has to handle this problem in the case of
<br>JUnit suites as well as TestNG suites; the only difference is that in
<br>JUnit this problem is less important because we think people mostly
<br>shouldn't write unit tests like that.
<br><br>-Dan
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15126606Re: Test Suites, Ant, Surefire, and JunitReport2008-01-27T17:10:49Z2008-01-27T17:10:49ZJason van Zyl-2<br>On 27-Jan-08, at 12:31 PM, Benjamin Bentmann wrote:
<br><div class='shrinkable-quote'><br>>
<br>>
<br>> Jason van Zyl-2 wrote:
<br>>>
<br>>> Correct me if I'm wrong here Benjamin but I think he's talking about
<br>>> stating what tests you want to run, and how they are grouped
<br>>> declaratively. As a best practice don't use suites and come up with a
<br>>> way in Surefire to group things and avoid grouping of things in code.
<br>>>
<br>>
<br>> Not quite sure if I understand you correctly. A TestNG suite XML
<br>> file is
<br>> quite declarative so I already have the possibility to declaratively
<br>> select
<br>> and group tests. In fear that I can't get Surefire where I wished
<br>> (one-class-per-suite), I opened <a href="http://jira.codehaus.org/browse/SUREFIRE-438" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-438</a><br>> which would allow to easily feed a bunch of suite XML files into the
<br>> plugin.
<br>>
<br>> But what I really want is to avoid bothering with suites at all.
</div><br>Sorry if I didn't explain it well but we are in 100% agreement. Make
<br>tests simple, no suites, and group declaratively.
<br><div class='shrinkable-quote'><br>> Sure,
<br>> TestNG has a powerful suite concept, nice, maybe we may want to use it
<br>> someday in the future but for now, we have just the simple use-case
<br>> with
<br>> non-related test classes, i.e. we are happy by grouping at the class
<br>> level.
<br>> Explicitly maintaining suites, either in code or via XML, would just
<br>> be lost
<br>> worktime here (although I already thought about a plugin that
<br>> automatically
<br>> creates single-class suites from the existing test classes and
<br>> prepares the
<br>> dozens of suite files before the actual Surefire run...)
<br>>
<br>> We have no suite(s), only a directory with some unrelated test
<br>> classes. Now
<br>> Surefire goes ahead and creates a suite, automatically, without my
<br>> control,
<br>> assuming the entire directory is a single suite. This assumption
<br>> does not
<br>> fit our needs.
<br>>
<br>>
<br>> dfabulich wrote:
<br>>>
<br>>> That option sounds really confusing. What would you call it? How
<br>>> would
<br>>> you document it? You'd have to know a lot about the guts of
<br>>> Surefire and
<br>>> TestNG for an option like that to make much sense.
<br>>>
<br>> Could think of something like "suiteMode=class|package|all":
<br>> This option controls how Surefire will assemble your test classes
<br>> into a
<br>> suite. "class" means to run each class individually in its own suite.
<br>> "package" groups all classes from the same package into a separate
<br>> suite.
<br>> "all" will aggregate all found test classes in a single suite. Each
<br>> created
<br>> suite will then be passed to the testing framework for execution and
<br>> will be
<br>> run independently of the other suites. Note that test classes that
<br>> have
<br>> dependencies on each other, must be executed in the same suite.
<br>> Therefore,
<br>> the values "class" and "package" should be used with care because
<br>> Surefire
<br>> itself cannot detect such inter-class dependencies.
<br>>
<br>>
<br>>
<br>> dfabulich wrote:
<br>>>
<br>>> "TestNG is designed to cover all categories of tests: unit, [...]"
<br>>> [...]
<br>>> If all you're doing is unit tests, you may prefer to use JUnit 4. :-)
<br>>>
<br>>
<br>> Well, no comments ;-)
<br>>
<br>> Say one starts of with JUnit, where Surefire executes classes
<br>> individually.
<br>> Then one goes and converts his tests to use TestNG (they have a tool
<br>> for
<br>> this). Conceptually, you have the same test design, but now Surefire
<br>> insists
<br>> on running all classes in a single job/black-box. I am sorry, but I
<br>> still
<br>> find this odd since I am lacking the required knowledge about the
<br>> guts of
<br>> Surefire and TestNG ;-)
<br>>
<br>> Regards,
<br>>
<br>>
<br>> Benjamin Bentmann
<br>> --
<br>> View this message in context: <a href="http://www.nabble.com/Test-Suites%2C-Ant%2C-Surefire%2C-and-JunitReport-tp15076378p15123679.html" target="_top">http://www.nabble.com/Test-Suites%2C-Ant%2C-Surefire%2C-and-JunitReport-tp15076378p15123679.html</a><br>> Sent from the Surefire - Developer mailing list archive at Nabble.com.
<br>>
</div><br>Thanks,
<br><br>Jason
<br><br>----------------------------------------------------------
<br>Jason van Zyl
<br>Founder, Apache Maven
<br>jason at sonatype dot com
<br>----------------------------------------------------------
<br><br>Simplex sigillum veri. (Simplicity is the seal of truth.)
<br><br><br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15123679Re: Test Suites, Ant, Surefire, and JunitReport2008-01-27T12:31:59Z2008-01-27T12:31:59ZBenjamin Bentmann<blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">Jason van Zyl-2 wrote:</div>
<div class="quote-message">Correct me if I'm wrong here Benjamin but I think he's talking about
<br>stating what tests you want to run, and how they are grouped
<br>declaratively. As a best practice don't use suites and come up with a
<br>way in Surefire to group things and avoid grouping of things in code.
</div>
</div></blockquote>
Not quite sure if I understand you correctly. A TestNG suite XML file is quite declarative so I already have the possibility to declaratively select and group tests. In fear that I can't get Surefire where I wished (one-class-per-suite), I opened <a href="http://jira.codehaus.org/browse/SUREFIRE-438" target="_top" rel="nofollow">http://jira.codehaus.org/browse/SUREFIRE-438</a> which would allow to easily feed a bunch of suite XML files into the plugin.
<br><br>But what I really want is to avoid bothering with suites at all. Sure, TestNG has a powerful suite concept, nice, maybe we may want to use it someday in the future but for now, we have just the simple use-case with non-related test classes, i.e. we are happy by grouping at the class level. Explicitly maintaining suites, either in code or via XML, would just be lost worktime here (although I already thought about a plugin that automatically creates single-class suites from the existing test classes and prepares the dozens of suite files before the actual Surefire run...)
<br><br>We have no suite(s), only a directory with some unrelated test classes. Now Surefire goes ahead and creates a suite, automatically, without my control, assuming the entire directory is a single suite. This assumption does not fit our needs.
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">That option sounds really confusing. What would you call it? How would
<br>you document it? You'd have to know a lot about the guts of Surefire and
<br>TestNG for an option like that to make much sense.
</div>
</div></blockquote>
Could think of something like "suiteMode=class|package|all":
<br><i>This option controls how Surefire will assemble your test classes into a suite. "class" means to run each class individually in its own suite. "package" groups all classes from the same package into a separate suite. "all" will aggregate all found test classes in a single suite. Each created suite will then be passed to the testing framework for execution and will be run independently of the other suites. Note that test classes that have dependencies on each other, must be executed in the same suite. Therefore, the values "class" and "package" should be used with care because Surefire itself cannot detect such inter-class dependencies.
<br></i><br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">dfabulich wrote:</div>
<div class="quote-message">"TestNG is designed to cover all categories of tests: unit, [...]"
<br>[...]
<br>If all you're doing is unit tests, you may prefer to use JUnit 4. :-)
</div>
</div></blockquote>
Well, no comments ;-)
<br><br>Say one starts of with JUnit, where Surefire executes classes individually. Then one goes and converts his tests to use TestNG (they have a tool for this). Conceptually, you have the same test design, but now Surefire insists on running all classes in a single job/black-box. I am sorry, but I still find this odd since I am lacking the required knowledge about the guts of Surefire and TestNG ;-)
<br><br>Regards,
<br><br><br>Benjamin Bentmann<p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15122556Re: Test Suites, Ant, Surefire, and JunitReport2008-01-27T10:53:58Z2008-01-27T10:53:58ZJason van Zyl-2<br>On 25-Jan-08, at 10:59 AM, Dan Fabulich wrote:
<br><div class='shrinkable-quote'><br>> Benjamin Bentmann wrote:
<br>>
<br>>> I see. Now, if the user can provide an option to Surefire,
<br>>> indicating that the unit tests do not have inter-class
<br>>> dependencies, wouldn't this allow to get back the old behavior?
<br>>> I.e. enabling Surefire to pass test classes individually into TestNG?
<br>>
<br>> That option sounds really confusing. What would you call it? How
<br>> would you document it? You'd have to know a lot about the guts of
<br>> Surefire and TestNG for an option like that to make much sense.
<br>>
</div><br>Correct me if I'm wrong here Benjamin but I think he's talking about
<br>stating what tests you want to run, and how they are grouped
<br>declaratively. As a best practice don't use suites and come up with a
<br>way in Surefire to group things and avoid grouping of things in code.
<br><br>If this model breaks down then maybe, as I've asked before, it makes
<br>sense to create separate plugins for the various test tools.
<br><br>> Not to mention that it'd be error prone.
<br><br>I think it would just be a practice for class to be self-contained,
<br>test one thing, and allow wiring up of sets/groups externally.
<br><div class='shrinkable-quote'><br>> Developer X would assess the tests, verifying that there are no
<br>> inter-class dependencies, and turn on the flag. Later, developer Y
<br>> could introduce an inter-class dependency and go to run the tests,
<br>> but find that it's not working. It'd be hard to figure out why
<br>> without knowing a lot about how Surefire is doing its job.
<br>>
<br>> More generally, I don't think class-level reporting really makes
<br>> sense in TestNG as it's designed today. Right now, TestNG provides
<br>> listeners with the following notifications:
<br>>
<br>> onStart
<br>> onTestStart
<br>> onTestSuccess
<br>> onTestFailure
<br>> onTestSkipped
<br>> onTestFailedButWithinSuccessPercentage
<br>> onConfigurationSuccess
<br>> onConfigurationFailure
<br>> onConfigurationSkip
<br>> onFinish
<br>>
<br>> Note that there's nothing here about "onClassStart/Finish" or even
<br>> "onGroupStart/Finish".
<br>>
<br>> If TestNG exposed a listener, we could easily attach to it, but I
<br>> see that as a feature of TestNG. We could certainly try to write
<br>> TestNG features in Surefire, but the TestNG code knows best whether,
<br>> for example, your classes have any inter-dependencies, when your
<br>> classes start and stop, and more generally how to construct a full
<br>> test run comprised of suites, groups, classes and tests.
<br>>
<br>> Surefire shouldn't be in the business of reimplementing that logic,
<br>> especially since we expect this feature to be generally useful. It
<br>> should be a part of TestNG, if it matters at all.
<br>>
<br>>> I would not have any problems with a single file per method.
<br>>> Indeed, I think
<br>>> I like the idea: Test methods are the units that succeed or fail,
<br>>> so if test
<br>>> method TestClass.testFoo() fails, I would investigate
<br>>> TestClass.testFoo-output.txt and can easily concentrate on its output
<br>>> without any need to skip over the output from all the other
<br>>> successful
<br>>> methods. I think that would be worth another option ;-)
<br>>
<br>> "TestNG is designed to cover all categories of tests: unit,
<br>> functional, end-to-end, integration, etc..."
<br>>
<br>> In TestNG, especially when one method @dependsOn another method, the
<br>> test failures aren't units, and aren't even meant to be units.
<br>> (This can be a huge benefit for slow integration tests, where
<br>> earlier tests leave behind fixtures that later tests are meant to
<br>> consume.)
<br>>
<br>> If all you're doing is unit tests, you may prefer to use JUnit 4. :-)
<br>>
<br>> Anyway, in thinking about it a little harder I realized that we
<br>> can't even necessarily split the stdout into one file per method,
<br>> since the tests may be run in parallel!
<br>>
<br>> I'm pretty sure we can do that logging thing, at least. :-p
<br>>
<br>> -Dan
</div><br>Thanks,
<br><br>Jason
<br><br>----------------------------------------------------------
<br>Jason van Zyl
<br>Founder, Apache Maven
<br>jason at sonatype dot com
<br>----------------------------------------------------------
<br><br>We know what we are, but know not what we may be.
<br><br>-- Shakespeare
<br><br><br><br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15100764Re: SUREFIRE-428: integration test to check class path order2008-01-25T17:34:57Z2008-01-25T17:34:57ZBrett Porter-7It would seem to be best in Surefire to me - the assertion in Maven
<br>should really be covered by a unit test?
<br><br>On 26/01/2008, at 5:25 AM, Dan Fabulich wrote:
<br><div class='shrinkable-quote'><br>>
<br>> Class path ordering is tricky, and changed recently. Benjamin
<br>> Bentmann provides a regression test in SUREFIRE-428.
<br>>
<br>> However, I'm not certain if I should apply it in Surefire, because
<br>> we use those integration tests to verify backwards compatibility
<br>> with older versions of Maven.
<br>>
<br>> My initial thought was that perhaps the test should be applied in
<br>> core-integration-testing; there are a few small Surefire integration
<br>> tests there (which I think is correct), and I think this could join
<br>> them to prevent regression for MNG-3118.
<br>>
<br>> Then again, since it's fundamentally an integration test, it doesn't
<br>> specifically belong to either side, so maybe Surefire is the best
<br>> place to install it...?
<br>>
<br>> Thoughts?
<br>>
<br>> -Dan
</div><br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>tag:old.nabble.com,2006:post-15100575Re: [testng-users] Re: Test Suites, Ant, Surefire, and JunitReport2008-01-25T15:15:35Z2008-01-25T15:15:35ZSteve Loughran-7On Jan 25, 2008 5:04 PM, Dan Fabulich <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=15100575&i=0" target="_top" rel="nofollow">dan@...</a>> wrote:
<div class='shrinkable-quote'><br>>
<br>> Steve Loughran wrote:
<br>>
<br>> >> What I propose is that, in order to avoid destroying information, Surefire
<br>> >> should generate XML that looks like Example 7 (all-in-one-file), and not
<br>> >> try to fake it to look like Example 2 (one-file-per-class). (TestNG's
<br>> >> junit-like output also generates files like Example 7.) However, when it
<br>> >> comes time to generate an HTML report, surefire-reports will discard
<br>> >> suite-level information, and treat large suites like Example 7 as if they
<br>> >> had been presented in separate files like Example 2. I'd argue that all
<br>> >> of the other JunitReport-like tools (including Ant) should probably follow
<br>> >> the same lead.
<br>> >
<br>> > I think that if everyone else has a bug, its hard to call it a bug. More
<br>> > a presentation choice :)
<br>>
<br>> I'd thought somebody might say that! :-) Still, do you think TestNG and
<br>> other tools should therefore generate multiple XML files, to be compatible
<br>> with the other (arguably broken) reporting tools?
</div><br>No,
<br><br>* junit report will take other sets of XSL sheets than those built in
<br>to Ant's own JAR. This allows anyone to fix the XSLs without waiting
<br>for ant's (fairly mature) release cycle.
<br>* CI tools are a separate issue. Don't know there
<br><br>Looking at the reports -and trying to imagine a time where I ever get
<br>to stop making meta-RPM-RPMs (don't ask), I do want to push the test
<br>formats forward in a way everyone is happy. I really dont like the way
<br>the current stuff sticks summary info as toplevel attributes (time,
<br>results), as it stops us streaming out the XML as we go. I'm starting
<br>to wonder if we couldnt move to a new format, and have the option of
<br>XSL generation of the legacy stuff, purely for those tools that havent
<br>been upgraded in sync.
<br><br>-steve
<br><p>From forum: <a href="http://old.nabble.com/Surefire---Developer-f21513.html" embed="fixTarget[21513]" target="_top" >Surefire - Developer</a></p>