Order of test methods

[Tomás Pollak-2]

Hi all,

I'm new to TestNG and I have a question related to the order the test methods are run. I have seen that a similar question was asked here.

My problem is the following:
I'm testing a webapp and there is a table that shows some items sorted by descending id, so the last one inserted is always on the top. I have no way of filtering the table. I have some tests that work with this, so initially in each test I was inserting a new item. Since inserting the test data is slow and the tests don't always modify the data I thought of moving the data creation to a @SetupClass method.

However the following order can happen, for classes A and B:
1) Setup A
2) Test1 in A
3) Setup B <-- inserts a new row
4) Test 1 in B
5) Test 2 in A <-- fails, because step 3) modified the expected data.

I don't have any dependencies between the tests.

Also, I am running this with a general "All tests" suite from Eclipse. In this suite I'm including all the tests in the package. This is because I don't want to maintain the xml file every time a new test class is added (we have many many tests, and it would be painful to keep track). The thread I mentioned before suggests using different test elements, but I guess this would mean that I have to manually enter every test class in my xml file, right?

Is there a better way to tell TestNG to run all the tests in a class together if possible? (i.e. not interleaving tests from different classes)

Thanks in advance,
Tomas

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng-users@...
To unsubscribe from this group, send email to testng-users+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

[krishnan.1000]

You can use groups and dependsongroups annotation description to create a test hierarchy.

On Sat, Oct 24, 2009 at 7:33 PM, Tomás Pollak <[e-mail]> wrote:

Hi all,

I'm new to TestNG and I have a question related to the order the test methods are run. I have seen that a similar question was asked here.

My problem is the following:
I'm testing a webapp and there is a table that shows some items sorted by descending id, so the last one inserted is always on the top. I have no way of filtering the table. I have some tests that work with this, so initially in each test I was inserting a new item. Since inserting the test data is slow and the tests don't always modify the data I thought of moving the data creation to a @SetupClass method.

However the following order can happen, for classes A and B:
1) Setup A
2) Test1 in A
3) Setup B <-- inserts a new row
4) Test 1 in B
5) Test 2 in A <-- fails, because step 3) modified the expected data.

I don't have any dependencies between the tests.

Also, I am running this with a general "All tests" suite from Eclipse. In this suite I'm including all the tests in the package. This is because I don't want to maintain the xml file every time a new test class is added (we have many many tests, and it would be painful to keep track). The thread I mentioned before suggests using different test elements, but I guess this would mean that I have to manually enter every test class in my xml file, right? 

Is there a better way to tell TestNG to run all the tests in a class together if possible? (i.e. not interleaving tests from different classes)

Thanks in advance,
Tomas

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng-users@...
To unsubscribe from this group, send email to testng-users+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

[Cédric Beust ♔]

Hi Tomás,

I can't reproduce what you are seeing, here is the output of my test:

    test.tmp.A.beforeClassA() 2159159
        test.tmp.A.g1() 2159159
        test.tmp.A.g2() 2159159
    test.tmp.B.beforeClassB() 16658781
        test.tmp.B.g() 16658781

As you can see, TestNG is keeping the tests together.

Can you trim down your code and post it?

Thanks.

--
Cédric

On Sat, Oct 24, 2009 at 7:33 PM, Tomás Pollak <[e-mail]> wrote:

Hi all,

I'm new to TestNG and I have a question related to the order the test methods are run. I have seen that a similar question was asked here.

My problem is the following:
I'm testing a webapp and there is a table that shows some items sorted by descending id, so the last one inserted is always on the top. I have no way of filtering the table. I have some tests that work with this, so initially in each test I was inserting a new item. Since inserting the test data is slow and the tests don't always modify the data I thought of moving the data creation to a @SetupClass method.

However the following order can happen, for classes A and B:
1) Setup A
2) Test1 in A
3) Setup B <-- inserts a new row
4) Test 1 in B
5) Test 2 in A <-- fails, because step 3) modified the expected data.

I don't have any dependencies between the tests.

Also, I am running this with a general "All tests" suite from Eclipse. In this suite I'm including all the tests in the package. This is because I don't want to maintain the xml file every time a new test class is added (we have many many tests, and it would be painful to keep track). The thread I mentioned before suggests using different test elements, but I guess this would mean that I have to manually enter every test class in my xml file, right?

Is there a better way to tell TestNG to run all the tests in a class together if possible? (i.e. not interleaving tests from different classes)

Thanks in advance,
Tomas

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng-users@...
To unsubscribe from this group, send email to testng-users+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---

[Tomás Pollak-2]

Hi Cedric,

thanks for the reply, and sorry for the delay. I've been quite busy lately.
I'm attaching a simple project it took me a while to reproduce this, because at first it happened just like your test, where everything worked fine.

The attachment is a simple maven project that you should be able to import in Eclipse and run the AllTests.xml suite.

Something very odd that I've noticed while making tests is that the names of the methods do influence their order. For example, if you remove the underscores from their names it works as expected (believe or not)!!

Please tell me if you need any more information.
Thanks,
Tomas

2009/10/25 Cédric Beust ♔ <cbeust@...>

Hi Tomás,

I can't reproduce what you are seeing, here is the output of my test:

    test.tmp.A.beforeClassA() 2159159
        test.tmp.A.g1() 2159159
        test.tmp.A.g2() 2159159
    test.tmp.B.beforeClassB() 16658781
        test.tmp.B.g() 16658781

As you can see, TestNG is keeping the tests together.

Can you trim down your code and post it?

Thanks.

--
Cédric

On Sat, Oct 24, 2009 at 7:33 PM, Tomás Pollak <[e-mail]> wrote:

Hi all,

I'm new to TestNG and I have a question related to the order the test methods are run. I have seen that a similar question was asked here.

My problem is the following:
I'm testing a webapp and there is a table that shows some items sorted by descending id, so the last one inserted is always on the top. I have no way of filtering the table. I have some tests that work with this, so initially in each test I was inserting a new item. Since inserting the test data is slow and the tests don't always modify the data I thought of moving the data creation to a @SetupClass method.

However the following order can happen, for classes A and B:
1) Setup A
2) Test1 in A
3) Setup B <-- inserts a new row
4) Test 1 in B
5) Test 2 in A <-- fails, because step 3) modified the expected data.

I don't have any dependencies between the tests.

Also, I am running this with a general "All tests" suite from Eclipse. In this suite I'm including all the tests in the package. This is because I don't want to maintain the xml file every time a new test class is added (we have many many tests, and it would be painful to keep track). The thread I mentioned before suggests using different test elements, but I guess this would mean that I have to manually enter every test class in my xml file, right?

Is there a better way to tell TestNG to run all the tests in a class together if possible? (i.e. not interleaving tests from different classes)

Thanks in advance,
Tomas

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "testng-users" group.
To post to this group, send email to testng-users@...
To unsubscribe from this group, send email to testng-users+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/testng-users?hl=en
-~----------~----~----~----~------~----~------~--~---