[jira] Created: (DERBY-3834) Convert derbynet/runtimeinfo to JUnit

Convert derbynet/runtimeinfo to JUnit
-------------------------------------

                 Key: DERBY-3834
                 URL: https://issues.apache.org/jira/browse/DERBY-3834
             Project: Derby
          Issue Type: Test
          Components: Test
    Affects Versions: 10.5.0.0
            Reporter: Erlend Birkenes
            Assignee: Erlend Birkenes
            Priority: Minor
             Fix For: 10.5.0.0




--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erlend Birkenes updated DERBY-3834:
-----------------------------------

    Attachment: Derby-3834_1.diff

Please review.

Added the new test to suites.All and remove from derbyall and deleted old files.


-Erlend

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.0.0
>            Reporter: Erlend Birkenes
>            Assignee: Erlend Birkenes
>            Priority: Minor
>             Fix For: 10.5.0.0
>
>         Attachments: Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erlend Birkenes updated DERBY-3834:
-----------------------------------

    Derby Info: [Patch Available]

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.0.0
>            Reporter: Erlend Birkenes
>            Assignee: Erlend Birkenes
>            Priority: Minor
>             Fix For: 10.5.0.0
>
>         Attachments: Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622736#action_12622736 ]

Myrna van Lunteren commented on DERBY-3834:
-------------------------------------------

I ran the test, but the first time, it fails for me.
If I run it again without cleaning up the dir, it passes.
Maybe some of the details regarding sessions and statements are different when you first create a database...

So maybe you can wrap the test in a singleUseDecorator and adjust the expected output to what it would be when there are no pre-existing databases.

Other comments:
- I'd like  it better if you could avoid using ExecProcUtil.execCmdDumpResults...
- There is a property, derby.tests.debug, which can be set if someone wants to debug a test. I suggest you remove the boolean 'print' and instead, in the method 'print()' use org.apache.derbyTesting.junit.println' rather than System.out.println.  That way, no change of the code is needed to debug the test.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.0.0
>            Reporter: Erlend Birkenes
>            Assignee: Erlend Birkenes
>            Priority: Minor
>             Fix For: 10.5.0.0
>
>         Attachments: Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Myrna van Lunteren updated DERBY-3834:
--------------------------------------

    Derby Info:   (was: [Patch Available])

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.0.0
>            Reporter: Erlend Birkenes
>            Assignee: Erlend Birkenes
>            Priority: Minor
>             Fix For: 10.5.0.0
>
>         Attachments: Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Myrna van Lunteren updated DERBY-3834:
--------------------------------------

    Fix Version/s:     (was: 10.5.0.0)

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.0.0
>            Reporter: Erlend Birkenes
>            Assignee: Erlend Birkenes
>            Priority: Minor
>         Attachments: Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tiago R. Espinha reassigned DERBY-3834:
---------------------------------------

    Assignee: Tiago R. Espinha  (was: Erlend Birkenes)

I'll be taking over this issue as part of DERBY-4090

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tiago R. Espinha updated DERBY-3834:
------------------------------------

    Attachment: DERBY-3834.stat
                DERBY-3834.patch

Attaching a patch to the issue.

This patch has Myrna's remarks in consideration. It no longer uses the ExecProcUtil but execJavaCmd instead. It also uses the println() in BaseTestCase which will only print when the tests are being ran on verbose mode.

This patch also fixes the issue where only the second run would pass. Since it is impossible to predict the order by which the prepared statements will appear under x_testRuntimeInfoLocale(), I worked around it by parsing just parts of the output.

Also, the methods are invoked by a single fixture testRunTests(), since the order by which they are ran does matter and we can't rely on JUnit to run them in order.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tiago R. Espinha updated DERBY-3834:
------------------------------------

    Derby Info: [Patch Available]

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tiago R. Espinha updated DERBY-3834:
------------------------------------

    Attachment: DERBY-3834.patch

Attaching a new patch that compiles without errors against JDK1.4.2.

It also accounts for the customizable port part (DERBY-4217), which the previous patch didn't.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724412#action_12724412 ]

Tiago R. Espinha commented on DERBY-3834:
-----------------------------------------

Ran suites.All and I got a 100% pass.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724513#action_12724513 ]

Kathey Marsden commented on DERBY-3834:
---------------------------------------

Thanks Tiago for reviving this test. I ran the test with IBM 1.6 and it passed and confirmed that it is skipped with weme.

You should remove references to the old test in functionTests/suites/
DerbyNetClientRemote.exclude
DerbyNetClientUseprocess.exclude
DerbyNetRemote.exclude
DerbyNetUseprocess.exclude
derbynetmats.runall
j9derbynetmats.runall

and run derbyall to make sure all references to the old test are removed.

In the test I think it would be good to comment that a separate database was necessary to ensure consistent output above the call to singleUseDatabaseDecorator.

I think x_testRuntimeInfo and x_testRuntimeInfoMethod  and related constants could be more descriptively named, like x_testRuntimeInfoWithActiveConn and x_testRuntimeInfoAfterConnClose.

I am a bit confused by x_testRuntimeInfoLocale, even in the original test.  I would have expected it to test non-English output, but instead runs with:
        "-Duser.language=err", "-Duser.country=DE",
which makes it revert to English I guess.  I think it would be better if it tested the localization with user.language=de and then just check the output for some German string.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12724682#action_12724682 ]

Tiago R. Espinha commented on DERBY-3834:
-----------------------------------------

I have been discussing the locale bit with Kathey and we were able to draw some conclusions, but there's still a pending issue.

The original test apparently did no verification at all on the locale front. Despite setting the user.country and user.language properties, the output it expects is actually in English, so it doesn't seem like it was expected for it to work properly.

Still, we agreed it would be a nice addition to the test, having it check the localized output. The problem arises with the fact that the output of runtimeinfo is dependent on the locale of the server and not of the client (unlike ping, for example). So for this to work, that one single fixture has to shutdown the server and spawn another instance with the changed locale.

Does anyone have ideas or comments on this?

It is also noteworthy that the test seems to pick up the localization properties if I define them with -D on the command line.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tiago R. Espinha updated DERBY-3834:
------------------------------------

    Attachment: DERBY-3834.patch

Attaching a new patch file.

This patch removes the test from the exclude and the runall files and it also addresses the locale issue. In the old test, it seems that the locale wasn't even being properly tested. In this patch however, in the x_testRuntimeInfoLocale() fixture I changed the locale to German, restarted the server and then the fixture is processed normally (accounting for German output). Just before the fixture ends, I reset the locale back to what it was originally and once more restart the server so that the last fixture is able to complete normally.

There's one detail I'd like to get some input on. I added a method to this test called waitForServerShutdown(), which, as the name suggests, holds on the execution until the server shuts down. This has the same purpose as NetworkServerTestSetup.waitForServerStart(); but in this case we also want to wait for it to shutdown. Would it be acceptable to create the waitForServerShutdown() method also in the NetworkServerTestSetup? It might be useful for other tests and it's pretty standard, it just waits for the server to shutdown. Plus, if other tests require it in the future, they won't have to re-implement it locally.

Finally, I also added the comment on the singleUseDatabase usage, like Kathey suggested.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725264#action_12725264 ]

Kathey Marsden commented on DERBY-3834:
---------------------------------------

Hi Tiago,

I am concerned that if this tests fails we will be left with a network server running or leave the tests set to the German locale, causing subsequent tests to fail.

I think it would be better to make a second test
LocaleRuntimeInfoTest or some such and then use:
   public NetworkServerTestSetup
        (
         Test test,
         String[] systemProperties,
         String[] startupArgs,
         boolean serverShouldComeUp
        )

to spawn the server with the correct locale by setting system properties.  It will make sure everything is shutdown and cleaned up regardless of whether the test fails.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725277#action_12725277 ]

Tiago R. Espinha commented on DERBY-3834:
-----------------------------------------

Hello Kathey,

Well, considering I'm manipulating the server that the test spawns automatically, it should never stay running. I'm not spawning an independent server just to accommodate the locale testing; instead, I just stop the server that the test automatically starts, change the locale and start the server again.

The odd case would be if this fixture fails, but a possible solution is to encapsulate this method in a try..catch statement. Since all the fixtures are actually invoked manually because the order in which they are ran matters, I can just catch the exception that this method may throw and set the locale back to normal.

I'm just laying this on the table because it seems to me a bit of overkill to create a whole new test with just one fixture that tests the locale.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725339#action_12725339 ]

Tiago R. Espinha commented on DERBY-3834:
-----------------------------------------

I just noticed that we have a SysinfoLocaleTest that tests just the localized output of Sysinfo. Would it be a viable option to have a single LocaleTest where all these outputs are tested?

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725344#action_12725344 ]

Kathey Marsden commented on DERBY-3834:
---------------------------------------

SysinfoLocaleTest  seems to be an embedded tests a and tests derby.ui.locale which is not relevant here.  Do you think it will adapt itself well to the network server runtimeinfo testing?

I guess I am still leaning toward a locale specific runtimeinfo test. I suppose catching an exception and resetting the locale would work too, but seems a little messier.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Test
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dag H. Wanvik updated DERBY-3834:
---------------------------------

    Issue Type: Improvement  (was: Test)

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Improvement
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/DERBY-3834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tiago R. Espinha updated DERBY-3834:
------------------------------------

    Attachment: DERBY-3834.patch

Attaching a new patch to this issue.

After having discussed this with Kathey on IRC, we agreed on an alternative solution for this issue. Instead of separating the locale test from the actual RuntimeInfo test, I created a LocaleTestSetup decorator that allows the test to run with different localizations.

The solution I found to manage the expected outputs was to use an HashMaps inside HashMaps. This way we're able to select the set of expected outputs for a certain locale easily, and then it is also easy to pick the correct output message within the results for that locale. It might not be the best solution performance-wise, but this way it is rather easy to add new locales to be tested, should we want to.

Still, some input and alternative ideas are appreciated.

> Convert derbynet/runtimeinfo to JUnit
> -------------------------------------
>
>                 Key: DERBY-3834
>                 URL: https://issues.apache.org/jira/browse/DERBY-3834
>             Project: Derby
>          Issue Type: Improvement
>          Components: Test
>    Affects Versions: 10.5.1.1
>            Reporter: Erlend Birkenes
>            Assignee: Tiago R. Espinha
>            Priority: Minor
>         Attachments: DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.patch, DERBY-3834.stat, Derby-3834_1.diff
>
>

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.