Derby
  1. Derby
  2. DERBY-5155

Make testcases pass in non-English locale

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.9.1.0
    • Component/s: Test
    • Labels:
    • Bug behavior facts:
      Deviation from standard

      Description

      There are a lot of derby testing depending on the English locale, which will fail in a non-English locale. A general resolution should be provided.

        Issue Links

          Activity

          Hide
          Houx Zhang added a comment -

          Cited from development mailing list:

          Hi, all. When I ran org.apache.derbyTesting.functionTests.tests.tools._Suite, I found 11 failure. i.e. for SysinfoCPCheckTest, I've got error message below:

          D:\derby\test>java junit.textui.TestRunner org.apache.derbyTesting.functionTests
          .tests.tools.SysinfoCPCheckTest
          .F
          Time: 0.265
          There was 1 failure:
          1) testClassPathChecker(org.apache.derbyTesting.functionTests.tests.tools.Sysinf
          oCPCheckTest)junit.framework.ComparisonFailure: null expected:<[Testing for pres
          ence of all Derby-related libraries; typically, only some are needed.]> but was:
          <[测试所有与 Derby 相关的库是否存在;通常只需要其中的某些库。]>
          at org.apache.derbyTesting.functionTests.tests.tools.SysinfoCPCheckTest.
          testClassPathChecker(SysinfoCPCheckTest.java:157)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
          java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
          sorImpl.java:25)
          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:
          112)

          FAILURES!!!
          Tests run: 1, Failures: 1, Errors: 0

          I seems derby testing can not work well in Chinese Locale, maybe all other nonEnglish locales. Following the Derby wiki, I have tried to use "LocaleTestSetup' to decorate the test case in Locale.English and it pass successfully!

          As mentioned above, there are still some more testcases (maybe a lot of) can not work in nonEnglish locale. How to deal with this problem? A not so smart method is to decorate them individually in Locale.English, maybe a better method is to add a decoration into the root of the regression test or add a new system property to resolve it thoroughly.

          Wish for your reply, thanks!

          Regards

          Show
          Houx Zhang added a comment - Cited from development mailing list: Hi, all. When I ran org.apache.derbyTesting.functionTests.tests.tools._Suite, I found 11 failure. i.e. for SysinfoCPCheckTest, I've got error message below: D:\derby\test>java junit.textui.TestRunner org.apache.derbyTesting.functionTests .tests.tools.SysinfoCPCheckTest .F Time: 0.265 There was 1 failure: 1) testClassPathChecker(org.apache.derbyTesting.functionTests.tests.tools.Sysinf oCPCheckTest)junit.framework.ComparisonFailure: null expected:<[Testing for pres ence of all Derby-related libraries; typically, only some are needed.]> but was: < [测试所有与 Derby 相关的库是否存在;通常只需要其中的某些库。] > at org.apache.derbyTesting.functionTests.tests.tools.SysinfoCPCheckTest. testClassPathChecker(SysinfoCPCheckTest.java:157) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java: 112) FAILURES!!! Tests run: 1, Failures: 1, Errors: 0 I seems derby testing can not work well in Chinese Locale, maybe all other nonEnglish locales. Following the Derby wiki, I have tried to use "LocaleTestSetup' to decorate the test case in Locale.English and it pass successfully! As mentioned above, there are still some more testcases (maybe a lot of) can not work in nonEnglish locale. How to deal with this problem? A not so smart method is to decorate them individually in Locale.English, maybe a better method is to add a decoration into the root of the regression test or add a new system property to resolve it thoroughly. Wish for your reply, thanks! Regards
          Hide
          Houx Zhang added a comment -

          Cited from development mailing list:

          As for a system property, try running with -Duser.language=en and see if
          the test passes.
          I thought as you suggest we had something in setup to force the right
          language but obviously we do not. The only related issue I see on a
          quick search of Jira is
          https://issues.apache.org/jira/browse/DERBY-4260. Special handling may
          be needed for spawned processes.

          Making the Junit tests pass regardless of the user locale would be a
          great part of a GSoC project.
          Perhaps there is an issue for this, that I am just missing.

          Kathey

          Show
          Houx Zhang added a comment - Cited from development mailing list: As for a system property, try running with -Duser.language=en and see if the test passes. I thought as you suggest we had something in setup to force the right language but obviously we do not. The only related issue I see on a quick search of Jira is https://issues.apache.org/jira/browse/DERBY-4260 . Special handling may be needed for spawned processes. Making the Junit tests pass regardless of the user locale would be a great part of a GSoC project. Perhaps there is an issue for this, that I am just missing. Kathey
          Hide
          Houx Zhang added a comment -

          In this moment, there are two kinds of testing in Derby. One is junit test framework, and the other is harness framework. Accordingly, resolvation on non-English locale can be considered as three aspects:

          1, for junit test framework, BaseTestCase, the root of all test cases, should be changed, by setting locale to English in setup() and restoring it in teardown().

          2, for harness test framework. RunTest and Runsuite, the entry for all .sql scripts, should be changed, by setting locale before testing and restoring itg after that.

          3, there are still some special situation. i.e. as pointed out by Knut in DERBY-4260, derbynet/NetworkServerControlClientCommandTest will launch a separate process to run java command, so it's necessary to pass "-Dderby.ui.locale=en_US" for the separate process.

          So, IMHO, the general resolvation for non-English locale can cover 1 and 2-----maybe two subtasks are needed? As to 3, we can deal with them individually.

          Wish for your comments.

          Show
          Houx Zhang added a comment - In this moment, there are two kinds of testing in Derby. One is junit test framework, and the other is harness framework. Accordingly, resolvation on non-English locale can be considered as three aspects: 1, for junit test framework, BaseTestCase, the root of all test cases, should be changed, by setting locale to English in setup() and restoring it in teardown(). 2, for harness test framework. RunTest and Runsuite, the entry for all .sql scripts, should be changed, by setting locale before testing and restoring itg after that. 3, there are still some special situation. i.e. as pointed out by Knut in DERBY-4260 , derbynet/NetworkServerControlClientCommandTest will launch a separate process to run java command, so it's necessary to pass "-Dderby.ui.locale=en_US" for the separate process. So, IMHO, the general resolvation for non-English locale can cover 1 and 2-----maybe two subtasks are needed? As to 3, we can deal with them individually. Wish for your comments.
          Hide
          Knut Anders Hatlen added a comment -

          I think RunTest and RunSuite already set user.language and user.country to English/US. See getCommandLine() in the ...functionTests.harness.jvm class. So hopefully there's nothing more to do for 2.

          Most of the JUnit tests are independent of the locale because they only test query results and the SQLState of errors. The exception is those that check the contents of messages from the database. My preference would be to handle the failures individually and only hard-code the locale for the tests that actually depend on a particular locale in order to succeed. If we force all our tests to run in English locale only, we lose some variation in the testing and may not discover bugs that only appear in non-English locales.

          Show
          Knut Anders Hatlen added a comment - I think RunTest and RunSuite already set user.language and user.country to English/US. See getCommandLine() in the ...functionTests.harness.jvm class. So hopefully there's nothing more to do for 2. Most of the JUnit tests are independent of the locale because they only test query results and the SQLState of errors. The exception is those that check the contents of messages from the database. My preference would be to handle the failures individually and only hard-code the locale for the tests that actually depend on a particular locale in order to succeed. If we force all our tests to run in English locale only, we lose some variation in the testing and may not discover bugs that only appear in non-English locales.
          Hide
          Dag H. Wanvik added a comment -

          I think I agree with Knut on this one.

          +1 to only hard-code the locale for the tests that actually depend on a particular locale in order to succeed.

          Show
          Dag H. Wanvik added a comment - I think I agree with Knut on this one. +1 to only hard-code the locale for the tests that actually depend on a particular locale in order to succeed.
          Hide
          Houx Zhang added a comment -

          I accept your wonderful advice, Knut. I have seen English-locale setting in getCommandLine(). Considering the harness testing have been converting to JUnit testing, we will see more tests benefit from the work.

          Then, just like Knut's advice, it seems the new relosution should be:

          Firstly, find the junit test cases that fails in non-English locale. For each of them, if it fails just because the content of message is different from expected, a assertion based on SQLState will be adopted; otherwise, a hard-code locale will be used.

          What's your opinion, please?

          Regards

          Show
          Houx Zhang added a comment - I accept your wonderful advice, Knut. I have seen English-locale setting in getCommandLine(). Considering the harness testing have been converting to JUnit testing, we will see more tests benefit from the work. Then, just like Knut's advice, it seems the new relosution should be: Firstly, find the junit test cases that fails in non-English locale. For each of them, if it fails just because the content of message is different from expected, a assertion based on SQLState will be adopted; otherwise, a hard-code locale will be used. What's your opinion, please? Regards
          Hide
          Bryan Pendleton added a comment -

          Yes, that seems like the best way to proceed.

          Also, we can work on each test one at a time; we don't have to find and fix
          all of them at once. As soon as you can identify a test which has a locale
          problem, see if you can fix it using the techniques you've identified, and
          submit a patch for that test. Over time, more and more tests will be fixed,
          and an increasing number of test suites will be able to be run in your configuration.

          Show
          Bryan Pendleton added a comment - Yes, that seems like the best way to proceed. Also, we can work on each test one at a time; we don't have to find and fix all of them at once. As soon as you can identify a test which has a locale problem, see if you can fix it using the techniques you've identified, and submit a patch for that test. Over time, more and more tests will be fixed, and an increasing number of test suites will be able to be run in your configuration.
          Hide
          Knut Anders Hatlen added a comment -

          Sounds like a good plan. +1 to the suggested approach.

          Show
          Knut Anders Hatlen added a comment - Sounds like a good plan. +1 to the suggested approach.
          Hide
          Houx Zhang added a comment -

          Thanks for your encourage, Bryan and Knut!

          I will go on this way. It seems a long-term work, as there are dozes of test fails because of locale when I ran the complete JUnit suite.

          Show
          Houx Zhang added a comment - Thanks for your encourage, Bryan and Knut! I will go on this way. It seems a long-term work, as there are dozes of test fails because of locale when I ran the complete JUnit suite.
          Hide
          Myrna van Lunteren added a comment -

          Are there any further tests that fail with non-English locale?

          Show
          Myrna van Lunteren added a comment - Are there any further tests that fail with non-English locale?
          Hide
          Myrna van Lunteren added a comment -

          I'm marking this fixed as it appears all subtasks and linked bugs are fixed too.

          Show
          Myrna van Lunteren added a comment - I'm marking this fixed as it appears all subtasks and linked bugs are fixed too.

            People

            • Assignee:
              Houx Zhang
              Reporter:
              Houx Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development