Derby
  1. Derby
  2. DERBY-4713

Subclasses of ScriptTestCase can not run correctly with the non-English default locale

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 10.8.1.2
    • Component/s: Test
    • Labels:
      None
    • Urgency:
      Normal

      Description

      If the default locale is not English, many subclasses of ScriptTestCase will fail.

      For example, run NetIjTest (a subclass of ScriptTestcase) in a Chinese OS, the test case will fail:

      D:\derby\test>java junit.textui.TestRunner org.apache.derbyTesting.functionTests
      .tests.derbynet.NetIjTest
      .F
      Time: 5.422
      There was 1 failure:
      1) testclientij(org.apache.derbyTesting.functionTests.tests.derbynet.NetIjTest)j
      unit.framework.ComparisonFailure: Output at line 34 expected:<[ERROR 42X05: Tabl
      e/View 'APP.NOTTHERE' does not exist.]> but was:<[?? 42X05????APP.NOTTHERE??
      ?]>
      at org.apache.derbyTesting.functionTests.util.CanonTestCase.compareCanon
      (CanonTestCase.java:106)
      at org.apache.derbyTesting.functionTests.util.ScriptTestCase.runTest(Scr
      iptTestCase.java:198)
      at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:
      109)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57
      )
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57
      )
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:23)
      at junit.extensions.TestSetup.run(TestSetup.java:27)

      FAILURES!!!
      Tests run: 1, Failures: 1, Errors: 0

      While, it succeed running with a English-language lcoale.

      D:\derby\test>java -Duser.language=en junit.textui.TestRunner org.apache.derbyTe
      sting.functionTests.tests.derbynet.NetIjTest
      .
      Time: 6.187

      OK (1 test)

      1. bit.out
        39 kB
        Yun Lee
      2. derby-4713.patch
        9 kB
        Yun Lee
      3. derby-4713.stat
        0.8 kB
        Yun Lee
      4. derby-4713-1.patch
        3 kB
        Yun Lee
      5. derby-4713-1.stat
        0.3 kB
        Yun Lee
      6. derby-4713-2.patch
        3 kB
        Yun Lee
      7. derby-4713-2.stat
        0.4 kB
        Yun Lee

        Issue Links

          Activity

          Hide
          Knut Anders Hatlen added a comment -

          Sounds like a good idea, since the script tests won't pass in any locale for which we have translations. Perhaps the LocaleTestSetup decorator could be used for this?

          Show
          Knut Anders Hatlen added a comment - Sounds like a good idea, since the script tests won't pass in any locale for which we have translations. Perhaps the LocaleTestSetup decorator could be used for this?
          Hide
          Yun Lee added a comment -

          Thanks Knut. What you suggested works indeed!

          There are two ways to resolve this issue. One is to do change directly in ScriptTeseCase, the other is to change all the affected subclasses. I chose the second one. As I think, maybe some future test cases will do testing baseing on non-English Locale, if we do change directly in ScriptTeseCase, we will lose expandability.

          Please check the patch, thanks!

          Show
          Yun Lee added a comment - Thanks Knut. What you suggested works indeed! There are two ways to resolve this issue. One is to do change directly in ScriptTeseCase, the other is to change all the affected subclasses. I chose the second one. As I think, maybe some future test cases will do testing baseing on non-English Locale, if we do change directly in ScriptTeseCase, we will lose expandability. Please check the patch, thanks!
          Hide
          Knut Anders Hatlen added a comment -

          Thanks for the patch, Yun.

          I don't think we lose expandability by doing this directly in ScriptTestCase (or perhaps better: in its super-class CanonTestCase?), since we can always add a locale parameter to one of the constructors if we need support for non-English locales later. That's the approach that was taken for the input and output encoding. The advantage of doing it in ScriptTestCase is that it will automatically be done correctly for tests we add later, even if the person who writes the test forgets to wrap it in a LocaleTestSetup.

          I'm curious to know why the new constructor in LocaleTestSetup was needed. Couldn't the existing constructor be used with Locale.US as argument? I was thinking that preserving country and variant of the default locale could mean that things like dates and numbers are formatted differently, and that it would be more robust if we set all the components of the locale to known values.

          Show
          Knut Anders Hatlen added a comment - Thanks for the patch, Yun. I don't think we lose expandability by doing this directly in ScriptTestCase (or perhaps better: in its super-class CanonTestCase?), since we can always add a locale parameter to one of the constructors if we need support for non-English locales later. That's the approach that was taken for the input and output encoding. The advantage of doing it in ScriptTestCase is that it will automatically be done correctly for tests we add later, even if the person who writes the test forgets to wrap it in a LocaleTestSetup. I'm curious to know why the new constructor in LocaleTestSetup was needed. Couldn't the existing constructor be used with Locale.US as argument? I was thinking that preserving country and variant of the default locale could mean that things like dates and numbers are formatted differently, and that it would be more robust if we set all the components of the locale to known values.
          Hide
          Kathey Marsden added a comment -

          I agree it is better to change in ScriptTestCase and then if someone wanted some Locale specific test they could override the behavior.

          Show
          Kathey Marsden added a comment - I agree it is better to change in ScriptTestCase and then if someone wanted some Locale specific test they could override the behavior.
          Hide
          Yun Lee added a comment -

          I agree with both of your advice. It's a good to doing change in ScriptTestCase directly with a defined Locale.US. I will realize it.

          Show
          Yun Lee added a comment - I agree with both of your advice. It's a good to doing change in ScriptTestCase directly with a defined Locale.US. I will realize it.
          Hide
          Yun Lee added a comment -

          Ask for help!

          I have tried to do changing in ScriptTestSetup directly, just in the derby-4713-1.patch. However, it doesn't work as expected. Some subclasses of ScriptTestSetupLangScripts, such as ImportExportIJTest, LangScripts, SURijTest and NistScripts (also a subclass of ScriptTestSetup) can not pass with a lot of '?' in the test result. I also attached bit.out (the output of bit test in LangScripts), in which you can see a lot of '?' in it.

          I have spent a lot of time on debugging this problem, but could not resolve it. I have also tried to do changing in CanonTestCase, the situation was also bad. In my debugging, I'm sure , the default Locale has changed indeed and the runTest() method of ScriptTestCase was running in a US Locale. I have also tried to use some other Locale, such as "en_CN", but they all failed in the same situation.

          I have to ask for help to resolve this problem. As to my patch attcahed last time, it can works well for every test. It seems in the patch attached this time, ij has still run in a Locale not as expected.

          Please give some advice, thanks very much!

          Show
          Yun Lee added a comment - Ask for help! I have tried to do changing in ScriptTestSetup directly, just in the derby-4713-1.patch. However, it doesn't work as expected. Some subclasses of ScriptTestSetupLangScripts, such as ImportExportIJTest, LangScripts, SURijTest and NistScripts (also a subclass of ScriptTestSetup) can not pass with a lot of '?' in the test result. I also attached bit.out (the output of bit test in LangScripts), in which you can see a lot of '?' in it. I have spent a lot of time on debugging this problem, but could not resolve it. I have also tried to do changing in CanonTestCase, the situation was also bad. In my debugging, I'm sure , the default Locale has changed indeed and the runTest() method of ScriptTestCase was running in a US Locale. I have also tried to use some other Locale, such as "en_CN", but they all failed in the same situation. I have to ask for help to resolve this problem. As to my patch attcahed last time, it can works well for every test. It seems in the patch attached this time, ij has still run in a Locale not as expected. Please give some advice, thanks very much!
          Hide
          Knut Anders Hatlen added a comment -

          Perhaps this happens because the default locale for a database is determined when the database is created, and the test harness reuses databases across tests. If that's the problem, we may get around it by creating a fresh database for these tests. See for example TestConfiguration.singleUseDatabaseDecorator().

          Show
          Knut Anders Hatlen added a comment - Perhaps this happens because the default locale for a database is determined when the database is created, and the test harness reuses databases across tests. If that's the problem, we may get around it by creating a fresh database for these tests. See for example TestConfiguration.singleUseDatabaseDecorator().
          Hide
          Yun Lee added a comment -

          Knut, I have also test the TestConfiguration.singleUseDatabaseDecorator() just by your advice, it still failed.

          I have checked my last test result( not using TestConfiguration.singleUseDatabaseDecorator()), it seems, the default locale has been changed to US already. In CN locale, when the canon line is '0 rows inserted/updated/deleted', the output is '????????? 0 ?'. After do changing in ScriptTestSetup directly, this line is OK, and the test fails on this canon line 'ERROR 42X14: 'NCLOB' is not a column in table or VTI 'APP.B'.' with the output line 'ERROR 42X14: ?NCLOB????? VTI?APP.B?????'. In other words, it fails only on error prompt, and pass on common sql result. Anyone can tell us which situation will cause this problem, please?

          Show
          Yun Lee added a comment - Knut, I have also test the TestConfiguration.singleUseDatabaseDecorator() just by your advice, it still failed. I have checked my last test result( not using TestConfiguration.singleUseDatabaseDecorator()), it seems, the default locale has been changed to US already. In CN locale, when the canon line is '0 rows inserted/updated/deleted', the output is '????????? 0 ?'. After do changing in ScriptTestSetup directly, this line is OK, and the test fails on this canon line 'ERROR 42X14: 'NCLOB' is not a column in table or VTI 'APP.B'.' with the output line 'ERROR 42X14: ?NCLOB????? VTI?APP.B?????'. In other words, it fails only on error prompt, and pass on common sql result. Anyone can tell us which situation will cause this problem, please?
          Hide
          Myrna van Lunteren added a comment -

          I've played with this a bit.
          Of course, I have en_US for my locale. So I tried to change this by running the tests (junit.textui.TestRunner) with -Duser.language=zh_CN.

          The second patch (patch1) didn't compile; I had to make an additional change to functionTests/util/IjTestCase - commenting out the 'throws Exception' in the setUp() method. As this IjTestCase is not currently used, I guess it doesn't matter so much.

          Both the first (patch) and second (patch1) patches pass with English locale, and both have multiple failures with -Duser.language=zh_CN.
          Any suggestions on how to test the patches when in US/English locale?

          Where did you try to put in Testconfiguration.singleUseDatabaseDecorator() when it failed? And in which way did that fail?

          Show
          Myrna van Lunteren added a comment - I've played with this a bit. Of course, I have en_US for my locale. So I tried to change this by running the tests (junit.textui.TestRunner) with -Duser.language=zh_CN. The second patch (patch1) didn't compile; I had to make an additional change to functionTests/util/IjTestCase - commenting out the 'throws Exception' in the setUp() method. As this IjTestCase is not currently used, I guess it doesn't matter so much. Both the first (patch) and second (patch1) patches pass with English locale, and both have multiple failures with -Duser.language=zh_CN. Any suggestions on how to test the patches when in US/English locale? Where did you try to put in Testconfiguration.singleUseDatabaseDecorator() when it failed? And in which way did that fail?
          Hide
          Yun Lee added a comment -

          Hi,Myrna. I have been being puzzled about this issue for a long time. For the first patch, I did get it pass on my PC, but it failed in your environment. For the second patch, it should work accordingly with the first patch, but it still failed.

          After running applying the patch test time and time again, I think I have caught it, the two patch can work well — while the second patch need remove the 'throws Exception' in the IJScriptTest.setUp() method.

          When run the test, it's best to clean and rebuild the project, clean the 'fail' document for test database. I've gotten all the subclasses of ScriptTestcase pass.

          In derby-4732-2.patch, I removed the 'throws Exception' in the IJScriptTest.setUp() method, the remaining parts are the same with derby-4732-1.patch. Please verify it, thanks!

          Show
          Yun Lee added a comment - Hi,Myrna. I have been being puzzled about this issue for a long time. For the first patch, I did get it pass on my PC, but it failed in your environment. For the second patch, it should work accordingly with the first patch, but it still failed. After running applying the patch test time and time again, I think I have caught it, the two patch can work well — while the second patch need remove the 'throws Exception' in the IJScriptTest.setUp() method. When run the test, it's best to clean and rebuild the project, clean the 'fail' document for test database. I've gotten all the subclasses of ScriptTestcase pass. In derby-4732-2.patch, I removed the 'throws Exception' in the IJScriptTest.setUp() method, the remaining parts are the same with derby-4732-1.patch. Please verify it, thanks!
          Hide
          Myrna van Lunteren added a comment -

          I've committed patch derby-4713-2 with revision 1055998.

          With this patch, I did not see any unexpected test failures in a run of suites.All (sane build, classes - ibm 1.6 - I saw one intermittent server ping issue in testSetPortPriority and testPing failed for me).

          Looking at the code of the patch, it seemed ike it would be an improvement.
          Running suites.All with -Duser.language=zh_CN I still saw a lot of failures, including some of the scriptTestCase tests, but I'm assuming that it's a problem with that setup, as Yun Lee said tests passed for him with the patch where before they failed.

          Show
          Myrna van Lunteren added a comment - I've committed patch derby-4713-2 with revision 1055998. With this patch, I did not see any unexpected test failures in a run of suites.All (sane build, classes - ibm 1.6 - I saw one intermittent server ping issue in testSetPortPriority and testPing failed for me). Looking at the code of the patch, it seemed ike it would be an improvement. Running suites.All with -Duser.language=zh_CN I still saw a lot of failures, including some of the scriptTestCase tests, but I'm assuming that it's a problem with that setup, as Yun Lee said tests passed for him with the patch where before they failed.
          Hide
          Knut Anders Hatlen added a comment -

          [bulk update] Close all resolved issues that haven't been updated for more than one year.

          Show
          Knut Anders Hatlen added a comment - [bulk update] Close all resolved issues that haven't been updated for more than one year.

            People

            • Assignee:
              Yun Lee
              Reporter:
              Yun Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development