OpenJPA
  1. OpenJPA
  2. OPENJPA-1336

Intermittent test failure in org.apache.openjpa.persistence.generationtype.TestGeneratedValues when running on MySQL

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: lib
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      As reported in OPENJPA-1168:

      Milosz Tylenda added a comment - 09/Jul/09 09:17 AM
      This reminds me of a test case which sometimes fails for me while running against MySQL and which I did not have time to look at. Maybe this is connected although the test case is single-threaded. The stack trace I am receiving is:

      testDefaultValues(org.apache.openjpa.persistence.generationtype.TestGeneratedValues) Time elapsed: 0.787 sec <<< FAILURE!
      junit.framework.AssertionFailedError
      at junit.framework.Assert.fail(Assert.java:47)
      at junit.framework.Assert.assertTrue(Assert.java:20)
      at junit.framework.Assert.assertFalse(Assert.java:34)
      at junit.framework.Assert.assertFalse(Assert.java:41)
      at org.apache.openjpa.persistence.generationtype.TestGeneratedValues.testDefaultValues(TestGeneratedValues.java:49)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:585)
      at junit.framework.TestCase.runTest(TestCase.java:154)
      at junit.framework.TestCase.runBare(TestCase.java:127)
      at org.apache.openjpa.persistence.test.PersistenceTestCase.runBare(PersistenceTestCase.java:466)
      at junit.framework.TestResult$1.protect(TestResult.java:106)
      at junit.framework.TestResult.runProtected(TestResult.java:124)
      at junit.framework.TestResult.run(TestResult.java:109)
      at junit.framework.TestCase.run(TestCase.java:118)
      at org.apache.openjpa.persistence.test.PersistenceTestCase.run(PersistenceTestCase.java:181)
      at junit.framework.TestSuite.runTest(TestSuite.java:208)
      at junit.framework.TestSuite.run(TestSuite.java:203)
      at sun.reflect.GeneratedMethodAccessor58.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:585)
      at org.apache.maven.surefire.junit.JUnitTestSet.execute(JUnitTestSet.java:213)
      at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
      at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
      at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:585)
      at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:345)
      at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1009)

      The TestGeneratedValues.java:49 contains:

      assertFalse(gv.getUuidstring().equals(gv2.getUuidstring()));

      This means UUID generator sometimes generates duplicates.

        Issue Links

          Activity

          Hide
          Rick Curtis added a comment -

          Attaching a patch that has a testcase that will always fail on MySQL and DB2. The net of the testcase is:
          /*

          • For some reason, Strings that are created off a byte[] that contains a
          • byte less than -100 AND is encoded in "ISO-8859-1" isn't stored properly
          • in DB2 and MySQL (others???).
            */

          I found that if I change org.apache.openjpa.lib.util.UUIDGenerator.nextString(int) to use default encoding (rather than "ISO-8859-1") everything works properly. I'm apprehensive to make the change because I'm not sure of the ramification of doing such.

          Any comments?

          Show
          Rick Curtis added a comment - Attaching a patch that has a testcase that will always fail on MySQL and DB2. The net of the testcase is: /* For some reason, Strings that are created off a byte[] that contains a byte less than -100 AND is encoded in "ISO-8859-1" isn't stored properly in DB2 and MySQL (others???). */ I found that if I change org.apache.openjpa.lib.util.UUIDGenerator.nextString(int) to use default encoding (rather than "ISO-8859-1") everything works properly. I'm apprehensive to make the change because I'm not sure of the ramification of doing such. Any comments?
          Hide
          Milosz Tylenda added a comment -

          Rick, good job figuring that out.

          My concern is that relying on the default encoding would make the UUID behaviour inconsistent across environments. I think that the default encoding is now something like "Windows-1252" on (English/USA) Windows and "UTF-8" on most Linuxes and can be further changed by users. So an OpenJPA application may stop working when switched its environment and finding the cause can be difficult.

          I would rather use a constant encoding ("ISO-8859-1" or any other but supported by every JVM - there is a list in Javadoc) in the UUID generator. Would change or narrow the range of generated byte values if required.

          Another factor is the encoding of the database. If we start sending strings containing "UTF-8" characters to a database which uses only one-byte encoding (quite common), another error might occur. That suggests me using "ISO-8859-1" as "the lowest common denominator" in the generator is a good idea.

          A user manual update might be reasonable after resolving this.

          Maybe I am missing something?

          Show
          Milosz Tylenda added a comment - Rick, good job figuring that out. My concern is that relying on the default encoding would make the UUID behaviour inconsistent across environments. I think that the default encoding is now something like "Windows-1252" on (English/USA) Windows and "UTF-8" on most Linuxes and can be further changed by users. So an OpenJPA application may stop working when switched its environment and finding the cause can be difficult. I would rather use a constant encoding ("ISO-8859-1" or any other but supported by every JVM - there is a list in Javadoc) in the UUID generator. Would change or narrow the range of generated byte values if required. Another factor is the encoding of the database. If we start sending strings containing "UTF-8" characters to a database which uses only one-byte encoding (quite common), another error might occur. That suggests me using "ISO-8859-1" as "the lowest common denominator" in the generator is a good idea. A user manual update might be reasonable after resolving this. Maybe I am missing something?
          Hide
          Jeremy Bauer added a comment -

          I documented this issue in a test case a while back, but unfortunately, did not add the caveat to the user docs.

          From TestGeneratedValues.testUUIDGenerators():
          + // Compare original hex values with new values. They should be equal.
          + // Note: UUID 'string' values are not compared. In most cases they will
          + // be the same, but in an environment where data is converted to
          + // a considerably different character encoding of the database (ex.
          + // Unicode -> EBCDIC) upon persistence, the uuid string returned by the
          + // database may not be equal to the original value. This is a common
          + // issue with string data, but even more likely for a uuids given that
          + // uuid strings are produced from pseudo-random byte arrays, which yield
          + // all sorts of variant characters.

          IMHO, using uuid-string is a bad idea since, depending on the platform (client and db) it could lead to data conversion issues. uuid-hex or uuid-type4-hex is a much better choice since invariant characters are used to represent the uuid. Changing the default encoding could drastically affect the behavior of existing apps so I wouldn't recommend it. Have you considered adding a configuration property to allow users to configure the default encoding to use when generating uuid strings? That would maintain backward compatibility and provide a configurable solution for issues such as these.

          Show
          Jeremy Bauer added a comment - I documented this issue in a test case a while back, but unfortunately, did not add the caveat to the user docs. From TestGeneratedValues.testUUIDGenerators(): + // Compare original hex values with new values. They should be equal. + // Note: UUID 'string' values are not compared. In most cases they will + // be the same, but in an environment where data is converted to + // a considerably different character encoding of the database (ex. + // Unicode -> EBCDIC) upon persistence, the uuid string returned by the + // database may not be equal to the original value. This is a common + // issue with string data, but even more likely for a uuids given that + // uuid strings are produced from pseudo-random byte arrays, which yield + // all sorts of variant characters. IMHO, using uuid-string is a bad idea since, depending on the platform (client and db) it could lead to data conversion issues. uuid-hex or uuid-type4-hex is a much better choice since invariant characters are used to represent the uuid. Changing the default encoding could drastically affect the behavior of existing apps so I wouldn't recommend it. Have you considered adding a configuration property to allow users to configure the default encoding to use when generating uuid strings? That would maintain backward compatibility and provide a configurable solution for issues such as these.
          Hide
          Milosz Tylenda added a comment -

          Resolved by Donald in OPENJPA-1621 by removing the questionable comparison.

          Show
          Milosz Tylenda added a comment - Resolved by Donald in OPENJPA-1621 by removing the questionable comparison.

            People

            • Assignee:
              Unassigned
              Reporter:
              Rick Curtis
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development