Avro
  1. Avro
  2. AVRO-1155

Stringable Date test in TestReflect fails if timezone doesn't match locale's default

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.2
    • Component/s: java
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The problem is that java.util.Date's string contructor cannot reliably parse its toString() representation in some combinations of timezone and locale.

      This started happening after AVRO-1146.

      1. AVRO-1155.patch
        1 kB
        Tom White
      2. AVRO-1155.patch
        2 kB
        Tom White

        Activity

        Hide
        Tom White added a comment -

        Here's the error I get when running with US locale settings and a UK timezone:

        -------------------------------------------------------------------------------
        Test set: org.apache.avro.reflect.TestReflect
        -------------------------------------------------------------------------------
        Tests run: 40, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.408 sec <<< FAILURE!
        testStringables(org.apache.avro.reflect.TestReflect)  Time elapsed: 0.012 sec  <<< ERROR!
        org.apache.avro.AvroRuntimeException: java.lang.reflect.InvocationTargetException
                at org.apache.avro.reflect.ReflectDatumReader.readString(ReflectDatumReader.java:129)
                at org.apache.avro.reflect.ReflectDatumReader.readString(ReflectDatumReader.java:114)
                at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150)
                at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:135)
                at org.apache.avro.reflect.TestReflect.checkBinary(TestReflect.java:659)
                at org.apache.avro.reflect.TestReflect.checkBinary(TestReflect.java:668)
                at org.apache.avro.reflect.TestReflect.checkStringable(TestReflect.java:620)
                at org.apache.avro.reflect.TestReflect.testStringables(TestReflect.java:611)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
                at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
                at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
                at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
                at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
                at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
                at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
                at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
                at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
                at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
                at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
                at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
                at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
                at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
                at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
                at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
                at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
                at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
                at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
        Caused by: java.lang.reflect.InvocationTargetException
                at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
                at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
                at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
                at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
                at org.apache.avro.reflect.ReflectDatumReader.readString(ReflectDatumReader.java:121)
                ... 36 more
        Caused by: java.lang.IllegalArgumentException
                at java.util.Date.parse(Date.java:598)
                at java.util.Date.<init>(Date.java:255)
                ... 41 more
        
        Show
        Tom White added a comment - Here's the error I get when running with US locale settings and a UK timezone: ------------------------------------------------------------------------------- Test set: org.apache.avro.reflect.TestReflect ------------------------------------------------------------------------------- Tests run: 40, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.408 sec <<< FAILURE! testStringables(org.apache.avro.reflect.TestReflect) Time elapsed: 0.012 sec <<< ERROR! org.apache.avro.AvroRuntimeException: java.lang.reflect.InvocationTargetException at org.apache.avro.reflect.ReflectDatumReader.readString(ReflectDatumReader.java:129) at org.apache.avro.reflect.ReflectDatumReader.readString(ReflectDatumReader.java:114) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150) at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:135) at org.apache.avro.reflect.TestReflect.checkBinary(TestReflect.java:659) at org.apache.avro.reflect.TestReflect.checkBinary(TestReflect.java:668) at org.apache.avro.reflect.TestReflect.checkStringable(TestReflect.java:620) at org.apache.avro.reflect.TestReflect.testStringables(TestReflect.java:611) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.avro.reflect.ReflectDatumReader.readString(ReflectDatumReader.java:121) ... 36 more Caused by: java.lang.IllegalArgumentException at java.util.Date.parse(Date.java:598) at java.util.Date.<init>(Date.java:255) ... 41 more
        Hide
        Tom White added a comment -

        Here's a fix to use a stable string representation for the Date object.

        Show
        Tom White added a comment - Here's a fix to use a stable string representation for the Date object.
        Hide
        Doug Cutting added a comment -

        Fixing this with Date-specific code in ReflectDatumWriter feels wrong. Rather perhaps Date is not a candidate for Stringable treatment after all, since it's toString() method does not always return a string that can be used to construct an equivalent Date instance. If that's the case, then I think we should instead just remove Date from the default stringableClasses list in ReflectData.java and remove it from the tests in TestReflectData.java. Thoughts?

        Show
        Doug Cutting added a comment - Fixing this with Date-specific code in ReflectDatumWriter feels wrong. Rather perhaps Date is not a candidate for Stringable treatment after all, since it's toString() method does not always return a string that can be used to construct an equivalent Date instance. If that's the case, then I think we should instead just remove Date from the default stringableClasses list in ReflectData.java and remove it from the tests in TestReflectData.java. Thoughts?
        Hide
        Tom White added a comment -

        I agree. The fact that Date's String constructor and toString() methods are deprecated is another reason not to rely on them.

        Here's a patch to remove Date from the list of built-in Stringables.

        Show
        Tom White added a comment - I agree. The fact that Date's String constructor and toString() methods are deprecated is another reason not to rely on them. Here's a patch to remove Date from the list of built-in Stringables.
        Hide
        Doug Cutting added a comment -

        +1

        Show
        Doug Cutting added a comment - +1
        Hide
        Tom White added a comment -

        I just committed this.

        Show
        Tom White added a comment - I just committed this.
        Hide
        ASF subversion and git services added a comment -

        Commit 1517866 from Doug Cutting in branch 'avro/trunk'
        [ https://svn.apache.org/r1517866 ]

        AVRO-1268. Corrected change log message. Date is not supported as a
        stringable. That support was removed in AVRO-1155.

        Show
        ASF subversion and git services added a comment - Commit 1517866 from Doug Cutting in branch 'avro/trunk' [ https://svn.apache.org/r1517866 ] AVRO-1268 . Corrected change log message. Date is not supported as a stringable. That support was removed in AVRO-1155 .
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in AvroJava #394 (See https://builds.apache.org/job/AvroJava/394/)
        AVRO-1268. Corrected change log message. Date is not supported as a
        stringable. That support was removed in AVRO-1155. (cutting: rev 1517866)

        • /avro/trunk/CHANGES.txt
        Show
        Hudson added a comment - SUCCESS: Integrated in AvroJava #394 (See https://builds.apache.org/job/AvroJava/394/ ) AVRO-1268 . Corrected change log message. Date is not supported as a stringable. That support was removed in AVRO-1155 . (cutting: rev 1517866) /avro/trunk/CHANGES.txt

          People

          • Assignee:
            Tom White
            Reporter:
            Tom White
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development