Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5285

Custom SerDes throw cast exception when there are complex nested structures containing NonSettableObjectInspectors.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.11.1
    • Fix Version/s: 0.12.0
    • Component/s: None
    • Labels:
      None

      Description

      The approach for HIVE-5199 fix is correct.However, the fix for HIVE-5199 is incomplete. Consider a complex nested structure containing the following object inspector hierarchy:
      SettableStructObjectInspector
      {
      ListObjectInspector<NonSettableStructObjectInspector>
      }

      In the above case, the cast exception can happen via MapOperator/FetchOperator as below:
      java.io.IOException: java.lang.ClassCastException: com.skype.data.hadoop.hive.proto.CustomObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
      at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:545)
      at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:489)
      at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
      at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1412)
      at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
      at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
      at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
      Caused by: java.lang.ClassCastException: com.skype.data.whaleshark.hadoop.hive.proto.ProtoMapObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.SettableMapObjectInspector
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:144)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.<init>(ObjectInspectorConverters.java:294)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters.getConverter(ObjectInspectorConverters.java:138)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$ListConverter.convert(ObjectInspectorConverters.java:251)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$StructConverter.convert(ObjectInspectorConverters.java:316)
      at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:529)
      ... 13 more

        Attachments

        1. HIVE-5285.2.patch.txt
          14 kB
          Hari Sankar Sivarama Subramaniyan
        2. HIVE-5285.1.patch.txt
          16 kB
          Hari Sankar Sivarama Subramaniyan

          Activity

            People

            • Assignee:
              hsubramaniyan Hari Sankar Sivarama Subramaniyan
              Reporter:
              hsubramaniyan Hari Sankar Sivarama Subramaniyan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: