Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11696

Exception when table-level serde is Parquet while partition-level serde is JSON

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Create a table with partitions and set the SerDe to be Json. The query "select * from tbl1" works fine.

      Now set table level SerDe to be Parquet serde. Based on HIVE-6785, table level and partition level can have different serdes.

      java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.UnsupportedOperationException: Cannot inspect java.util.ArrayList
              at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:154)
              at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1764)
              at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
              at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
              at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
              at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
              at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
              at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
              at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.UnsupportedOperationException: Cannot inspect java.util.ArrayList
              at org.apache.hadoop.hive.ql.exec.ListSinkOperator.process(ListSinkOperator.java:93)
              at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:812)
              at org.apache.hadoop.hive.ql.exec.LimitOperator.process(LimitOperator.java:54)
              at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:812)
              at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
              at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:812)
              at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
              at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:425)
              at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:417)
              at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
              ... 13 more
      Caused by: java.lang.UnsupportedOperationException: Cannot inspect java.util.ArrayList
              at org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveArrayInspector.getList(ParquetHiveArrayInspector.java:112)
              at org.apache.hadoop.hive.serde2.SerDeUtils.buildJSONString(SerDeUtils.java:310)
              at org.apache.hadoop.hive.serde2.SerDeUtils.getJSONString(SerDeUtils.java:202)
              at org.apache.hadoop.hive.serde2.DelimitedJSONSerDe.serializeField(DelimitedJSONSerDe.java:61)
              at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.doSerialize(LazySimpleSerDe.java:246)
              at org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe.serialize(AbstractEncodingAwareSerDe.java:55)
              at org.apache.hadoop.hive.ql.exec.DefaultFetchFormatter.convert(DefaultFetchFormatter.java:71)
              at org.apache.hadoop.hive.ql.exec.DefaultFetchFormatter.convert(DefaultFetchFormatter.java:40)
              at org.apache.hadoop.hive.ql.exec.ListSinkOperator.process(ListSinkOperator.java:90)
      

        Attachments

        1. HIVE-11696.2.patch
          21 kB
          Aihua Xu
        2. HIVE-11696.patch
          20 kB
          Aihua Xu

          Issue Links

            Activity

              People

              • Assignee:
                aihuaxu Aihua Xu
                Reporter:
                aihuaxu Aihua Xu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: