Hive
  1. Hive
  2. HIVE-6059

Add union type support in LazyBinarySerDe

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.12.0
    • Fix Version/s: None
    • Component/s: File Formats
    • Labels:
      None

      Description

      We need the support to type union in LazyBinarySerDe, which is required to the join query with any union types in its select values. The reduce values in Join operation is serialized/deserialized using LazyBinarySerDe, otherwise we will see some errors like:

      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardObjectInspector(ObjectInspectorUtils.java:106)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardObjectInspector(ObjectInspectorUtils.java:156)
      at org.apache.hadoop.hive.ql.exec.JoinUtil.getStandardObjectInspectors(JoinUtil.java:98)
      at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:261)
      at org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp(JoinOperator.java:61)
      at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
      at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:150)
      
      1. Hive Issue 0.jpeg
        1.18 MB
        Doug Houck
      2. Hive Issue 1.jpeg
        1.02 MB
        Doug Houck
      3. Hive Issue 2.jpeg
        1.74 MB
        Doug Houck
      4. Hive Issue 3.jpeg
        974 kB
        Doug Houck
      5. repro.tar
        10 kB
        Johndee Burks

        Activity

        Hide
        Johndee Burks added a comment -

        Doug Houck The file repro.tar has all the reproduction information. The ddl needs to be changed for the schema file path. Run the dml after and the problem should reproduce.

        Show
        Johndee Burks added a comment - Doug Houck The file repro.tar has all the reproduction information. The ddl needs to be changed for the schema file path. Run the dml after and the problem should reproduce.
        Hide
        Doug Houck added a comment -

        Steps to re-create, if helpful.

        Show
        Doug Houck added a comment - Steps to re-create, if helpful.

          People

          • Assignee:
            Unassigned
            Reporter:
            Chaoyu Tang
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development