Hive
  1. Hive
  2. HIVE-6059

Add union type support in LazyBinarySerDe

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.12.0
    • Fix Version/s: None
    • Component/s: File Formats
    • Labels:
      None

      Description

      We need the support to type union in LazyBinarySerDe, which is required to the join query with any union types in its select values. The reduce values in Join operation is serialized/deserialized using LazyBinarySerDe, otherwise we will see some errors like:

      Caused by: java.lang.NullPointerException
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardObjectInspector(ObjectInspectorUtils.java:106)
      at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardObjectInspector(ObjectInspectorUtils.java:156)
      at org.apache.hadoop.hive.ql.exec.JoinUtil.getStandardObjectInspectors(JoinUtil.java:98)
      at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.initializeOp(CommonJoinOperator.java:261)
      at org.apache.hadoop.hive.ql.exec.JoinOperator.initializeOp(JoinOperator.java:61)
      at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
      at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:150)
      
      1. repro.tar
        10 kB
        Johndee Burks
      2. Hive Issue 3.jpeg
        974 kB
        Doug Houck
      3. Hive Issue 2.jpeg
        1.74 MB
        Doug Houck
      4. Hive Issue 1.jpeg
        1.02 MB
        Doug Houck
      5. Hive Issue 0.jpeg
        1.18 MB
        Doug Houck

        Activity

        Hide
        Doug Houck added a comment -

        Steps to re-create, if helpful.

        Show
        Doug Houck added a comment - Steps to re-create, if helpful.
        Hide
        Johndee Burks added a comment -

        Doug Houck The file repro.tar has all the reproduction information. The ddl needs to be changed for the schema file path. Run the dml after and the problem should reproduce.

        Show
        Johndee Burks added a comment - Doug Houck The file repro.tar has all the reproduction information. The ddl needs to be changed for the schema file path. Run the dml after and the problem should reproduce.

          People

          • Assignee:
            Unassigned
            Reporter:
            Chaoyu Tang
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development