Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15434

Add UDF to allow interrogation of uniontype values

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 2.3.0, 3.0.0
    • Component/s: UDF
    • Labels:
    • Target Version/s:
    • Release Note:
      Added UDF to allow interrogation of uniontype values.

      Description

      Overview

      As stated in the documention:

      UNIONTYPE support is incomplete The UNIONTYPE datatype was introduced in Hive 0.7.0 (HIVE-537), but full support for this type in Hive remains incomplete. Queries that reference UNIONTYPE fields in JOIN (HIVE-2508), WHERE, and GROUP BY clauses will fail, and Hive does not define syntax to extract the tag or value fields of a UNIONTYPE. This means that UNIONTYPEs are effectively look-at-only.

      It is essential to have a usable uniontype. Until full support is added to Hive users should at least have the ability to inspect and extract values for further comparison or transformation.

      Proposal

      I propose to add a GenericUDF that has 2 modes of operation. Consider the following schema and data that contains a union:

      Schema:

      struct<field1:uniontype<int,string>>
      

      Query:

      hive> select field1 from thing;
      {0:0}
      {1:"one"}
      

      Explode to Struct

      This method will recursively convert all unions within the type to structs with fields named tag_n, n being the tag number. Only the tag_* field that matches the tag of the union will be populated with the value. In the case above the schema of field1 will be converted to:

      struct<tag_0:int,tag_1:string>
      
      hive> select extract_union(field1) from thing;
      {"tag_0":0,"tag_1":null}
      {"tag_0":null,"tag_1":one}
      
      hive> select extract_union(field1).tag_0 from thing;
      0
      null
      

      Extract the specified tag

      This method will simply extract the value of the specified tag. If the tag number matches then the value is returned, if it does not, then null is returned.

      hive> select extract_union(field1, 0) from thing;
      0
      null
      

        Attachments

        1. HIVE-15434.01.patch
          29 kB
          David Maughan
        2. HIVE-15434.02.patch
          29 kB
          David Maughan

          Issue Links

            Activity

              People

              • Assignee:
                nahguam David Maughan
                Reporter:
                nahguam David Maughan
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: