Details
Major Description
Overview
As stated in the documention:
UNIONTYPE support is incomplete The UNIONTYPE datatype was introduced in Hive 0.7.0 (
HIVE-537), but full support for this type in Hive remains incomplete. Queries that reference UNIONTYPE fields in JOIN (HIVE-2508), WHERE, and GROUP BY clauses will fail, and Hive does not define syntax to extract the tag or value fields of a UNIONTYPE. This means that UNIONTYPEs are effectively look-at-only.
It is essential to have a usable uniontype. Until full support is added to Hive users should at least have the ability to inspect and extract values for further comparison or transformation.
Proposal
I propose to add a GenericUDF that has 2 modes of operation. Consider the following schema and data that contains a union:
Schema:
struct<field1:uniontype<int,string>>
Query:
hive> select field1 from thing;
{0:0}
{1:"one"}
Explode to Struct
This method will recursively convert all unions within the type to structs with fields named tag_n, n being the tag number. Only the tag_* field that matches the tag of the union will be populated with the value. In the case above the schema of field1 will be converted to:
struct<tag_0:int,tag_1:string>
hive> select extract_union(field1) from thing;
{"tag_0":0,"tag_1":null}
{"tag_0":null,"tag_1":one}
hive> select extract_union(field1).tag_0 from thing;
0
null
Extract the specified tag
This method will simply extract the value of the specified tag. If the tag number matches then the value is returned, if it does not, then null is returned.
hive> select extract_union(field1, 0) from thing;
0
null