[HIVE-22224] Support Parquet-Avro Timestamp Type

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.5, 2.3.6
• Component/s: Database/Schema
• Labels: parquet pull-request-available
• Assignee: cdmikechen
• Reporter: cdmikechen
• Due: 01/Sep/19
• Created: 20/Sep/19 16:30
• Updated: 16/Apr/22 00:18

Description

When user create an external table and import a parquet-avro data with 1.8.2 version which supported logical_type in Hive2.3 or before version, Hive can not read timestamp type column data correctly.

Hive will read it as LongWritable which it actually stores as long(logical_type=timestamp-millis).So we may add some codes in org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableTimestampObjectInspector.java to let Hive cast long type to timestamp type.

Some code like below:

 

public Timestamp getPrimitiveJavaObject(Object o) {
  if (o instanceof LongWritable) 

{     return new Timestamp(((LongWritable) o).get());   }

  return o == null ? null : ((TimestampWritable) o).getTimestamp();
}

 

Issue Links

• links to: GitHub Pull Request #871
• links to: GitHub Pull Request #3002