[HIVE-22006] Hive parquet timestamp compatibility, part 2

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: All Versions
• Assignee: Unassigned
• Reporter: H. Vetinari
• Created: 17/Jul/19 11:44
• Updated: 22/Dec/20 20:56

Description

The interaction between HIVE / IMPALA / SPARK writing timestamps is a major source of headaches in every scenario where such interaction cannot be avoided.

HIVE-9482 added hive.parquet.timestamp.skip.conversion, which only affects the reading of timestamps.

It formulates the next steps as:
> Later fix will change the write path to not convert, and stop the read-conversion even for files written by Hive itself.

At the very least, HIVE needs a switch to also turn off the conversion on writes. That would at least allow a setup where all three of HIVE / IMPALA / SPARK can be configured not to convert on read/write, and can hence safely work on the same data

Issue Links

• is related to: HIVE-21348 Execute the TIMESTAMP types roadmap