[IMPALA-5050] Add support to read TIMESTAMP_MILLIS and TIMESTAMP_MICROS to the parquet scanner

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Affects Version/s: Impala 2.9.0
• Fix Version/s: Impala 3.2.0
• Component/s: Backend
• Epic Link: Switch Parquet timestamp format to INT64
• Target Version: Product Backlog
• Assignee: Csaba Ringhofer
• Reporter: Lars Volker
• Created: 09/Mar/17 01:03
• Updated: 20/Feb/19 23:31
• Resolved: 15/Nov/18 15:57

Description

This requires updating parquet.thrift to a version that includes the TIMESTAMP_MICROS logical type.

Issue Links

• blocks: IMPALA-2111 Parquet TIMESTAMP is in milliseconds
• incorporates: IMPALA-7568 Implement timezone aware parquet stat filtering for timestamp columns
• is related to: IMPALA-4994 Push conversion and validation into dictionary construction
• is related to: IMPALA-7521 CLONE - Speed up sub-second unix time->TimestampValue conversions
• is related to: IMPALA-7723 Recognize int64 timestamps in CREATE TABLE LIKE PARQUET
• relates to: IMPALA-7568 Implement timezone aware parquet stat filtering for timestamp columns

Sub-Tasks

• Impala Doc: Add support to read TIMESTAMP_MILLIS and TIMESTAMP_MICROS to the parquet scanner: Closed