[IMPALA-7521] CLONE - Speed up sub-second unix time->TimestampValue conversions

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Fix Version/s: Impala 3.1.0
• Component/s: Backend
• Labels: performance timestamp
• Epic Color: ghx-label-3
• Assignee: Csaba Ringhofer
• Reporter: Csaba Ringhofer
• Created: 03/Sep/18 17:34
• Updated: 02/May/19 21:43
• Resolved: 05/Oct/18 07:27

Description

Currently Impala converts from sub-second unix time to TimestampValue (which is split do date_ and time_ similarly to boost::posix_time::ptime ) by first splitting the input into seconds and sub-seconds part, converting the seconds part with boost::posix_time::from_time_t(), and then adding the sub-seconds part to this timestamp. This can be done much faster by splitting the sub-second input into date_ and time_ directly.

Avoiding boost::posix_time::from_time_t() would be also nice because it can only deal with timestamps from 1677 to 2262, which adds extra complexity to the related code.

Issue Links

• is a clone of: IMPALA-7417 Hit DCHECK in Parquet fuzz test (duplicate)
• is related to: IMPALA-7595 Check failed: IsValidTime(time_) at timestamp-value.h:322
• is related to: IMPALA-8268 Speed up TimestampValue->sub-second unix time conversions
• relates to: IMPALA-5050 Add support to read TIMESTAMP_MILLIS and TIMESTAMP_MICROS to the parquet scanner