[IMPALA-7472] Consider removing TimestampValue::FromSubsecondUnixTime

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Backend
• Labels: timestamp
• Epic Color: ghx-label-1
• Assignee: Unassigned
• Reporter: Csaba Ringhofer
• Created: 21/Aug/18 17:38
• Updated: 08/Jun/21 02:13

Description

TimestampValue::FromSubsecondUnixTime converts a double interpreted as unix time in seconds to TimestampValue. Impala uses it in two cases:
1: double <-> timestamp casting
2: aggregate function AVG on timestamps - it converts timestamp to double, calculates the average on doubles, and converts the average back to timestamp.

My concern about this is double's varying's precision depending on the distance from 0 (1970-01-01) - subsec timestamps around 2018 are rounded/distorted with several nanosecs, which can add up if a large number of timestamps are averaged.

I think that decimal (with nanosec precision) could be used for both cases:
1. Decimal could used as an intermediate stage - double < - >decimal and decimal < - > timestamp conversions have well defined rounding rules since decimal V2.
2. Decimal could be used as aggregation state - this would probably make the AVG slower, but it would be much more precise.

Issue Links

• is related to: IMPALA-9584 test_analytic_fns is flaky (small fractional differences in AVG)
• is related to: IMPALA-5121 avg() on timestamp col is wrong with -use_local_tz_for_unix_timestamp_conversions