[IMPALA-10654] Improve the precision of DecimalValue<T>::ToDouble

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Labels: ramp-up
• Epic Color: ghx-label-8
• Assignee: Amogh Margoor
• Reporter: Amogh Margoor
• Created: 13/Apr/21 08:50
• Updated: 03/Dec/21 05:48
• Resolved: 03/May/21 08:59

Description

From discussion of IMPALA-10350, it was noted that DecimalValue<T>::ToDouble is not accurate.

Current approach is:

 static_cast<double>(value_) / pow(10.0, scale).

Inaccuracy is due to fact that only integers from −2^53 to 2^53 can be represented accurately by double precision without any loss. Hence, above approach would not work for numbers like -0.43149576573887316. For DecimalValue representing -0.43149576573887316, value_ would be -43149576573887316 and scale would be 17. As value_ < -2^53, result would not be accurate.

Hence through discussion in IMPALA-10350, we propose to use thirdparty library https://github.com/lemire/fast_double_parser, which handles above scenario in a performant manner. Library's internal representation of Decimal is similar to the Impala's DecimalValue and function compute_float_64 can be used for the conversion.