[IMPALA-10350] Impala loses double precision because of DECIMAL->DOUBLE cast

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Labels: correctness ramp-up
• Epic Color: ghx-label-7
• Assignee: Amogh Margoor
• Reporter: Zoltán Borók-Nagy
• Created: 23/Nov/20 17:05
• Updated: 26/Apr/21 17:12

Description

Impala might loses presision of double values. Reproduction: 

create table double_tbl (d double) stored as textfile;
insert into double_tbl values (-0.43149576573887316);

 Then inspect the data file:

$ hdfs dfs -cat /test-warehouse/double_tbl/424097c644088674-c55b910100000000_175064830_data.0.txt
 -0.4314957657388731

The same happens if we store our data in Parquet.

Hive writes don't lose precision. If the data was written by Hive then Impala can read the values correctly:

$ bin/run-jdbc-client.sh -t NOSASL -q "select * from double_tbl;"
Using JDBC Driver Name: org.apache.hive.jdbc.HiveDriver
Connecting to: jdbc:hive2://localhost:21050/;auth=noSasl
Executing: select * from double_tbl
----[START]----
-0.43149576573887316
----[END]----

Sub-Tasks

• Improve the precision of DecimalValue<T>::ToDouble: Resolved
• Replace StringToFloatInternal that converts String to Float using fast_double_parser library: Resolved