[HIVE-4736] Float aggregate of single value loses precission

• Type: Sub-task
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: vectorization-branch
• Component/s: Query Processor
• Assignee: Remus Rusanu
• Reporter: Remus Rusanu
• Created: 14/Jun/13 06:50
• Updated: 01/Nov/13 23:48

Description

I am seeing differences that are small but greater than expected in the output of TPC-H query 1. Thanks to Jitendra’s patch, the negative result error is gone. However, I am seeing that the results for

Sum(l_extendedprice * ( 1 - l_discount ))
Sum(l_extendedprice * ( 1 - l_discount ) * ( 1 + l_tax ))

Differ in the low-order 8 or so digits compared with text. I know the input is float so it should have all zeros in the low digits when cast to double. I would not expect the answers to match exactly but I would expect errors in the last few digits, not the last 8. I could be wrong, but I think this may be worth investigating. Any ideas?

I also noticed this, where I ran the query against a single row worth of input (order 1, line 1):

SELECT l_returnflag,
l_linestatus,
Sum(l_quantity) AS sum_qty,
Sum(l_extendedprice) AS sum_base_price,
Sum(l_extendedprice * ( 1 - l_discount )) AS sum_disc_price,
Sum(l_extendedprice * ( 1 - l_discount ) * ( 1 + l_tax )) AS sum_charge,
Avg(l_quantity) AS avg_qty,
Avg(l_extendedprice) AS avg_price,
Avg(l_discount) AS avg_disc,
Count AS count_order
FROM lineitem
WHERE l_shipdate <= '1998-09-19'
and l_orderkey = 1 and l_linenumber = 1
GROUP BY l_returnflag,
l_linestatus
ORDER BY l_returnflag,
l_linestatus;

input row
1 155190 7706 1 17.0 21168.23 0.04 0.02 N O 1996-03-13 1996-02-12 1996-03-22 DELIVER IN PERSON TRUCK egular courts above the

Result

V (orc)
N O 17.0 21168.23046875 20321.501268925873 20727.931285219973 17.0 21168.23046875 0.03999999910593033 1

NV (text)
N O 17.0 21168.23046875 20321.5 20727.9296875 17.0 21168.23046875 0.03999999910593033 1