[HIVE-10971] count(*) with count(distinct) gives wrong results when hive.groupby.skewindata=true

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.1
• Component/s: Logical Optimizer
• Assignee: WangMeng
• Reporter: WangMeng
• Created: 09/Jun/15 08:52
• Updated: 17/Jun/15 02:23
• Resolved: 10/Jun/15 07:09

Description

When hive.groupby.skewindata=true, the following query based on TPC-H gives wrong results:

set hive.groupby.skewindata=true;

select l_returnflag, count(*), count(distinct l_linestatus)
from lineitem
group by l_returnflag
limit 10;

The query plan shows that it generates only one MapReduce job instead of two theoretically, which is dictated by hive.groupby.skewindata=true.

The problem arises only when

count(*)

and

count(distinct)

exist together.

Issue Links

• is duplicated by: HIVE-7261 Calculation works wrong when hive.groupby.skewindata is true and count(*) count(distinct) group by work simultaneously