[IMPALA-8026] Actual row counts for nested loop join are way too high while the query is executing

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 3.1.0
• Fix Version/s: Impala 3.2.0
• Component/s: Backend
• Epic Color: ghx-label-4
• Assignee: Tim Armstrong
• Reporter: Paul Rogers
• Created: 28/Dec/18 23:48
• Updated: 19/Sep/19 15:34
• Resolved: 08/Jan/19 01:27

Description

Consider this extract from a query plan:

Operator                      #Rows  Est. #Rows
--------------------------------------------------------------
…
|  10:HASH JOIN               9.53M      18.14K 
|  |--19:EXCHANGE                 1           1
|  |  00:SCAN HDFS                1           1
|  06:NESTED LOOP JOIN        4.88B     863.84K 
|  |--18:EXCHANGE                 1           1
|  |  04:SCAN HDFS                1           1
|  05:HASH JOIN               9.53M     863.84K

If the above is to be believed, the 06 nested loop join produced 5 billion rows. But, the actual number is far too huge for that: joining 1 row with 10 million rows cannot produce 500 times that number of rows.

It appears that the nested loop join actually processed and returned the 9.5 million rows, since that is the same number produced by the 10 hash join which joins a single row with the output of the nested loop join.

Because this same bogus result appears across multiple plans, it is likely that the actual number is completely wrong and bears no relation to the number of rows actually returned.

Issue Links

• is duplicated by: IMPALA-8956 Row count incorrect in summary while query running