Author: Matthew Jacobs <firstname.lastname@example.org>
Date: Tue Dec 13 14:57:01 2016 -0800
IMPALA-4654: KuduScanner must return when ReachedLimit()
Fixes a bug in the KuduScanner where the scan node's limit
was not respected and thus the scanner thread would
continue executing until the scan range was fully consumed.
This could result in completed queries leaving fragments
running and those threads could be using significant CPU and
For example, the query 'select * from tpch_kudu.lineitem
limit 90' when running in the minicluster and lineitem is
partitioned into 3 hash partitions would end up leaving a
scanner thread running for ~60 seconds. In real world
scenarios this can cause unexpected resource consumption.
This could build up over time leading to query failures if
these queries are submitted frequently.
The fix is to ensure KuduScanner::GetNext() returns with
eos=true when it finds ReachedLimit=true. An unnecessary and
somewhat confusing flag 'batch_done' was being returned by a
helper function DecodeRowsIntoRowBatch, which isn't
necessary and was removed in order to make it more clear how
the code in GetNext() should behave.
Reviewed-by: Alex Behm <email@example.com>
Reviewed-by: Tim Armstrong <firstname.lastname@example.org>
Reviewed-by: Dan Hecht <email@example.com>
Tested-by: Internal Jenkins