Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Not A Bug
-
Impala 2.3.0
-
None
-
None
-
#hadoop
$ hadoop version
Hadoop 2.6.0-cdh5.5.4
#impala
> version;
Shell version: Impala Shell v2.3.0-cdh5.5.4 (e65ded2) built on Mon Apr 25 10:55:11 PDT 2016
Server version: impalad version 2.3.0-cdh5.5.4 RELEASE (build e65ded24350974ae6b4e475557b358b718fad29e)
#os
$ lsb_release -d;
Description: Red Hat Enterprise Linux Server release 7.2 (Maipo)
$ uname -a
Linux gcphxd11.uk.db.com 3.10.0-327.28.2.el7.x86_64 #1 SMP Mon Jun 27 14:48:28 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
#JDBC driver
Cloudera_ImpalaJDBC41_2.5.32#hadoop $ hadoop version Hadoop 2.6.0-cdh5.5.4 #impala > version; Shell version: Impala Shell v2.3.0-cdh5.5.4 (e65ded2) built on Mon Apr 25 10:55:11 PDT 2016 Server version: impalad version 2.3.0-cdh5.5.4 RELEASE (build e65ded24350974ae6b4e475557b358b718fad29e) #os $ lsb_release -d; Description: Red Hat Enterprise Linux Server release 7.2 (Maipo) $ uname -a Linux gcphxd11.uk.db.com 3.10.0-327.28.2.el7.x86_64 #1 SMP Mon Jun 27 14:48:28 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux #JDBC driver Cloudera_ImpalaJDBC41_2.5.32
Description
While running performance tests I'm noticing that quite often execution of all threads are blocked around invocation of ResultSet#next() when there are no more records in result set and ResultSet#next() will eventually return false.
Particularly in my test I'm periodically starting 10 connections with each connection to run the same query 10 times - i.e. 10*10=100 queries for every run.
The query is 'select * from <some_table> limit 10'
The issue doesn't take place in single-threaded mode, only with multiple connections working simultaneously.
See related log lines generated by the attached reproduction code:
16:33:24.781 1304: BEFORE rs.next()
16:33:24.781 1304: AFTER rs.next() next = true
16:33:24.781 1304: BEFORE rs.next()
16:33:24.781 1304: AFTER rs.next() next = true
16:33:24.781 1304: BEFORE rs.next() //this is the last rs.next() for cycle no. 1304
//nothing here for 25 second, looks like locking issue, notice that several threads are calling ResultSet.next for the last time
16:33:50.049 1312: AFTER rs.next() next = false
16:33:50.049 1309: AFTER rs.next() next = false
16:33:50.049 1309: AFTER handling ResultSet
16:33:50.049 1307: AFTER rs.next() next = false
16:33:50.049 1308: AFTER rs.next() next = false
16:33:50.049 1314: BEFORE stmt.executeQuery
16:33:50.049 1304: AFTER rs.next() next = false //rs.next() for cycle no. 1304 returned in 25 seconds
16:33:50.049 1313: AFTER rs.next() next = false
16:33:50.049 1313: AFTER handling ResultSet
16:33:50.049 1304: AFTER handling ResultSet