[DRILL-4696] select four table inner join result.Waiting for a long time after drill report error java.lang.OutOfMemoryError: Java heap space

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.6.0
• Component/s: Functions - Drill

Environment

Test Environment:
SUSE Linux Enterprise Server 11 SP3 (x86_64) cluster
MySQL 5.7.11 Enterprise Server - Advanced Edition
Drill cluster

• Flags: Important
• Assignee: Unassigned
• Reporter: david_hudavy
• Created: 26/May/16 08:37
• Updated: 27/Jun/16 16:55

Description

Test Environment:
cluster 10-3
MySQL 5.7.11 Enterprise Server - Advanced Edition
Drill cluster

Test Scope:
select performance of huge table(30M records).

MySQL table: a b c d inner join (Four table each have 30M records)

– four table inner join: (take time Drill Crash)
0: jdbc:drill:zk=SC-1:6181,SC-2:6181,PL-3:618> select
. . . . . . . . . . . . . . . . . . . . . . .> d.d_5,
. . . . . . . . . . . . . . . . . . . . . . .> c.c_3,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_4,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_5,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_6,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_7,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_8 ,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_9 ,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_10,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_11 ,
. . . . . . . . . . . . . . . . . . . . . . .> a.a_12,
. . . . . . . . . . . . . . . . . . . . . . .> b.b_4
. . . . . . . . . . . . . . . . . . . . . . .> from mysql.user_data.a,mysql.user_data.b,mysql.user_data.c,mysql.user_data.d
. . . . . . . . . . . . . . . . . . . . . . .> where c.c_3=a.a_3 and a.a_3 =b.b_3 and b.b_3=d.d_3
. . . . . . . . . . . . . . . . . . . . . . .> and c.c_3='0';
Drill Crash
2016-05-13 09:52:35,131 [28cacd19-0f04-cbb1-b418-73a76dcd6ebe:frag:0:0] ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in FragmentExecutor.
java.lang.OutOfMemoryError: Java heap space
at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2157) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1964) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3316) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:463) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3040) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2288) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2681) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2547) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2505) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1370) ~[mysql-connector-java-5.1.38-bin.jar:5.1.38]
at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208) ~[commons-dbcp-1.4.jar:1.4]
at org.apache.drill.exec.store.jdbc.JdbcRecordReader.setup(JdbcRecordReader.java:177) ~[drill-jdbc-storage-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:108) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ScanBatch.<init>(ScanBatch.java:136) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:40) ~[drill-jdbc-storage-1.6.0.jar:1.6.0]
at org.apache.drill.exec.store.jdbc.JdbcBatchCreator.getBatch(JdbcBatchCreator.java:33) ~[drill-jdbc-storage-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:146) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:169) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:126) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:169) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:126) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:169) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:126) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:169) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:126) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:169) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getRecordBatch(ImplCreator.java:126) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getChildren(ImplCreator.java:169) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getRootExec(ImplCreator.java:100) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.physical.impl.ImplCreator.getExec(ImplCreator.java:78) ~[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:231) ~[drill-java-exec-1.6.0.jar:1.6.0]

Issue Links

• relates to: DRILL-4177 select * from table;Node ran out of Heap memory, exiting.java.lang.OutOfMemoryError: GC overhead limit exceeded