Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
Impala 4.1.0
-
None
-
None
-
ghx-label-8
Description
The huge values clause of the insert statement in TestParquetBloomFilter.test_fallback_from_dict_if_no_bloom_tbl_props could cause FE OOM:
https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5524/testReport/query_test.test_parquet_bloom_filter/TestParquetBloomFilter/test_fallback_from_dict_if_no_bloom_tbl_props_protocol__beeswax___exec_option____test_replan___1___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/
query_test/test_parquet_bloom_filter.py:176: in test_fallback_from_dict_if_no_bloom_tbl_props False) query_test/test_parquet_bloom_filter.py:228: in _create_table_dict_overflow self.execute_query(insert_stmt, vector.get_value('exec_option')) common/impala_test_suite.py:836: in wrapper return function(*args, **kwargs) common/impala_test_suite.py:868: in execute_query return self.__execute_query(self.client, query, query_options) common/impala_test_suite.py:961: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:212: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:189: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:365: in __execute_query handle = self.execute_query_async(query_string, user=user) beeswax/impala_beeswax.py:359: in execute_query_async handle = self.__do_rpc(lambda: self.imp_service.query(query,)) beeswax/impala_beeswax.py:522: in __do_rpc raise ImpalaBeeswaxException(self.__build_error_message(b), b) E ImpalaBeeswaxException: ImpalaBeeswaxException: E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'> E MESSAGE: OutOfMemoryError: GC overhead limit exceeded
impalad.INFO
I0404 14:24:30.203562 19115 Frontend.java:1871] 7d4c91ed04f27bc4:d32f782600000000] Analyzing query: insert into test_fallback_from_dict_if_no_bloom_tbl_props_a60c835b.fallback_from_dict values (0),(2),(4),(6),(8),(10),(12),(14),(16),(18),(20),(22),(24),(26),(28),(30),(32),(34),(36),(38),(40),(42),(44),(46),(48),(50),(52),(54),(56),(58),(60)... ... I0404 14:25:18.025733 19115 jni-util.cc:286] 7d4c91ed04f27bc4:d32f782600000000] java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68) at java.lang.StringBuilder.<init>(StringBuilder.java:89) at org.apache.impala.analysis.SelectListItem.toSql(SelectListItem.java:84) at org.apache.impala.analysis.SelectStmt.toSql(SelectStmt.java:1235) at org.apache.impala.analysis.StatementBase.toSql(StatementBase.java:138) at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:308) at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:269) at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:262) at org.apache.impala.analysis.SetOperationStmt$SetOperand.analyze(SetOperationStmt.java:102) at org.apache.impala.analysis.SetOperationStmt.analyzeOperands(SetOperationStmt.java:388) at org.apache.impala.analysis.SetOperationStmt.analyze(SetOperationStmt.java:318) at org.apache.impala.analysis.UnionStmt.analyze(UnionStmt.java:49) at org.apache.impala.analysis.InsertStmt.analyze(InsertStmt.java:306) at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:506) at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:468) at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2012) at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1920) at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1744) at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
I saw this twice in another ubuntu-16.04-dockerised-tests job: