[IMPALA-8183] TestRPCTimeout.test_reportexecstatus_retry times out

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 3.2.0
• Fix Version/s: Impala 3.2.0
• Component/s: Distributed Exec
• Epic Color: ghx-label-7
• Assignee: Thomas Tauber-Marshall
• Reporter: Andrew Sherman
• Created: 11/Feb/19 17:50
• Updated: 14/Feb/19 05:29
• Resolved: 13/Feb/19 03:39

Description

There are 2 forms of failure, where the test itself times out, and where the whole test run times out, suspiciously just after running test_reportexecstatus_retry

Error Message
Failed: Timeout >7200s
Stacktrace
custom_cluster/test_rpc_timeout.py:143: in test_reportexecstatus_retry
self.execute_query_verify_metrics(self.TEST_QUERY, None, 10)
custom_cluster/test_rpc_timeout.py:45: in execute_query_verify_metrics
self.execute_query(query, query_options)
common/impala_test_suite.py:601: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:632: in execute_query
return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:699: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:174: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:183: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:360: in __execute_query
self.wait_for_finished(handle)
beeswax/impala_beeswax.py:384: in wait_for_finished
time.sleep(0.05)
E Failed: Timeout >7200s

Test run timed out. This probably happened due to a hung thread which can be confirmed by looking at the stacktrace of running impalad processes at /data/jenkins/workspace/xxx/repos/Impala/logs/timeout_stacktrace