[IMPALA-6588] test_compute_stats_tablesample failing with "Cancelled"

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: Impala 3.0, Impala 2.12.0
• Component/s: Backend
• Labels: broken-build
• Target Version: Impala 2.12.0
• Epic Color: ghx-label-3
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 25/Feb/18 19:58
• Updated: 03/May/18 15:28
• Resolved: 28/Feb/18 16:11

Description

https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/1417

03:24:03  TestStatsExtrapolation.test_compute_stats_tablesample[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: text/none] 
03:24:03 [gw9] linux2 -- Python 2.7.12 /home/ubuntu/Impala/bin/../infra/python/env/bin/python
03:24:03 metadata/test_stats_extrapolation.py:108: in test_compute_stats_tablesample
03:24:03     self.__run_sampling_test(no_column_tbl, columns, part_test_tbl_base, 10, 7)
03:24:03 metadata/test_stats_extrapolation.py:131: in __run_sampling_test
03:24:03     .format(tbl, cols, perc, seed))
03:24:03 common/impala_connection.py:160: in execute
03:24:03     return self.__beeswax_client.execute(sql_stmt, user=user)
03:24:03 beeswax/impala_beeswax.py:173: in execute
03:24:03     handle = self.__execute_query(query_string.strip(), user=user)
03:24:03 beeswax/impala_beeswax.py:341: in __execute_query
03:24:03     self.wait_for_completion(handle)
03:24:03 beeswax/impala_beeswax.py:361: in wait_for_completion
03:24:03     raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
03:24:03 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
03:24:03 E    Query aborted:Cancelled
03:24:03 ---------------------------- Captured stderr setup -----------------------------
03:24:03 -- connecting to: localhost:21000
03:24:03 SET sync_ddl=False;
03:24:03 -- executing against localhost:21000
03:24:03 DROP DATABASE IF EXISTS `test_compute_stats_tablesample_4f26fb11` CASCADE;
03:24:03 
03:24:03 SET sync_ddl=False;
03:24:03 -- executing against localhost:21000
03:24:03 CREATE DATABASE `test_compute_stats_tablesample_4f26fb11`;
03:24:03 
03:24:03 MainThread: Created database "test_compute_stats_tablesample_4f26fb11" for test ID "metadata/test_stats_extrapolation.py::TestStatsExtrapolation::()::test_compute_stats_tablesample[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 5000, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: text/none]"
03:24:03 ----------------------------- Captured stderr call -----------------------------
03:24:03 -- executing against localhost:21000
03:24:03 set compute_stats_min_sample_size=0;
03:24:03 
03:24:03 -- executing against localhost:21000
03:24:03 use functional;
03:24:03 
03:24:03 -- executing against localhost:21000
03:24:03 describe formatted functional.alltypes;
03:24:03 
03:24:03 -- executing against localhost:21000
03:24:03 create external table test_compute_stats_tablesample_4f26fb11.alltypes like functional.alltypes location 'hdfs://localhost:20500/test-warehouse/alltypes';
03:24:03 
03:24:03 -- executing against localhost:21000
03:24:03 alter table test_compute_stats_tablesample_4f26fb11.alltypes recover partitions;
03:24:03 
03:24:03 -- executing against localhost:21000
03:24:03 compute stats test_compute_stats_tablesample_4f26fb11.no_columns() tablesample system (10) repeatable (7);
03:24:03 
03:24:03 ======= 1 failed, 1817 passed, 64 skipped, 41 xfailed in 2456.05 seconds =======

This was probably introduced by one of my DiskIoMgr patches - I fixed failure with a similar symptoms earlier, which was caused by a race between scanner threads and the main thread issuing ranges, leading to an DiskIoMgr method returning CANCELLED. However, I'm not sure that a similar race is possible in the mt_dop > 1 scan node.

Issue Links

• is broken by: IMPALA-4835 HDFS scans should operate with a constrained number of I/O buffers