[IMPALA-6592] Fix test gap - no test of handling for invalid/unsupported Parquet codec

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.12.0
• Fix Version/s: Impala 3.0, Impala 2.12.0
• Component/s: Backend
• Labels: crash
• Epic Color: ghx-label-7
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 27/Feb/18 03:39
• Updated: 08/Mar/18 17:55
• Resolved: 08/Mar/18 17:55

Description

I wasn't able to get a full stacktrace or core, but the frame where it crashed was:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000001d953e0, pid=5401, tid=140122118891264
#
# JRE version: Java(TM) SE Runtime Environment (8.0_05-b13) (build 1.8.0_05-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.5-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [impalad+0x19953e0]  impala::BaseScalarColumnReader::Reset(impala::HdfsFileDesc const&, parquet::ColumnChunk const&, int)+0x2d0
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:

This was an ASAN build. The parquet fuzz test was running, so it's the likely culprit:

10:28:05 ==================================== ERRORS ====================================
10:28:05  ERROR at teardown of TestScannersFuzzing.test_fuzz_alltypes[exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format: parquet/none] 
10:28:05 [gw0] linux2 -- Python 2.6.6 /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/bin/../infra/python/env/bin/python
10:28:05 conftest.py:285: in cleanup
10:28:05     request.instance.execute_query_expect_success(request.instance.client, "use default")
10:28:05 common/impala_test_suite.py:520: in wrapper
10:28:05     return function(*args, **kwargs)
10:28:05 common/impala_test_suite.py:527: in execute_query_expect_success
10:28:05     result = cls.__execute_query(impalad_client, query, query_options)
10:28:05 common/impala_test_suite.py:612: in __execute_query
10:28:05     return impalad_client.execute(query, user=user)
10:28:05 common/impala_connection.py:160: in execute
10:28:05     return self.__beeswax_client.execute(sql_stmt, user=user)
10:28:05 beeswax/impala_beeswax.py:173: in execute
10:28:05     handle = self.__execute_query(query_string.strip(), user=user)
10:28:05 beeswax/impala_beeswax.py:339: in __execute_query
10:28:05     handle = self.execute_query_async(query_string, user=user)
10:28:05 beeswax/impala_beeswax.py:335: in execute_query_async
10:28:05     return self.__do_rpc(lambda: self.imp_service.query(query,))
10:28:05 beeswax/impala_beeswax.py:456: in __do_rpc
10:28:05     raise ImpalaBeeswaxException("Not connected", None)
10:28:05 E   ImpalaBeeswaxException: ImpalaBeeswaxException:
10:28:05 E    Not connected
10:28:05 ---------------------------- Captured stderr setup -----------------------------
10:28:05 SET sync_ddl=False;
10:28:05 -- executing against localhost:21000
10:28:05 DROP DATABASE IF EXISTS `test_fuzz_alltypes_d2734b63` CASCADE;
10:28:05 
10:28:05 SET sync_ddl=False;
10:28:05 -- executing against localhost:21000
10:28:05 CREATE DATABASE `test_fuzz_alltypes_d2734b63`;
10:28:05 
10:28:05 MainThread: Created database "test_fuzz_alltypes_d2734b63" for test ID "query_test/test_scanners_fuzz.py::TestScannersFuzzing::()::test_fuzz_alltypes[exec_option: {'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format: parquet/none]"
10:28:05 ----------------------------- Captured stderr call -----------------------------
10:28:05 MainThread: Using random seed 1519669602
10:28:05 -- executing against localhost:21000
10:28:05 create table test_fuzz_alltypes_d2734b63.alltypes like functional_parquet.alltypes;

Issue Links

• is broken by: IMPALA-4835 HDFS scans should operate with a constrained number of I/O buffers
• is depended upon by: IMPALA-4835 HDFS scans should operate with a constrained number of I/O buffers