[IMPALA-7070] Failed test: query_test.test_nested_types.TestParquetArrayEncodings.test_thrift_array_of_arrays on S3

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 3.0
• Fix Version/s: Impala 3.3.0
• Component/s: Backend
• Labels: broken-build flaky s3 test-failure
• Target Version: Impala 3.3.0
• Epic Color: ghx-label-6
• Assignee: Unassigned
• Reporter: Dimitris Tsirogiannis
• Created: 24/May/18 17:04
• Updated: 18/Aug/19 20:58
• Resolved: 18/Aug/19 20:58

Description

 

Error Message

query_test/test_nested_types.py:406: in test_thrift_array_of_arrays     "col1 array<array<int>>") query_test/test_nested_types.py:579: in _create_test_table     check_call(["hadoop", "fs", "-put", local_path, location], shell=False) /usr/lib64/python2.6/subprocess.py:505: in check_call     raise CalledProcessError(retcode, cmd) E   CalledProcessError: Command '['hadoop', 'fs', '-put', '/data/jenkins/workspace/impala-asf-2.x-core-s3/repos/Impala/testdata/parquet_nested_types_encodings/bad-thrift.parquet', 's3a://impala-cdh5-s3-test/test-warehouse/test_thrift_array_of_arrays_11da5fde.db/ThriftArrayOfArrays']' returned non-zero exit status 1

Stacktrace

query_test/test_nested_types.py:406: in test_thrift_array_of_arrays
    "col1 array<array<int>>")
query_test/test_nested_types.py:579: in _create_test_table
    check_call(["hadoop", "fs", "-put", local_path, location], shell=False)
/usr/lib64/python2.6/subprocess.py:505: in check_call
    raise CalledProcessError(retcode, cmd)
E   CalledProcessError: Command '['hadoop', 'fs', '-put', '/data/jenkins/workspace/impala-asf-2.x-core-s3/repos/Impala/testdata/parquet_nested_types_encodings/bad-thrift.parquet', 's3a://impala-cdh5-s3-test/test-warehouse/test_thrift_array_of_arrays_11da5fde.db/ThriftArrayOfArrays']' returned non-zero exit status 1

Standard Error

SET sync_ddl=False;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_thrift_array_of_arrays_11da5fde` CASCADE;

SET sync_ddl=False;
-- executing against localhost:21000
CREATE DATABASE `test_thrift_array_of_arrays_11da5fde`;

MainThread: Created database "test_thrift_array_of_arrays_11da5fde" for test ID "query_test/test_nested_types.py::TestParquetArrayEncodings::()::test_thrift_array_of_arrays[exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]"
-- executing against localhost:21000
create table test_thrift_array_of_arrays_11da5fde.ThriftArrayOfArrays (col1 array<array<int>>) stored as parquet location 's3a://impala-cdh5-s3-test/test-warehouse/test_thrift_array_of_arrays_11da5fde.db/ThriftArrayOfArrays';

18/05/20 18:31:03 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
18/05/20 18:31:03 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
18/05/20 18:31:03 INFO impl.MetricsSystemImpl: s3a-file-system metrics system started
18/05/20 18:31:06 INFO Configuration.deprecation: fs.s3a.server-side-encryption-key is deprecated. Instead, use fs.s3a.server-side-encryption.key
put: rename `s3a://impala-cdh5-s3-test/test-warehouse/test_thrift_array_of_arrays_11da5fde.db/ThriftArrayOfArrays/bad-thrift.parquet._COPYING_' to `s3a://impala-cdh5-s3-test/test-warehouse/test_thrift_array_of_arrays_11da5fde.db/ThriftArrayOfArrays/bad-thrift.parquet': Input/output error
18/05/20 18:31:08 INFO impl.MetricsSystemImpl: Stopping s3a-file-system metrics system...
18/05/20 18:31:08 INFO impl.MetricsSystemImpl: s3a-file-system metrics system stopped.
18/05/20 18:31:08 INFO impl.MetricsSystemImpl: s3a-file-system metrics system shutdown complete.

Issue Links

• is blocked by: IMPALA-6910 Multiple tests failing on S3 build: error reading from HDFS file