[IMPALA-12419] TestIcebergTable.test_migrated_table_field_id_resolution fails in S3 build

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 4.3.0
• Epic Color: ghx-label-10
• Assignee: Unassigned
• Reporter: Wenzhe Zhou
• Created: 04/Sep/23 19:24
• Updated: 08/Feb/24 12:11

Description

TestIcebergTable.test_migrated_table_field_id_resolution fails in S3 build

query_test/test_iceberg.py:246: in test_migrated_table_field_id_resolution
    "iceberg_migrated_alter_test", "parquet")
common/file_utils.py:58: in create_iceberg_table_from_directory
    check_call(['hdfs', 'dfs', '-rm', '-f', '-r', hdfs_dir])
/data/jenkins/workspace/impala-cdw-master-core-s3/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/subprocess.py:190: in check_call
    raise CalledProcessError(retcode, cmd)
E   CalledProcessError: Command '['hdfs', 'dfs', '-rm', '-f', '-r', '/test-warehouse/iceberg_migrated_alter_test']' returned non-zero exit status 1

Standard Error

SET client_identifier=query_test/test_iceberg.py::TestIcebergTable::()::test_migrated_table_field_id_resolution[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single;
SET sync_ddl=False;
-- executing against localhost:21000

DROP DATABASE IF EXISTS `test_migrated_table_field_id_resolution_eb4581e8` CASCADE;

-- 2023-09-04 03:37:39,538 INFO     MainThread: Started query 4149ca931eb6d16c:97d680a800000000
SET client_identifier=query_test/test_iceberg.py::TestIcebergTable::()::test_migrated_table_field_id_resolution[protocol:beeswax|exec_option:{'test_replan':1;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single;
SET sync_ddl=False;
-- executing against localhost:21000

CREATE DATABASE `test_migrated_table_field_id_resolution_eb4581e8`;

-- 2023-09-04 03:37:45,054 INFO     MainThread: Started query 3d4215586e7766ad:333cb5bd00000000
-- 2023-09-04 03:37:45,356 INFO     MainThread: Created database "test_migrated_table_field_id_resolution_eb4581e8" for test ID "query_test/test_iceberg.py::TestIcebergTable::()::test_migrated_table_field_id_resolution[protocol: beeswax | exec_option: {'test_replan': 1, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]"
Picked up JAVA_TOOL_OPTIONS:  -javaagent:/data/jenkins/workspace/impala-cdw-master-core-s3/repos/Impala/fe/target/dependency/jamm-0.4.0.jar
23/09/04 03:37:46 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
23/09/04 03:37:46 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
23/09/04 03:37:46 INFO impl.MetricsSystemImpl: s3a-file-system metrics system started
23/09/04 03:37:46 INFO Configuration.deprecation: No unit for fs.s3a.connection.request.timeout(0) assuming SECONDS
23/09/04 03:37:48 INFO impl.MetricsSystemImpl: Stopping s3a-file-system metrics system...
23/09/04 03:37:48 INFO impl.MetricsSystemImpl: s3a-file-system metrics system stopped.
23/09/04 03:37:48 INFO impl.MetricsSystemImpl: s3a-file-system metrics system shutdown complete.
23/09/04 03:37:48 WARN fs.FileSystem: Failed to initialize fileystem s3a://impala-test-uswest2-2: java.nio.file.AccessDeniedException: impala-test-uswest2-2: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))
rm: impala-test-uswest2-2: org.apache.hadoop.fs.s3a.auth.NoAuthWithAWSException: No AWS Credentials provided by TemporaryAWSCredentialsProvider SimpleAWSCredentialsProvider EnvironmentVariableCredentialsProvider IAMInstanceCredentialsProvider : com.amazonaws.SdkClientException: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY))

Issue Links

• depends upon: HADOOP-18945 S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS