[HADOOP-18945] S3A: IAMInstanceCredentialsProvider failing: Failed to load credentials from IMDS

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0, 3.3.7-aws
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 18/Oct/23 10:31
• Updated: 20/May/24 10:05
• Resolved: 23/Oct/23 13:25

Description

Failures in impala test VMs using iAM for auth

Failed to open file as a parquet file: java.net.SocketTimeoutException: re-open s3a://impala-test-uswest2-1/test-warehouse/test_pre_gregorian_date_parquet_2e80ae30.db/hive2_pre_gregorian.parquet at 84 on s3a://impala-test-uswest2-1/test-warehouse/test_pre_gregorian_date_parquet_2e80ae30.db/hive2_pre_gregorian.parquet: org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException: +: Failed to load credentials from IMDS

Issue Links

• is depended upon by: IMPALA-12419 TestIcebergTable.test_migrated_table_field_id_resolution fails in S3 build
• is related to: HADOOP-19181 S3A) IAMCredentialsProvider throttling results in AWS auth failures
• links to: GitHub Pull Request #6202
• links to: Issue on sdk v2 - Getting java.net.SocketTimeout Error when loading Token from IMDSv2