[FLINK-23487] IRSA doesn't work with S3

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13.1
• Fix Version/s: 1.14.0
• Component/s: FileSystems
• Labels: pull-request-available
• Assignee: Ingo Bürk
• Reporter: Ingo Bürk
• Created: 23/Jul/21 12:07
• Updated: 14/Jul/22 05:44
• Resolved: 06/Aug/21 05:25

Description

Using IRSA (IAM Role for Service Account) currently doesn't work with s3 (both presto and hadoop) and fails with Access Denied. This has been brought up previously in FLINK-18676, but seems to be broken still or again.

We have tested a patch which updates the version for AWS and presto-hive in Flink, and doing so we successfully verified that it works with s3-presto. However, this didn't yet fix s3-hadoop. The patch will be posted in the comments.

Another curious finding was FLINK-17859 which states that presto-hive had been updated. However, this seems to be the case neither in the posted Flink 1.8 link nor current master.

Issue Links

• causes: FLINK-23635 PrestoS3FileSystemITCase.testConfigKeysForwarding
• relates to: FLINK-17859 [flink-s3-fs-presto] Upgrade the dependent presto-hive jar to latest version
• relates to: FLINK-18676 Update version of aws to support use of default constructor of "WebIdentityTokenCredentialsProvider"
• links to: GitHub Pull Request #16592
• links to: GitHub Pull Request #16717