[FLINK-18676] Update version of aws to support use of default constructor of "WebIdentityTokenCredentialsProvider"

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.12.0
• Component/s: FileSystems
• Labels: pull-request-available
• Assignee: Robert Metzger
• Reporter: Ravi Bhushan Ratnakar
• Created: 22/Jul/20 17:43
• Updated: 23/Jul/21 12:08
• Resolved: 27/Oct/20 10:46

Description

Background:

I am using Flink 1.11.0 on kubernetes platform. To give access of aws services to taskmanager/jobmanager, we are using "IAM Roles for Service Accounts" . I have configured below property in flink-conf.yaml to use credential provider.

fs.s3a.aws.credentials.provider: com.amazonaws.auth.WebIdentityTokenCredentialsProvider

 

Issue:

When taskmanager/jobmanager is starting up, during this it complains that "WebIdentityTokenCredentialsProvider" doesn't have "public constructor" and container doesn't come up.

 

Solution:

Currently the above credential's class is being used from "flink-s3-fs-hadoop" which gets "aws-java-sdk-core" dependency from "flink-s3-fs-base". In "flink-s3-fs-base",  version of aws is 1.11.754 . The support of default constructor for "WebIdentityTokenCredentialsProvider" is provided from aws version 1.11.788 and onward.

Issue Links

• is related to: FLINK-23487 IRSA doesn't work with S3
• links to: GitHub Pull Request #13208
• links to: GitHub Pull Request #13758