[HADOOP-15642] Update aws-sdk version to 1.11.375

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: build, fs/s3
• Target Version/s: 3.2.0
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 30/Jul/18 21:12
• Updated: 14/Apr/21 11:56
• Resolved: 16/Aug/18 17:02

Description

Move to a later version of the AWS SDK library for a different set of features and issues.

proposed version: 1.11.375

One thing which doesn't work on the one we ship with is the ability to create assumed role sessions >1h, as there's a check in the client lib for role-duration <= 3600 seconds. I'll assume more recent SDKs delegate duration checks to the far end.

see: https://aws.amazon.com/about-aws/whats-new/2018/03/longer-role-sessions/

• assuming later versions will extend assumed role life, docs will need changing,
• Adding a test in HADOOP-15583 which expects an error message if you ask for a duration of 3h; this should act as the test to see what happens.
• think this time would be good to explicitly write down the SDK update process

Issue Links

• causes: HADOOP-15781 S3A assumed role tests failing due to changed error text in AWS exceptions
• is depended upon by: HADOOP-9991 Fix up Hadoop POMs, roll up JARs to latest versions
• is related to: HADOOP-15583 Stabilize S3A Assumed Role support
• is superceded by: HADOOP-16117 Update AWS SDK to 1.11.563
• relates to: HADOOP-15040 Upgrade AWS SDK to 1.11.271: NPE in 1.11.199 bug spams logs w/ Yarn Log Aggregation