[HADOOP-19045] HADOOP-19045. S3A: CreateSession Timeout after 10 seconds

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0, 3.5.0, 3.4.1
• Component/s: fs/s3
• Labels: pull-request-available
• Target Version/s: 3.5.0
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 18/Jan/24 18:30
• Updated: 11/Feb/24 01:10
• Resolved: 07/Feb/24 15:23

Description

s3a client timeout settings are getting down to http client, but not sdk timeouts, so you can't have a longer timeout than the default. This surfaces in the inability to tune the timeouts for CreateSession calls even now the latest SDK does pick it up

The default value of fs.s3a.connection.request.timeout is now 60s; if an S3 store takes longer than this to return then the operation will be reported by the SDK as a timeout.

When cherrypicking there are two patches

• change constants.java and add test for passdown
• remove core-default and test/core-site values of 0 and update docs. Without this (or a config override) the 10s timeout is maintained

Issue Links

• contains: HADOOP-19022 S3A : ITestS3AConfiguration#testRequestTimeout failure
• links to: GitHub Pull Request #6470
• links to: GitHub Pull Request #6532