[HADOOP-19027] S3A: S3AInputStream doesn't recover from HTTP/channel exceptions

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0, 3.5.0, 3.4.1
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 05/Jan/24 17:57
• Updated: 12/Feb/24 08:31
• Resolved: 16/Jan/24 14:14

Description

S3AInputStream doesn't seem to recover from Http exceptions raised through HttpClient or through OpenSSL.

• review the recovery code to make sure it is retrying enough, it looks suspiciously like it doesn't
• detect the relevant openssl, shaded httpclient and unshaded httpclient exceptions, map to a standard one and treat as comms error in our retry policy

This is not the same as the load balancer/proxy returning 443/444 which we map to AWSNoResponseException. We can't reuse that as it expects to be created from an software.amazon.awssdk.awscore.exception.AwsServiceException exception with the relevant fields...changing it could potentially be incompatible.

Issue Links

• relates to: HADOOP-15541 AWS SDK can mistake stream timeouts for EOF and throw SdkClientExceptions
• relates to: HADOOP-17764 S3AInputStream read does not re-open the input stream on the second read retry attempt
• links to: GitHub Pull Request #6425