[HADOOP-18927] S3ARetryHandler to treat SocketExceptions as connectivity failures

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.6
• Fix Version/s: 3.4.0
• Component/s: fs/s3
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 10/Oct/23 09:46
• Updated: 27/Jan/24 06:48
• Resolved: 12/Oct/23 16:49

Description

i've got a v1 sdk stack trace where a TCP connection reset is breaking a large upload. that should be recoverable with retries.

com.amazonaws.SdkClientException: Unable to execute HTTP request: Connection reset by peer: Unable to execute HTTP request: Connection reset by peer at...

proposed:

• S3ARetryPolicy to map SocketException to connectivity failure
• See if we can create a test for this, ideally under the aws sdk.

I'm now unsure about how well we handle these io problems...a quick experiment with the 3.3.5 release shows that the retry policy retries on whatever exception chain has an unknown host for the endpoint.

Issue Links

• Is contained by: HADOOP-18889 S3A: V2 SDK client does not work with third-party store