[HADOOP-13130] s3a failures can surface as RTEs, not IOEs

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.2
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: fs/s3
• Target Version/s: 2.8.0
• Hadoop Flags: Reviewed
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 11/May/16 10:57
• Updated: 30/Aug/16 01:15
• Resolved: 21/May/16 15:41

Description

S3A failures happening in the AWS library surface as AmazonClientException derivatives, rather than IOEs. As the amazon exceptions are runtime exceptions, any code which catches IOEs for error handling breaks.

The fix will be to catch and wrap. The hard thing will be to wrap it with meaningful exceptions rather than a generic IOE. Furthermore, if anyone has been catching AWS exceptions, they are going to be disappointed. That means that fixing this situation could be considered "incompatible" —but only for code which contains assumptions about the underlying FS and the exceptions they raise.

Issue Links

• is a clone of: HADOOP-13165 S3AFileSystem toString can throw NPE when cannedACL is not set
• is related to: HADOOP-13154 S3AFileSystem printAmazonServiceException/printAmazonClientException appear copy & paste of AWS examples