[HADOOP-14204] S3A multipart commit failing, "UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort"

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
• Component/s: fs/s3
• Target Version/s: 2.8.1
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 20/Mar/17 15:46
• Updated: 08/Jun/17 00:13
• Resolved: 21/Mar/17 13:24

Description

Stack trace seen trying to commit a multipart upload, as the EMR code (which takes a List<String> etags is trying to sort that list directly, which it can't do if the list doesn't want to be sorted.

later versions of the SDK clone the list before sorting.

We need to make sure that the list passed in can be sorted.

Issue Links

• blocks: HADOOP-11694 Über-jira: S3a phase II: robustness, scale and performance
• is related to: SPARK-20038 FileFormatWriter.ExecuteWriteTask.releaseResources() implementations to be re-entrant