Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
3.0.0-alpha2
-
None
-
Reviewed
Description
We hit a hang bug when testing S3A with parallel renames.
I narrowed this down to the newer AWS Java SDK. It only happens under load, and appears to be a failure to wake up a waiting thread on timeout/error.
I've created a github issue here:
https://github.com/aws/aws-sdk-java/issues/1102
I can post a Hadoop scale test which reliably reproduces this after some cleanup. I have posted an SDK-only test here which reproduces the issue without Hadoop:
https://github.com/ajfabbri/awstest
I have a support ticket open and am working with Amazon on this bug so I'll take this issue.
Attachments
Attachments
Issue Links
- breaks
-
HADOOP-14589 LocalDynamoDB missing from latest AWS SDK releases
- Resolved
- is depended upon by
-
HADOOP-9991 Fix up Hadoop POMs, roll up JARs to latest versions
- Open
-
HADOOP-13204 Über-jira: S3a phase III: scale and tuning
- Resolved
- is duplicated by
-
HADOOP-14596 AWS SDK 1.11+ aborts() on close() if > 0 bytes in stream; logs error
- Resolved
- is related to
-
HADOOP-16055 Upgrade AWS SDK to 1.11.271 in branch-2
- Resolved