Hadoop Common
  1. Hadoop Common
  2. HADOOP-997

Implement S3 retry mechanism for failed block transfers

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.12.0
    • Component/s: fs
    • Labels:
      None

      Description

      HADOOP-882 improves S3FileSystem so that when certain communications problems with S3 occur the operation is retried. However, the retry mechanism cannot handle a block transfer failure, since blocks may be very large and we don't want to buffer them in memory. This improvement is to write a wrapper (using java.lang.reflect.Proxy if possible - see discussion in HADOOP-882) that can retry block transfers.

      1. HADOOP-997-v3.patch
        46 kB
        Tom White
      2. HADOOP-997-v2.patch
        45 kB
        Tom White
      3. HADOOP-997.patch
        39 kB
        Tom White

        Issue Links

          Activity

            People

            • Assignee:
              Tom White
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development