Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1054

Remove unnecessary sleep after failure in nextBlockOutputStream

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.3, 0.20-append, 0.21.0, 0.22.0
    • Fix Version/s: 0.20-append, 0.20.205.0, 0.21.0
    • Component/s: hdfs-client
    • Labels:
      None

      Description

      If DFSOutputStream fails to create a pipeline, it currently sleeps 6 seconds before retrying. I don't see a great reason to wait at all, much less 6 seconds (especially now that HDFS-630 ensures that a retry won't go back to the bad node). We should at least make it configurable, and perhaps something like backoff makes some sense.

      1. HDFS-1054.20-security.1.patch
        1 kB
        Jitendra Nath Pandey
      2. hdfs-1054.txt
        2 kB
        Todd Lipcon
      3. hdfs-1054.txt
        2 kB
        Todd Lipcon
      4. hdfs-1054-0.20-append.txt
        1 kB
        Todd Lipcon

        Activity

        Todd Lipcon created issue -
        Todd Lipcon made changes -
        Field Original Value New Value
        Attachment hdfs-1054.txt [ 12440729 ]
        Todd Lipcon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Todd Lipcon made changes -
        Summary Make sleep after failure in nextBlockOutputStream smarter and configurable Remove unnecessary sleep after failure in nextBlockOutputStream
        Jakob Homan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Todd Lipcon made changes -
        Attachment hdfs-1054.txt [ 12442793 ]
        Todd Lipcon made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jakob Homan made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Todd Lipcon made changes -
        Fix Version/s 0.20-append [ 12315103 ]
        Todd Lipcon made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Todd Lipcon made changes -
        Attachment hdfs-1054-0.20-append.txt [ 12446919 ]
        dhruba borthakur made changes -
        Affects Version/s 0.20-append [ 12315103 ]
        Affects Version/s 0.20.3 [ 12314814 ]
        Affects Version/s 0.21.0 [ 12314046 ]
        dhruba borthakur made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Fix Version/s 0.21.0 [ 12314046 ]
        Fix Version/s 0.20-append [ 12315103 ]
        Resolution Fixed [ 1 ]
        Todd Lipcon made changes -
        Fix Version/s 0.20-append [ 12315103 ]
        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Suresh Srinivas made changes -
        Fix Version/s 0.20.205.0 [ 12316392 ]
        Suresh Srinivas made changes -
        Fix Version/s 0.20.205.0 [ 12316392 ]
        Jitendra Nath Pandey made changes -
        Attachment HDFS-1054.20-security.1.patch [ 12492802 ]
        Suresh Srinivas made changes -
        Fix Version/s 0.20.205.0 [ 12316392 ]

          People

          • Assignee:
            Todd Lipcon
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development