Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5946

Webhdfs DN choosing code is flawed

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0, 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: namenode, webhdfs
    • Labels:
      None

      Description

      HDFS-5891 improved the performance of redirecting webhdfs clients to a DN. Instead of attempting a connection with a 1-minute timeout, the NN skips decommissioned nodes.

      The logic appears flawed. It finds the index of the first decommissioned node, if any, then:

      • Throws an exception if index = 0, even if other nodes later in the list are not decommissioned.
      • Else picks a random node prior to the index. Let's say there are 10 replicas, 2nd location is decommissioned. All clients will be redirected to the first location even though there are 8 other valid locations.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                daryn Daryn Sharp
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated: