Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12049

Recommissioning live nodes stalls the NN

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: 2.6.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      A node refresh will recommission included nodes that are alive and in decommissioning or decommissioned state. The recommission will scan all blocks on the node, find over replicated blocks, chose an excess, queue an invalidate.

      The process is expensive and worsened by overhead of storage types (even when not in use). It can be especially devastating because the write lock is held for the entire node refresh. Recommissioning 67 nodes with ~500k blocks/node stalled rpc services for over 4 mins.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                daryn Daryn Sharp
              • Votes:
                0 Vote for this issue
                Watchers:
                23 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: