Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14186 blockreport storm slow down namenode restart seriously in large cluster
  3. HDFS-14554

Avoid to process duplicate blockreport of datanode when namenode in startup safemode

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: namenode
    • Labels:
      None

      Description

      When NameNode restart and enter stage of startup safemode, load of NameNode could be very high since blockreport request storm from datanodes together. Sometimes datanode may request blockreport times because NameNode doesn't return in time and DataNode meet timeoutexception then retry, Thus it sharpens load of NameNode.
      So we should check and filter duplicate blockreport request from one datannode and reduce load of Namenode.

        Attachments

        1. HDFS-14554.001.patch
          3 kB
          He Xiaoqiao

          Activity

            People

            • Assignee:
              hexiaoqiao He Xiaoqiao
              Reporter:
              hexiaoqiao He Xiaoqiao
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: