Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14186 blockreport storm slow down namenode restart seriously in large cluster
  3. HDFS-14554

Avoid to process duplicate blockreport of datanode when namenode in startup safemode

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: namenode
    • Labels:
      None

      Description

      When NameNode restart and enter stage of startup safemode, load of NameNode could be very high since blockreport request storm from datanodes together. Sometimes datanode may request blockreport times because NameNode doesn't return in time and DataNode meet timeoutexception then retry, Thus it sharpens load of NameNode.
      So we should check and filter duplicate blockreport request from one datannode and reduce load of Namenode.

        Attachments

        1. HDFS-14554.001.patch
          3 kB
          Xiaoqiao He

          Activity

            People

            • Assignee:
              hexiaoqiao Xiaoqiao He
              Reporter:
              hexiaoqiao Xiaoqiao He
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: