Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14186 blockreport storm slow down namenode restart seriously in large cluster
  3. HDFS-14554

Avoid to process duplicate blockreport of datanode when namenode in startup safemode

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Implemented
    • None
    • None
    • namenode
    • None

    Description

      When NameNode restart and enter stage of startup safemode, load of NameNode could be very high since blockreport request storm from datanodes together. Sometimes datanode may request blockreport times because NameNode doesn't return in time and DataNode meet timeoutexception then retry, Thus it sharpens load of NameNode.
      So we should check and filter duplicate blockreport request from one datannode and reduce load of Namenode.

      Attachments

        1. HDFS-14554.001.patch
          3 kB
          Xiaoqiao He

        Activity

          People

            hexiaoqiao Xiaoqiao He
            hexiaoqiao Xiaoqiao He
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: