Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1147

Reduce NN startup time by reducing the processing time of block reports

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • namenode
    • None

    Description

      The NameNode restart times are impacted to a large extent by the processing time of block reports. For a cluster with 150 millions blocks, the block report processing in the NN can take upto 20 minutes or so. The NN is open for business only after it has processed the block reports from most of the datanodes. If we can reduce the processing time of a block report, that will directly reduce the restart time of the NN.

      Attachments

        Issue Links

          duplicates

          Sub-task - The sub-task of the issue HDFS-1295 Improve namenode restart times by short-circuiting the first block reports from datanodes

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              dhruba Dhruba Borthakur
              dhruba Dhruba Borthakur
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: