Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11313

Segmented Block Reports

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.6.2
    • None
    • datanode, namenode
    • None

    Description

      Block reports from a single DataNode can be currently split into multiple RPCs each reporting a single DataNode storage (disk). The reports are still large since disks are getting bigger. Splitting blockReport RPCs into multiple smaller calls would improve NameNode performance and overall HDFS stability.
      This was discussed in multiple jiras. Here the approach is to let NameNode divide blockID space into segments and then ask DataNodes to report replicas in a particular range of IDs.

      Attachments

        1. SegmentedBlockReports.pdf
          122 kB
          Vinitha Reddy Gankidi

        Issue Links

          Activity

            People

              redvine Vinitha Reddy Gankidi
              shv Konstantin Shvachko
              Votes:
              0 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

                Created:
                Updated: