Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11313

Segmented Block Reports

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.6.2
    • Fix Version/s: None
    • Component/s: datanode, namenode
    • Labels:
      None

      Description

      Block reports from a single DataNode can be currently split into multiple RPCs each reporting a single DataNode storage (disk). The reports are still large since disks are getting bigger. Splitting blockReport RPCs into multiple smaller calls would improve NameNode performance and overall HDFS stability.
      This was discussed in multiple jiras. Here the approach is to let NameNode divide blockID space into segments and then ask DataNodes to report replicas in a particular range of IDs.

        Attachments

        1. SegmentedBlockReports.pdf
          122 kB
          Vinitha Reddy Gankidi

          Issue Links

            Activity

              People

              • Assignee:
                redvine Vinitha Reddy Gankidi
                Reporter:
                shv Konstantin Shvachko
              • Votes:
                0 Vote for this issue
                Watchers:
                29 Start watching this issue

                Dates

                • Created:
                  Updated: