Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9011

Support splitting BlockReport of a storage into multiple RPC

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently if a DataNode has too many blocks (more than 1m by default), it sends multiple RPC to the NameNode for the block report, each RPC contains report for a single storage. However, in practice we've seen sometimes even a single storage can contains large amount of blocks and the report even exceeds the max RPC data length. It may be helpful to support sending multiple RPC for the block report of a storage.

        Attachments

        1. HDFS-9011.002.patch
          50 kB
          Jing Zhao
        2. HDFS-9011.001.patch
          48 kB
          Jing Zhao
        3. HDFS-9011.000.patch
          34 kB
          Jing Zhao

          Issue Links

            Activity

              People

              • Assignee:
                jingzhao Jing Zhao
                Reporter:
                jingzhao Jing Zhao
              • Votes:
                0 Vote for this issue
                Watchers:
                27 Start watching this issue

                Dates

                • Created:
                  Updated: