Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13356

Balancer:Set default value of minBlockSize to 10mb

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

       It seems we can run into a problem while a rolling upgrade with this.
      The Balancer is upgraded after NameNodes, so once NN is upgraded it will expect minBlockSize parameter in getBlocks(). The Balancer cannot send it yet, so NN will use the default, which you set to 0. So NN will start unexpectedly sending small blocks to the Balancer. So we should

      1. either change the default in protobuf to 10 MB
      2. or treat minBlockSize == 0 in NameNodeRpcServer as a signal to use the configuration variable DFSConfigKeys.DFS_BALANCER_GETBLOCKS_MIN_BLOCK_SIZE_KEY.

      If you agree, we should create a follow up jira. I wanted to backport this down the chain of branches, but this upgrade scenario is stopping me.

      Konstantin Shvachko commented this in  HDFS-13222 jira.

      https://issues.apache.org/jira/browse/HDFS-13222?focusedCommentId=16414855&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16414855

        Attachments

        1. HDFS-13356.00.patch
          0.9 kB
          Bharat Viswanadham
        2. HDFS-13356.01.patch
          0.9 kB
          Bharat Viswanadham
        3. HDFS-13356.02.patch
          0.9 kB
          Bharat Viswanadham

          Issue Links

            Activity

              People

              • Assignee:
                bharatviswa Bharat Viswanadham
                Reporter:
                bharatviswa Bharat Viswanadham
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: