Details
Description
It seems we can run into a problem while a rolling upgrade with this.
The Balancer is upgraded after NameNodes, so once NN is upgraded it will expect minBlockSize parameter in getBlocks(). The Balancer cannot send it yet, so NN will use the default, which you set to 0. So NN will start unexpectedly sending small blocks to the Balancer. So we should
- either change the default in protobuf to 10 MB
- or treat minBlockSize == 0 in NameNodeRpcServer as a signal to use the configuration variable DFSConfigKeys.DFS_BALANCER_GETBLOCKS_MIN_BLOCK_SIZE_KEY.
If you agree, we should create a follow up jira. I wanted to backport this down the chain of branches, but this upgrade scenario is stopping me.
shv commented this in HDFS-13222 jira.
Attachments
Attachments
Issue Links
- is required by
-
HDFS-13222 Update getBlocks method to take minBlockSize in RPC calls
-
- Resolved
-