[HBASE-20548] Master fails to startup on large clusters, refreshing block distribution

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.4
• Fix Version/s: 3.0.0-alpha-1, 2.1.0, 1.3.3, 2.0.1, 1.4.5
• Hadoop Flags: Reviewed
• Assignee: Thiruvel Thirumoolan
• Reporter: Thiruvel Thirumoolan
• Created: 09/May/18 01:48
• Updated: 01/Feb/19 20:23
• Resolved: 24/May/18 23:26

Description

On our large clusters with, master has failed to startup within specified time and aborted itself since it was initializing HDFS block distribution. Enable table also takes time for larger tables for the same reason. My proposal is to refresh HDFS block distribution at the end of master initialization and not at retainAssignment()'s createCluster(). This would address HBASE-16570's intention, but avoid the problems we ran into.

cc aoxiang tedyu

Issue Links

• is related to: HBASE-20545 Improve performance of BaseLoadBalancer.retainAssignment
• links to: Review Board (branch-1.4)
• links to: Review Board (branch-2.0)
• links to: Review Board (master)