[HBASE-17902] Backport HBASE-16367 "Race between master and region server initialization may lead to premature server abort" to 1.3

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.2
• Hadoop Flags: Reviewed
• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 11/Apr/17 15:35
• Updated: 01/Aug/18 06:23
• Resolved: 17/Apr/17 16:38

Description

This is to fix the case where hbase master always dies shortly after start.

It turned out that master initialization thread was racing with HRegionServer#preRegistrationInitialization() (initializeZooKeeper, actually) since HMaster extends HRegionServer.
Through additional logging in master:

    this.oldLogDir = createInitialFileSystemLayout();
    HFileSystem.addLocationsOrderInterceptor(conf);
    LOG.info("creating splitLogManager");

I found that execution didn't reach the last log line before region server declared cluster Id being null.

branch-1.3 has been in quiet mode leading up to the release of 1.3.1
Once 1.3.1 is released, the fix can go into branch-1.3

Issue Links

• relates to: HBASE-16367 Race between master and region server initialization may lead to premature server abort