[HBASE-24753] HA masters based on raft

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: master
• Assignee: Unassigned
• Reporter: Duo Zhang
• Created: 22/Jul/20 02:44
• Updated: 02/Oct/20 11:19

Description

For better availability, for moving bootstrap information from zookeeper to our own service so finally we could remove the dependency on zookeeper completely.

This has been in my mind for a long time, and since the there is a dicussion in HBASE-11288 about how to storing root table, and also in HBASE-24749, we want to have better performance on a filesystem can not support list and rename well, where requires a storage engine at the bottom to store the storefiles information for meta table, I think it is the time to throw this idea out.

The basic solution is to build a raft group to store the bootstrap information, for now it is cluster id(it is on the file system already?) and the root table. For region servers they will always go to the leader to ask for the information so they can always see the newest data, and for client, we enable 'follower read', to reduce the load of the leader(and there are some solutions to even let 'follower read' to always get the newest data in raft).

With this solution in place, as long as root table will not be in a format of region(we could just use rocksdb to store it locally), the cyclic dependency in HBASE-24749 has also been solved, as we do not need to find a place to store the storefiles information for root table any more.

Issue Links

• is related to: HBASE-24950 Another solution for Splittable Meta
• relates to: HBASE-10296 Replace ZK with a consensus lib(paxos,zab or raft) running within master processes to provide better master failover performance and state consistency