[HBASE-15867] Move HBase replication tracking from ZooKeeper to HBase

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 2.1.0
• Component/s: Replication
• Flags: Patch
• Assignee: Zheng Hu
• Reporter: Joseph
• Due: 05/Aug/16
• Created: 19/May/16 21:55
• Updated: 17/May/23 02:56
• Resolved: 17/May/23 02:56

Description

Move the WAL file and offset tracking out of ZooKeeper and into an HBase table called hbase:replication.
The largest three new changes will be two classes ReplicationTableBase, TableBasedReplicationQueues, and TableBasedReplicationQueuesClient. As of now ReplicationPeers and HFileRef's tracking will not be implemented. Subtasks have been filed for these two jobs.

Issue Links

• depends upon: HBASE-13773 Replication should not use ZooKeeper at all for coordination
• depends upon: HBASE-16096 Replication keeps accumulating znodes
• is a parent of: HBASE-27109 Move replication queue storage from zookeeper to a separated HBase table
• is a parent of: HBASE-27110 Move replication peer storage from zookeeper to other storage systems
• is blocked by: HBASE-26233 The region replication framework should not be built upon the general replication framework
• is broken by: HBASE-18070 Enable memstore replication for meta replica
• is duplicated by: HBASE-14379 Replication V2
• is related to: HBASE-10295 Refactor the replication implementation to eliminate permanent zk node
• links to: Design Doc

Sub-Tasks

• Refactor ReplicationQueues.isThisOurZnode() to ReplicationQueues.isThisOurRegionServer: Resolved
• Adding WAL files and tracking offsets in HBase.: Resolved
• Add Javadocs for ReplicationQueuesHBaseImpl: Resolved
• Modify queue naming standards to speed up ReplicationQueuesHBaseImpl: Resolved
• Implement ClaimQueues on top of HBase: Resolved
• Create a ReplicationQueuesClientHBaseImpl: Resolved
• Fix ReplicationTableBase initialization to make it non-blocking: Resolved
• Fix flakey tests: Resolved
• Fix table based replication related configs: Resolved
• Make TableBasedReplicationQueuesImpl check that queue exists before attempting to remove it: Resolved
• Add in UI description of Replication Table: Resolved
• Figure out retry limit and timing for replication queue table operations: Resolved
• Cannot open regions after non-graceful shutdown due to deadlock with Replication Table: Resolved
• Backport replication tracking via system table to Branch-1: Resolved
• Implement HFile ref's tracking (bulk loading) in ReplicationQueuesHBaseImpl and ReplicationQueuesClientHBaseImpl: Resolved
• Removing peers seem to be leaving spare queues: Resolved
• Move Replication Peers into an HBase table too: Resolved
• Metrics for overall replication backlog length: Resolved
• Race Condition inside of TestReplicationStateHBase Unit Tests.: Resolved
• Removing peers seems to be leaving spare queues: Resolved
• Rename the KeeperException to ReplicationException in ReplicationQueuesClient for abstracting: Resolved
• Use nonce and procedure v2 for add/remove replication peer: Resolved
• Add table based replication peers/queues storage back: Resolved
• Make sure the RS/Master can works fine when using table based replication storage layer: Resolved
• Make the API with multi-operations in TableReplicationQueueStorage to be atomic: Resolved
• Consider rolling update from zookeeper based replication storage to table based replication storage: Resolved
• Don't split the region in hbase:replication to ensure that multi-mutations in a batch from replication storage to be atomic: Resolved
• Introduce a global configuration to decide whether we use table or zk based replication storage: Resolved
• Let all replication related UTs can work fine on both ZK based storage and Table based storage: Resolved
• Implement a master based ReplicationTracker: Resolved
• Introduce a new API to sync the live region server list more effectively: Resolved