[HBASE-16144] Replication queue's lock will live forever if RS acquiring the lock has died prematurely

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.4.0, 1.2.2, 0.98.20, 1.1.6, 2.0.0
• Fix Version/s: 1.3.0, 1.1.6, 1.2.3, 2.0.0
• Hadoop Flags: Reviewed
• Release Note: If zk based replication queue is used and useMulti is false, we will schedule a chore to clean up the orphan replication queue lock on zk.
• Assignee: Phil Yang
• Reporter: Phil Yang
• Created: 29/Jun/16 06:55
• Updated: 09/Nov/17 01:17
• Resolved: 18/Jul/16 02:02

Description

In default, we will use multi operation when we claimQueues from ZK. But if we set hbase.zookeeper.useMulti=false, we will add a lock first, then copy nodes, finally clean old queue and the lock.

However, if the RS acquiring the lock crash before claimQueues done, the lock will always be there and other RS can never claim the queue.