[KUDU-987] Follower tablets with transactions in REPLICATING state cannot be tombstoned

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Public beta
• Component/s: tablet
• Target Version/s: Public beta
• Code Review: http://gerrit.sjc.cloudera.com:8080/#/c/7474/
• Assignee: Todd Lipcon
• Reporter: Mike Percy
• Created: 13/Aug/15 02:11
• Updated: 29/Feb/16 09:31
• Resolved: 18/Aug/15 20:47

Description

I am hitting an issue where tablets with transactions in REPLICATING state, where the leader cannot finish replicating due to cluster availability or leader outage, gets stuck in a tablet flush call at shutdown time. This is because when the maintenance manager shuts down it waits for outstanding ops to complete, and if one of those ops is a flush that never completes, it hangs.

We think the solution is to shut down consensus before shutting down the MM and having consensus abort the in-flights back to committed at shutdown time.

Issue Links

• is duplicated by: KUDU-1014 Flush gets stuck forever on a QUIESCING tablet
• is related to: KUDU-689 scanners awaiting safe time block handlers and block shutdown
• is related to: KUDU-887 Test and stabilize DeleteTable