[HBASE-21254] Need to find a way to limit the number of proc wal files

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.1.1, 2.0.3
• Component/s: proc-v2
• Hadoop Flags: Reviewed
• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 29/Sep/18 02:14
• Updated: 24/Oct/18 03:15
• Resolved: 12/Oct/18 03:53

Description

For regionserver, we have a max wal file limitation, if we reach the limitation, we will trigger a flush on specific regions so that we can delete old wal files. But for proc wals, we do not have this mechanism, and it will be worse after HBASE-21233, as if there is an old procedure which can not make progress and do not persist its state, we need to keep the old proc wal file for ever...

Issue Links

• relates to: HBASE-21323 Should not skip force updating for a sub procedure even if it has been finished
• supercedes: HBASE-21235 Rename the closed procedure wal files so that we do not need to call recoverLease when restarting
• links to: Review Board