[HBASE-24292] A "stuck" master should not idle as active without taking action

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.3.0
• Component/s: master, Region Assignment
• Assignee: Rahul Kumar
• Reporter: Nick Dimiduk
• Created: 30/Apr/20 18:02
• Updated: 17/Mar/21 17:35

Description

The master schedules a SCP for the region server hosting meta. However, due to a misconfiguration, the cluster cannot make progress. After fixing the configuration issue and restarting, the cluster still cannot make progress. After the configured period (15 minuets), the master enters a "holding pattern" where it retains Active master status, but isn't taking any action.

This "brown-out" state is toxic. It should either keep trying to make progress, or it should abort. Staying up and not doing anything is the wrong thing to do.