[KUDU-2287] Add replica metric tracking time since there was a valid leader

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.8.0
• Component/s: ksck, metrics, supportability
• Target Version/s: 1.8.0
• Code Review: http://gerrit.cloudera.org:8080/10076
• Assignee: Attila Bukor
• Reporter: Todd Lipcon
• Created: 08/Feb/18 01:30
• Updated: 02/Jun/18 10:34
• Resolved: 02/Jun/18 09:06

Description

Currently monitoring systems can report that the Kudu cluster is perfectly healthy when in fact some tablet has gotten "stuck" with no leader (eg due to some network connectivity problem or a bug). If we exposed a numeric metric on a tablet indicating the time since a replica was healthy, or number of failed election attempts, etc, we could easily monitor for this case.

Issue Links

• is related to: KUDU-2288 Client should fail fast upon access to an unavailable tablet