[HBASE-16302] age of last shipped op and age of last applied op should be histograms

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0, 2.0.0
• Component/s: Replication
• Hadoop Flags: Reviewed
• Assignee: Ashu Pachauri
• Reporter: Ashu Pachauri
• Created: 28/Jul/16 22:41
• Updated: 10/Mar/17 04:57
• Resolved: 29/Nov/16 08:26

Description

Replication exports metric ageOfLastShippedOp as an indication of how much replication is lagging. But, with multiwal enabled, it's not representative because replication could be lagging for a long time for one wal group (something wrong with a particular region) while being fine for others. The ageOfLastShippedOp becomes a useless metric for alerting in such a case.

Also, since there is no mapping between individual replication sources and replication sinks, the age of last applied op can be a highly spiky metric if only certain replication sources are lagging.

We should use histograms for these metrics and use maximum value of this histogram to report replication lag when building stats.

Issue Links

• relates to: HBASE-17579 Backport HBASE-16302 to 1.3.1