[HBASE-17673] Monitored RPC Handler not shown in the WebUI

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 1.2.4, 1.1.8, 2.0.0
• Fix Version/s: 1.4.0, 2.0.0
• Hadoop Flags: Reviewed
• Assignee: Allan Yang
• Reporter: Allan Yang
• Created: 21/Feb/17 04:10
• Updated: 28/Feb/17 19:51
• Resolved: 28/Feb/17 15:45

Description

This issue has been fixed once in HBASE-14674. But, I noticed that almost all RS in our production environment still have this problem. Strange thing is that newly started servers seems do not affected. Digging for a while, then I realize the CircularFifoBuffer introduced by HBASE-10312 is the root cause. The RPC hander's monitoredTask only create once, if the server is flooded with tasks, RPC monitoredTask can be purged by CircularFifoBuffer, and then never visible in WebUI.
So my solution is creating a list for RPC monitoredTask separately. It is OK to do so since the RPC handlers remain in a fixed number. It won't increase or decrease during the lifetime of the server.

Issue Links

• is broken by: HBASE-10312 Flooding the cluster with administrative actions leads to collapse
• relates to: HBASE-17256 Rpc handler monitoring will be removed when the task queue is full
• relates to: HBASE-14674 Rpc handler / task monitoring seems to be broken after 0.98