[HBASE-21162] Revert suspicious change to BoundedByteBufferPool and disable use of direct buffers for IPC reservoir by default

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.4.7
• Fix Version/s: 1.4.8
• Assignee: Andrew Kyle Purtell
• Reporter: Andrew Kyle Purtell
• Created: 06/Sep/18 16:30
• Updated: 01/Feb/19 20:12
• Resolved: 11/Sep/18 01:26

Description

We had a production incident where we traced the issue to a direct buffer leak. On a hunch we tried setting hbase.ipc.server.reservoir.enabled = false and after that no native memory leak could be observed in any regionserver process under the triggering load.

On HBASE-19239 (Fix findbugs and error-prone issues) I made a change to BoundedByteBufferPool that is suspicious given this finding. It was committed to branch-1.4 and branch-1. I'm going to revert this change.

In addition the allocation of direct memory for the server RPC reservoir is a bit problematic in that tracing native memory or direct buffer leaks to a particular class or compilation unit is difficult, so I also propose allocating the reservoir on the heap by default instead. Should there be a leak it is much easier to do an analysis of a heap dump with familiar tools to find it.