[HBASE-22663] The HeapAllocationRatio in WebUI is not accurate because all of the heap allocation will happen in another separated allocator named HEAP

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.3.0
• Hadoop Flags: Reviewed
• Assignee: Zheng Hu
• Reporter: Zheng Hu
• Created: 08/Jul/19 03:50
• Updated: 01/Jul/22 21:45
• Resolved: 20/Jul/19 15:31

Description

We use the following method to calculate the heap allocation ratio:

public double getHeapAllocationRatio() {
    long heapAllocBytes = heapAllocationBytes.sum(), poolAllocBytes = poolAllocationBytes.sum();
    double heapDelta = heapAllocBytes - lastHeapAllocationBytes;
    double poolDelta = poolAllocBytes - lastPoolAllocationBytes;
    lastHeapAllocationBytes = heapAllocBytes;
    lastPoolAllocationBytes = poolAllocBytes;
    if (Math.abs(heapDelta + poolDelta) < 1e-3) {
      return 0.0;
    }
    return heapDelta / (heapDelta + poolDelta);
  }

While we also defined a static allocator instance, named HEAP:

  // The on-heap allocator is mostly used for testing, but also some non-test usage, such as
  // scanning snapshot, we won't have an RpcServer to initialize the allocator, so just use the
  // default heap allocator, it will just allocate ByteBuffers from heap but wrapped by an ByteBuff.
  public static final ByteBuffAllocator HEAP = ByteBuffAllocator.createOnHeap();

Almost all of the heap allocation happen in the HEAP, which means the heap allocation ratio caculated from the rpc ByteBuffAllocator will be not accurate.

Will prepare a patch to address this minor bug.

Issue Links

• links to: GitHub Pull Request #365