[HBASE-15064] BufferUnderflowException after last Cell fetched from an HFile Block served from L2 offheap cache

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: io
• Hadoop Flags: Reviewed
• Assignee: Anoop Sam John
• Reporter: deepankar
• Created: 02/Jan/16 00:26
• Updated: 17/Jun/22 05:16
• Resolved: 23/Mar/16 09:34

Description

While running the newer patches on our production system, I saw this error come couple of times

ipc.RpcServer: Unexpected throwable object 
2016-01-01 16:42:56,090 ERROR [B.defaultRpcServer.handler=20,queue=20,port=60020] ipc.RpcServer: Unexpected throwable object 
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:500)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:249)
at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:494)
at org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decode(FastDiffDeltaEncoder.java:402) 
at org.apache.hadoop.hbase.io.encoding.FastDiffDeltaEncoder$1.decodeNext(FastDiffDeltaEncoder.java:517) 
at org.apache.hadoop.hbase.io.encoding.BufferedDataBlockEncoder$BufferedEncodedSeeker.next(BufferedDataBlockEncoder.java:815)
at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:138)

Looking at the get code

if (this.curItem.remaining() == 0) {
      if (items.length - 1 == this.curItemIndex) {
        // means cur item is the last one and we wont be able to read a long. Throw exception
        throw new BufferUnderflowException();
      }
      this.curItemIndex++;
      this.curItem = this.items[this.curItemIndex];
    }
return this.curItem.get();

Can the new currentItem have zero elements (position == limit), does it make sense to change the if to while ? while (this.curItem.remaining() == 0). This logic is repeated may make sense abstract to a new function if we plan to change to if to while

Issue Links

• is related to: HBASE-11425 Cell/DBB end-to-end on the read-path